Senior Data Engineer-1

Prudentials purpose is to be partners for every life and protectors for every future. Our purpose encourages everything we do by creating a culture in which diversity is celebrated and inclusion assured for our people customers and partners. We provide a platform for our people to do their best work and make an impact to the business and we support our peoples career ambitions. We pledge to make Prudential a place where you can Connect Grow and Succeed.

Job Description:

Design build test deploy and maintain batch and near-real-time data pipelines on the UDP platform.

Develop data ingestion transformation enrichment reconciliation and publishing processes.

Build and maintain data processing for commissions calculations including rules rates adjustments incentives deductions and calculation outputs.

Implement reusable data models and curated datasets aligned with enterprise and UDP standards.

Ensure data accuracy completeness lineage traceability security and auditability across pipelines and calculation processes.

Implement automated data quality checks controls alerts and exception handling.

Optimize pipeline performance reliability scalability and cost.

Use source control automated testing CI/CD environment controls and deployment standards.

Monitor production pipelines investigate failures and data discrepancies and provide operational support.

Contribute to the enhancement and maintenance of core UDP platform capabilities engineering standards and reusable components.

Who we are looking for:

Strong hands-on experience with Python SQL Databricks Spark and data pipeline development.

Experience designing ETL/ELT solutions data models transformations and workflow orchestration.

Experience implementing complex calculation logic reconciliation and financial or commissions-related data processing.

Good understanding of data quality observability lineage metadata data contracts and access controls.

Experience with Delta Lake Unity Catalog Git automated testing CI/CD and cloud data platforms.

Understanding of APIs file-based integration event processing and secure data exchange.

Good knowledge of Agile delivery software engineering practices and production support.

Strong problem analysis troubleshooting communication and collaboration skills.

Insurance or financial services experience is an advantage.

Working Experience:

At least 5 years of experience in data engineering or software engineering.

Hands-on experience delivering and supporting production data pipelines and enterprise data platforms.

Education:

Bachelor in Computer Science Data Engineering Information Systems Engineering or equivalent.

Language:

Fluent written and spoken English.

Prudential is an equal opportunity employer. We provide equality of opportunity of benefits for all who apply and who perform work for our organisation irrespective of sex race age ethnic origin educational social and cultural background marital status pregnancy and maternity religion or belief disability or part-time / fixed-term work or any other status protected by applicable law. We encourage the same standards from our recruitment and third-party suppliers taking into account the context of grade job and location. We also allow for reasonable adjustments to support people with individual physical or mental health requirements.