drjobs Data Engineer

Data Engineer

Employer Active

1 Vacancy
drjobs

Job Alert

You will be updated with latest job alerts via email
Valid email field required
Send jobs
Send me jobs like this
drjobs

Job Alert

You will be updated with latest job alerts via email

Valid email field required
Send jobs
Jobs by Experience drjobs

8years

Job Location drjobs

New York County, NY - USA

Monthly Salary drjobs

Not Disclosed

drjobs

Salary Not Disclosed

Vacancy

1 Vacancy

Job Description

Do you love a career where you Experience Grow & Contribute at the same time while earning at least 10% above the market If so we are excited to have bumped onto you.


We are an IT Solutions Integrator/Consulting Firm helping our clients hire the right professional for an exciting long term project. Here are a few details.



Requirements

We are seeking a highly skilled Data Engineer to join our data engineering team at Prudential PGIM. The successful candidate will play a critical role in building optimizing and maintaining scalable and reliable data pipelines to support enterprise-level data analytics and business intelligence initiatives.


Key Responsibilities:

  • Develop test and maintain robust data pipelines using Python and PySpark.

  • Leverage Microsoft Fabric tools and services to manage data integration ingestion and transformation processes.

  • Ensure high-quality data delivery by implementing rigorous data validation and cleansing routines.

  • Automate workflows to streamline data processing and reduce manual intervention.

  • Collaborate with data analysts data scientists and business stakeholders to understand data needs and deliver scalable solutions.

  • Monitor troubleshoot and optimize pipeline performance and reliability.

  • Document processes data flows and best practices to ensure maintainability and compliance.

  • Participate in Agile/Scrum ceremonies and contribute to sprint planning and retrospectives.


Required Skills & Qualifications:

Skill
Requirement Level
MS Fabric
Hands-on experience with Microsoft Fabric including Dataflows Pipelines and Lakehouses
SQL
Advanced SQL skills for querying transforming and analyzing large datasets
Python
Strong Python programming skills especially in data manipulation and scripting
PySpark Notebooks
Experience with distributed data processing using PySpark in notebook environments
ETL Development
Experience designing and implementing Extract Transform Load (ETL) solutions
Data Quality
Familiarity with data profiling validation and quality assurance techniques
Automation
Proficiency in automation frameworks for data processes and workflows

Preferred Qualifications:

  • Experience with Azure Data Services Databricks or other cloud platforms.

  • Familiarity with Git CI/CD pipelines and DevOps practices in data engineering.

  • Background in financial services or investment management is a strong plus.

  • Knowledge of data governance and compliance standards.


Soft Skills:

  • Strong analytical and problem-solving abilities.

  • Excellent communication and interpersonal skills.

  • Ability to work independently and manage multiple priorities in a fast-paced environment.

  • Collaborative team player with a proactive attitude.



Benefits



Education

B.E/

Employment Type

Full Time

About Company

Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.