Senior Data Engineer
Senior Data Engineer
Posted on :
15-06-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Monthly Salary
Not Disclosed
Salary Not Disclosed
Vacancy
1 Vacancy
Posted on :
15-06-2025
Job Description
P.S: My client Location is 2 blocks from the Metro-North Train Station Stop in White Plains NY. So if the candidates live in NYC they can take the train from Grand Central Station.
Project Overview:
Responsible for managing critical Business-As-Usual (BAU) services that support enterprise-wide data operations. These services include the development maintenance and monitoring of data pipelines integrations and reporting infrastructure that are essential for ongoing business functions.
Responsible for managing critical Business-As-Usual (BAU) services that support enterprise-wide data operations. These services include the development maintenance and monitoring of data pipelines integrations and reporting infrastructure that are essential for ongoing business functions.
Key responsibilities include:
1) Maintaining and troubleshooting ETL workflows (Pentaho Databricks ADF)
2) Supporting daily data loads and ensuring data availability for business reporting
3) Responding to ad-hoc requests from business users
4) Coordinating with DBAs and application teams for incident resolution
5) Performing enhancements to support evolving business data needs these BAU services are essential for keeping business operations running smoothly and delivering timely insights across multiple departments
1) Maintaining and troubleshooting ETL workflows (Pentaho Databricks ADF)
2) Supporting daily data loads and ensuring data availability for business reporting
3) Responding to ad-hoc requests from business users
4) Coordinating with DBAs and application teams for incident resolution
5) Performing enhancements to support evolving business data needs these BAU services are essential for keeping business operations running smoothly and delivering timely insights across multiple departments
Job Functions & Responsibilities
ETL & Data Integration:
Design develop and optimize ETL pipelines using Azure Databricks ADF and Pentaho to support enterprise data workflows. Implement and maintain data movement transformation and integration across multiple systems.
Ensure seamless data exchange between cloud on-prem and hybrid environments.
Work with Globalscape FTP for secure file transfers and automation.
ETL & Data Integration:
Design develop and optimize ETL pipelines using Azure Databricks ADF and Pentaho to support enterprise data workflows. Implement and maintain data movement transformation and integration across multiple systems.
Ensure seamless data exchange between cloud on-prem and hybrid environments.
Work with Globalscape FTP for secure file transfers and automation.
API Development and Integration:
Develop consume and integrate RESTful and SOAP APIs to facilitate data.
Work with API gateways and authentication methods such Oauth JWT certificate and API keys.
Implement and optimize API-based data extractions and real-time data integrations
Develop consume and integrate RESTful and SOAP APIs to facilitate data.
Work with API gateways and authentication methods such Oauth JWT certificate and API keys.
Implement and optimize API-based data extractions and real-time data integrations
Data Quality & Governance:
Implement data validation cleansing and enrichment techniques.
Develop and execute data reconciliation processes to ensure accuracy and completeness.
Adhere to data governance policies and security compliance standards.
Implement data validation cleansing and enrichment techniques.
Develop and execute data reconciliation processes to ensure accuracy and completeness.
Adhere to data governance policies and security compliance standards.
BAU Support & Performance Optimization:
Troubleshoot and resolve ETL failures data load issues and performance bottlenecks**.
Optimize SQL stored procedures and complex queries for better performance.
Support ongoing enhancements and provide operational support for existing data pipelines.
Troubleshoot and resolve ETL failures data load issues and performance bottlenecks**.
Optimize SQL stored procedures and complex queries for better performance.
Support ongoing enhancements and provide operational support for existing data pipelines.
Collaboration & Documentation:
Work closely with Data Analysts Business Analysts and stakeholders to understand data needs.
Document ETL processes data mappings and workflows for maintainability and knowledge sharing.
Provide guidance and best practices to ensure scalability and efficiency of data solutions.
Work closely with Data Analysts Business Analysts and stakeholders to understand data needs.
Document ETL processes data mappings and workflows for maintainability and knowledge sharing.
Provide guidance and best practices to ensure scalability and efficiency of data solutions.
Required Skills & Experience:
7 years of experience in ETL development data integration and SQL scripting.
Strong expertise in Azure Databricks ADF (Azure Data Factory) and Pentaho.
Experience handling secure file transfers using Globalscape FTP.
Hands-on experience in developing and consuming APIs (REST/SOAP).
Experience working with API security protocols (Oauth JWT API Keys etc.).
Proficiency in SQL stored procedures performance tuning and query optimization.
Understanding of data modeling data warehousing and data governance best practices.
Hands-on experience with cloud-based data platforms (Azure/AWS) is a plus.
Strong problem-solving skills troubleshooting abilities and ability to work independently.
Excellent communication skills and ability to work in a fast-paced environment.
7 years of experience in ETL development data integration and SQL scripting.
Strong expertise in Azure Databricks ADF (Azure Data Factory) and Pentaho.
Experience handling secure file transfers using Globalscape FTP.
Hands-on experience in developing and consuming APIs (REST/SOAP).
Experience working with API security protocols (Oauth JWT API Keys etc.).
Proficiency in SQL stored procedures performance tuning and query optimization.
Understanding of data modeling data warehousing and data governance best practices.
Hands-on experience with cloud-based data platforms (Azure/AWS) is a plus.
Strong problem-solving skills troubleshooting abilities and ability to work independently.
Excellent communication skills and ability to work in a fast-paced environment.
Preferred Qualifications:
Experience working in large-scale enterprise data integration projects.
Knowledge of Python PySpark for big data processing.
Familiarity with CI/CD for data pipelines (Azure DevOps GitHub Actions etc.).
Experience working in large-scale enterprise data integration projects.
Knowledge of Python PySpark for big data processing.
Familiarity with CI/CD for data pipelines (Azure DevOps GitHub Actions etc.).
Education & Certifications:
Bachelors or Masters degree in a relevant field like Computer science Data Engineering or related technical field
Bachelors or Masters degree in a relevant field like Computer science Data Engineering or related technical field
Nice to have below certifications:
a) Databricks certified Data Engineer
b) Azure Data Engineer associate
a) Databricks certified Data Engineer
b) Azure Data Engineer associate
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
