Azure Data Engineer

Location: Hamilton New Jersey (Local candidates preferred for onsite/F2F interview)

Job Summary

We are looking for an experienced Azure Data Engineer with strong expertise in Azure data platforms ETL/ELT development big data processing and data warehousing. The ideal candidate should have experience in the Banking Financial Services and Insurance (BFSI) domain and be comfortable working with distributed teams across design development and testing phases.

Required Technical Skills

• Azure Data Factory (ADF)
• Azure Databricks
• Python
• Apache Spark
• SQL Server
• Azure Cosmos DB
• Azure Data Lake
• Data Warehouse Design
• ETL/ELT Development
• Data Integration & Data Engineering

Desired Skills

• Power BI
• Azure DevOps
• Event Hub
• Data Governance
• GDPR/HIPAA Compliance
• Performance Tuning & Cost Optimization

Key Responsibilities

Data Ingestion & Integration

• Design and develop scalable data ingestion pipelines using Azure Data Factory and Event Hub.
• Integrate data from multiple sources including on-premises systems cloud platforms APIs and databases.
• Ensure reliable secure and efficient movement of data across environments.

Data Transformation

• Build ETL/ELT workflows using Databricks Spark and Python.
• Cleanse transform normalize and enrich data for analytics and reporting.
• Implement business rules and data validation processes.

Data Storage & Management

• Design implement and maintain Azure Data Lake and Data Warehouse solutions.
• Optimize storage architecture for scalability performance and cost efficiency.
• Manage structured and unstructured data using SQL Server and Cosmos DB.

Collaboration

• Partner with business stakeholders architects analysts and development teams.
• Ensure data quality availability and consistency for reporting and analytics initiatives.
• Work effectively with multi-location teams.

Security & Compliance

• Implement data governance and security best practices.
• Ensure compliance with BFSI regulations and standards including GDPR and HIPAA requirements.
• Maintain data privacy and access controls.

Performance & Cost Optimization

• Monitor troubleshoot and optimize data pipelines and workloads.
• Improve performance of Databricks jobs and Spark workloads.
• Leverage Azure services for dynamic scaling and cost management.

Required Qualifications

• Strong BFSI domain experience.
• Hands-on experience with Azure Data Factory Databricks Python Spark SQL Server and Cosmos DB.
• Experience designing enterprise-scale data pipelines and data warehouse solutions.
• Excellent communication and stakeholder management skills.
• Experience working with geographically distributed teams.