Senior Data Engineer

We are seeking an experienced Senior Data Engineer with 810 years of experience in designing developing and managing modern data platforms data pipelines and cloud-based analytics solutions. The ideal candidate will have strong expertise in Azure Data Services large-scale data processing data warehousing ETL/ELT frameworks and cloud-native data architectures. The role requires hands-on experience in building scalable secure and high-performance data solutions that support enterprise analytics reporting AI and business intelligence initiatives.

• Design develop and maintain scalable data platforms and data pipelines on Microsoft Azure.
• Build and optimize batch and real-time data ingestion frameworks from multiple structured and unstructured data sources.
• Design and implement data lake data warehouse and lakehouse architectures to support analytics and reporting workloads.
• Develop and manage ETL/ELT processes using modern cloud-native data engineering practices.
• Implement data transformation cleansing validation and quality frameworks to ensure data accuracy and reliability.
• Collaborate with business stakeholders data analysts data scientists and application teams to understand data requirements and deliver scalable solutions.
• Optimize data storage processing and query performance across enterprise data platforms.
• Implement security governance monitoring and compliance best practices across Azure environments.
• Support integration of data platforms with AI/ML business intelligence and enterprise applications.
• Participate in architecture reviews code reviews troubleshooting and technical mentoring activities.
• Ensure high availability scalability and operational excellence of data platforms and pipelines.

• 810 years of experience in Data Engineering Data Warehousing and Enterprise Data Platform development.
• Strong hands-on experience with Microsoft Azure Data Services.
• Expertise in Azure Data Factory (ADF) Azure Synapse Analytics Azure Data Lake Storage (ADLS Gen2) and Azure SQL Database.
• Experience building and managing large-scale ETL/ELT pipelines and data integration solutions.
• Strong proficiency in SQL query optimization and database performance tuning.
• Hands-on experience with PySpark Apache Spark and distributed data processing frameworks.
• Strong programming skills in Python Scala or Java.
• Experience with dimensional modeling data warehousing concepts and modern lakehouse architectures.
• Experience working with structured semi-structured and unstructured data.
• Strong understanding of data governance data quality metadata management and security best practices.
• Experience with REST APIs data integration patterns and enterprise system connectivity.
• Hands-on experience with Git CI/CD pipelines and DevOps practices.
• Strong analytical problem-solving and communication skills.

• Experience with Microsoft Fabric OneLake Dataflows and Fabric Data Engineering workloads.
• Experience with Databricks Delta Lake and lakehouse implementations.
• Knowledge of real-time streaming technologies such as Azure Event Hubs Apache Kafka or Azure Stream Analytics.
• Experience supporting AI/ML and advanced analytics workloads through enterprise data platforms.
• Familiarity with Power BI datasets semantic models and enterprise reporting architectures.
• Experience with data governance tools such as Microsoft Purview.
• Microsoft Azure Data Engineering certifications are highly preferred.
• Experience working in Agile/Scrum environments.

• Experience with Microsoft Fabric Data Engineering and Analytics solutions.
• Exposure to MLOps and DataOps practices.
• Knowledge of containerization technologies such as Docker and Kubernetes.
• Experience with Infrastructure as Code (Terraform ARM Templates or Bicep).
• Familiarity with Snowflake BigQuery or other cloud data warehouse platforms.
• Experience with enterprise-scale data migration and modernization projects.
• Understanding of Generative AI Vector Databases and data platforms supporting AI workloads.