Design develop and maintain scalable and secure data pipelines using Azure Databricks and Azure Data Factory.
Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.
Develop cloud-based data engineering solutions leveraging Azure services.
Write clean reusable and efficient code in Python and PySpark for data transformation and automation.
Implement data integration solutions across multiple platforms and systems.
Design and develop data models using normalization and denormalization techniques.
Manage and optimize Azure storage solutions including:
Azure Data Lake Storage (ADLS)
Azure Blob Storage
Troubleshoot and optimize performance bottlenecks in data pipelines and Spark jobs.
Support data warehousing reporting and BI integration use cases.
Collaborate with analytics and business teams to deliver actionable insights.
Participate in code reviews and ensure adherence to coding standards and Git version control practices.
Contribute to the design and deployment of enterprise-scale data warehouse architectures.
Stay updated with emerging Azure cloud technologies and best practices.
Job Description: Design develop and maintain scalable and secure data pipelines using Azure Databricks and Azure Data Factory. Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets. Develop cloud-based data engineering solutions leveraging Azure servi...
Job Description:
Design develop and maintain scalable and secure data pipelines using Azure Databricks and Azure Data Factory.
Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.
Develop cloud-based data engineering solutions leveraging Azure services.
Write clean reusable and efficient code in Python and PySpark for data transformation and automation.
Implement data integration solutions across multiple platforms and systems.
Design and develop data models using normalization and denormalization techniques.
Manage and optimize Azure storage solutions including:
Azure Data Lake Storage (ADLS)
Azure Blob Storage
Troubleshoot and optimize performance bottlenecks in data pipelines and Spark jobs.
Support data warehousing reporting and BI integration use cases.
Collaborate with analytics and business teams to deliver actionable insights.
Participate in code reviews and ensure adherence to coding standards and Git version control practices.
Contribute to the design and deployment of enterprise-scale data warehouse architectures.
Stay updated with emerging Azure cloud technologies and best practices.