Senior Data Engineer
Share
Job Location:
Monthly Salary:
Experience Required:
Posted on:
4 days ago
Vacancies:
1 Vacancy
Job Summary
- Design develop and maintain enterprise-scale data engineering solutions on Microsoft Azure.
- Build and optimize data pipelines using Azure Databricks and Azure Data Factory.
- Develop and maintain PySpark and Python-based data processing frameworks.
- Implement and manage data governance and metadata solutions using Azure Purview Collibra and Atlan.
- Perform data modelling to support analytical reporting and governance use cases.
- Consume and integrate external and internal APIs to ingest structured and semi-structured data.
- Ensure data quality lineage cataloging and compliance across data platforms.
- 10 years of overall experience in Data Engineering and Data Platform development.
- Strong hands-on expertise in Microsoft Azure data services.
- Proven experience with Azure Databricks PySpark and Python for large-scale data processing.
- Solid understanding of data concepts data modelling and data architecture.
- Practical experience with data governance tools such as Azure Purview Collibra and Atlan.
- Hands-on experience in API consumption and data integration.
- Architect and implement scalable secure and governed data solutions on Azure.
- Build reusable and maintainable PySpark-based data pipelines.
- Translate business and governance requirements into effective data models.
- Integrate multiple data sources using APIs and cloud-native tools.
- Ensure end-to-end data lineage cataloging and compliance.
- A digital-first cloud-focused environment leveraging Microsoft Azure.
- Opportunities to work on complex large-scale data engineering initiatives.
- A culture that values Agile ways of working and continuous improvement.
- Exposure to advanced data governance platforms and modern data architectures.
As a Senior Data Engineer you will: Design develop and maintain enterprise-scale data engineering solutions on Microsoft Azure. Build and optimize data pipelines using Azure Databricks and Azure Data Factory. Develop and maintain PySpark and Python-based data processing frameworks. Implement and manage data governance and metadata solutions using Azure Purview Collibra and Atlan. Perform data modelling to support analytical reporting and governance use cases. Consume and integrate external and internal APIs to ingest structured and semi-structured data. Ensure data quality lineage cataloging and compliance across data platforms. What You Bring to the Table: 10 years of overall experience in Data Engineering and Data Platform development. Strong hands-on expertise in Microsoft Azure data services. Proven experience with Azure Databricks PySpark and Python for large-scale data processing. Solid understanding of data concepts data modelling and data architecture. Practical experience with data governance tools such as Azure Purview Collibra and Atlan. Hands-on experience in API consumption and data integration. You Should Possess the Ability to: Architect and implement scalable secure and governed data solutions on Azure. Build reusable and maintainable PySpark-based data pipelines. Translate business and governance requirements into effective data models. Integrate multiple data sources using APIs and cloud-native tools. Ensure end-to-end data lineage cataloging and compliance. What We Bring to the Table: A digital-first cloud-focused environment leveraging Microsoft Azure. Opportunities to work on complex large-scale data engineering initiatives. A culture that values Agile ways of working and continuous improvement. Exposure to advanced data governance platforms and modern data architectures. Lets Connect Want to discuss this opportunity in more detail Feel free to reach out. Recruiter: Aswin Dhanvandhar Phone:; Extn :141 Email: LinkedIn:
Company Industry
IT Services and IT Consulting
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
