Azure Data Engineer
Azure Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Azure Databricks
Role Overview:
We are seeking a highly skilled Azure Databricks Developer with relevant years of experience in Azure Cloud services and handson expertise with Databricks Azure data factoryADB Python and PySpark. The ideal candidate will be responsible for designing implementing and optimizing data processing pipelines and will play a critical role in our data engineering projects.
Key Responsibilities:
Data Engineering:
- Design develop and optimize largescale data processing pipelines using Azure Databricks Azure data factoryetc.
- Implement complex data transformations and processing workflows with PySpark.
- Develop and maintain scalable data architecture and pipelines to support various data projects.
Azure Cloud Services:
- Utilize various Azure services such as Azure Data Factory Azure Data Lake Storage and Azure Synapse Analytics to build endtoend data solutions.
- Monitor troubleshoot and optimize Azure cloud resources and Databricks clusters for performance and costefficiency.
Data Integration:
- Integrate data from multiple sources and ensure data consistency and integrity.
- Develop and implement data validation and quality checks.
Collaboration:
- Work closely with data scientists analysts and other stakeholders to understand data requirements and deliver solutions.
- Participate in agile development processes and contribute to continuous improvement practices.
Documentation and Reporting:
- Document data workflows architecture and processes for knowledge sharing and maintenance.
- Generate reports and visualizations to communicate insights and progress to stakeholders.
Required Skills and Experience:
Technical Skills:
- Strong experience with Azure Databricks Azure data factoryetc and PySpark.
- Proficiency in Python programming with a focus on data processing and manipulation.
- Experience with Azure services such as Azure Data Factory Azure Data Lake Storage and Azure Synapse Analytics.
- Solid understanding of data warehousing concepts and ETL processes.
- Familiarity with CI/CD practices and tools for data pipelines.
Professional Experience:
- Rel years of handson experience in data engineering and cloudbased data processing.
- Proven track record of working with largescale data systems and distributed computing.
- Experience in performance tuning and optimization of data processing jobs.
Soft Skills:
- Excellent problemsolving skills and attention to detail.
- Strong communication skills to collaborate effectively with crossfunctional teams.
- Ability to work independently and manage multiple tasks in a dynamic environment.
Required Experience:
Manager
Employment Type
Full-Time
