Senior Data Engineer
Senior Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
- Design develop and maintain scalable and efficient data pipelines using Azure Data Factory Databricks and other relevant tools.
- Collaborate with data architects data scientists and other stakeholders to build robust and flexible data models and architectures.
- Implement endtoend ETL (Extract Transform Load) processes ensuring data is correctly transformed loaded and optimized for downstream analytics.
- Manage large datasets and work with structured semistructured and unstructured data.
- Monitor and optimize data pipeline performance to ensure timely and accurate data delivery.
- Work closely with data teams to ensure data quality consistency and integrity across various systems.
- Build and maintain cloud data solutions using Azure services ensuring scalability security and high availability.
- Provide guidance and mentorship to junior data engineers fostering an environment of learning and growth.
- Troubleshoot and resolve issues related to data integration pipeline performance and system errors.
- Stay up to date with the latest trends and advancements in data engineering and cloud technologies.
Required Qualifications:
- Bachelor’s or Master’s degree in Computer Science Engineering or related field (or equivalent experience).
- Minimum of 5 years of experience in data engineering with a strong focus on cloud platforms and data architecture.
- Strong handson experience with Azure Data Factory for building and managing ETL pipelines.
- Extensive experience with Databricks for big data processing analytics and machine learning workflows.
- Indepth knowledge of data architecture design data modeling and data warehousing concepts.
- Expertise in working with cloud platforms especially Microsoft Azure (Azure Data Lake Azure SQL Azure Synapse Analytics Mongo Atlas etc.).
- Proficient in SQL and scripting languages (Python Scala or similar).
- Familiarity with data governance and security practices within cloud environments.
- Strong problemsolving and debugging skills.
- Excellent communication skills with the ability to collaborate effectively with crossfunctional teams.
Preferred Qualifications:
- Experience with additional cloud data platforms such as AWS or Google Cloud.
- Familiarity with tools like Azure Synapse Analytics Azure Data Lake and Azure SQL Database.
- Knowledge of machine learning frameworks and integrating them into data pipelines.
- Experience with CI/CD for data pipeline automation and deployment.
- Strong understanding of Agile development methodologies.
- Certification in Microsoft Azure (Azure Data Engineer) is a plus. Nice to have skills:
- Familiarity with Large Language Models (LLMs) and their data requirements.
- Experience with data governance frameworks and data security best practices.
- Experience with containerization technologies (e.g. Docker Kubernetes) and serverless architectures.
- Knowledge on Data Security Data Access Management including Data Encryption.
- Experience with RESTful APIs message brokers and eventdriven architectures.
Employment Type
Full Time
