Senior Data Engineer (Azure / PySpark)
Senior Data Engineer (Azure / PySpark)
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Key Responsibilities
Design develop and maintain scalable ETL pipelines using PySpark and Azure Data Services (Data Factory Data Lake Blob Storage Synapse Databricks).
Work closely with clients to gather requirements present solutions and ensure deliverables align with business goals.
Optimize data workflows for performance scalability and cost-efficiency.
Ensure data quality through validation checks monitoring and automated alerts.
Collaborate with cross-functional teams including data scientists BI developers and business analysts.
Create and maintain comprehensive documentation for data pipelines architecture and processes.
Support pre-sales or solutioning discussions by providing technical insights.
Requirements
Required Qualifications
Bachelor s degree in Computer Science Data Science or a related technical field.
5 years of professional experience in data engineering.
Strong hands-on expertise in PySpark for large-scale data processing.
Proficiency in Python for scripting automation and data manipulation.
Proven experience with Azure cloud ecosystem: Data Factory Data Lake Blob Storage Synapse Databricks.
Solid understanding of ETL design patterns data modeling and data warehousing.
Experience interacting directly with clients or business stakeholders to define and deliver data solutions.
Preferred Skills
CI/CD experience with Azure DevOps or similar tools.
Familiarity with Delta Lake data governance and security best practices.
Exposure to performance tuning for Spark jobs and Azure pipelines.
Soft Skills
Excellent communication skills for both technical and non-technical audiences.
Strong problem-solving and analytical abilities.
Ability to work independently and collaboratively in a fast-paced environment.
Benefits
Required Qualifications Bachelor s degree in Computer Science, Data Science, or a related technical field. 5+ years of hands-on experience in data engineering roles. Proven experience with data lake and data warehouse technologies such as: Databricks Apache Spark Azure Data Lake Azure Blob Storage Azure Data Factory ETL Pipelines Spark SQL Strong experience with relational databases (e.g., PostgreSQL, MySQL, Microsoft SQL Server). Solid background in Python for data manipulation and pipeline development. Hands-on experience with the Azure cloud ecosystem and CI/CD practices. Deep understanding of data modeling principles and data architecture best practices. Preferred/Additional Skills (Nice to Have) Familiarity with DevOps tools and practices. Experience working with Linux environments. Exposure to the Hadoop ecosystem. Understanding of Docker containers and their use in data engineering. Experience optimizing and indexing large-scale datasets for high performance. Soft Skills Strong communication skills (verbal and written) in English. High attention to detail and a technical problem-solving mindset. Ability to work collaboratively with clients and cross-functional teams. Customer-oriented approach with a passion for delivering impactful data solutions. Experience mentoring or leading junior team members is a plus.
Employment Type
Full Time
