Senior Data Engineer
Senior Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Purpose
As a Senior Data Engineer you will lead the design and optimization of scalable data pipelines and modern data architectures to support business objectives. You will drive engineering best practices mentor peers and embed automation for testing and monitoring. Collaborating with cross-functional teams you will translate business needs into reusable cost-effective data products. You will stay abreast of emerging technologies to ensure the team remains at the forefront of the evolving data landscape.
Key Responsibilities
Data Pipeline Architecture and Implementation
Design and implement scalable data pipelines for batch and real-time workloads.
- Build robust efficient and maintainable pipelines using modern frameworks (e.g. Spark Delta Lake dbt Airflow).
Design and Maintenance of Data Lakehouse Solutions
- Architect and maintain data lakehouse and data warehouse solutions.
- Develop well-structured performant data models using dimensional/star schema design (Kimball methodology).
Optimization of Data Infrastructure
- Drive performance tuning of Spark jobs and SQL.
- Monitor and optimize storage and computing costs with workload profiling and KPI dashboards.
Engineering Standards and Mentorship
- Promote best engineering practices and lead code reviews.
- Mentor junior engineers and facilitate knowledge-sharing sessions.
Data Quality Assurance
- Implement automated testing monitoring and validation frameworks.
- Establish data validation layers and integration tests to ensure reliability.
Technology Exploration
- Evaluate emerging technologies (e.g. Iceberg new ingestion tools).
- Recommend and integrate new tools where appropriate.
Cross-Functional Collaboration
- Work closely with BI Data Science and Business teams to align solutions with business KPIs and requirements.
Shared Team Responsibilities
- Develop reusable components naming conventions and documentation standards.
- Conduct regular design reviews and peer code reviews.
- Implement CI/CD pipelines for data jobs.
- Maintain shared knowledge bases and facilitate onboarding and continuous learning.
Key Skills & Experience
Education:
-
Bachelors degree in Computer Science Engineering or similar.
-
(Preferred) Masters degree in a related field.
Experience:
-
5 years in data engineering or related fields.
Technical Skills:
-
Data Factory Python Git Databricks.
-
SQL Server Azure Platform.
-
Power BI (data modeling and DAX development).
-
Linux OS concepts.
-
Data modeling and ETL (Kimball concepts).
Working Environment
-
Office hours multi-tasking environment.
-
Interaction with stakeholders at various levels across teams.
-
Use of modern data engineering tools and cloud services.
Employment Type
Full Time
