IT engineer data lakehouse
IT engineer data lakehouse
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Design develop and operate scalable and maintainable data pipelines in the Azure Databricks environment
Develop all technical artefacts as code implemented in professional IDEs with full version control and CI/CD automation
Enable datadriven decisionmaking in Human Resources (HR) Purchasing (PUR) and Finance (FIN) by ensuring high data availability quality and reliability
Implement data products and analytical assets using software engineering principles in close alignment with business domains and functional IT
Apply rigorous software engineering practices such as modular design testdriven development and artifact reuse in all implementations
Global delivery footprint; crossfunctional data engineering support across HR PUR & FIN domains
Collaboration with business stakeholders functional IT partners product owners architects ML/AI engineers and Power BI developers
Agile productteam structure embedded in an enterprisescale Azure environment
Main Tasks:
Design scalable batch and streaming pipelines in Azure Databricks using PySpark and/or Scala
Implement ingestion from structured and semistructured sources (e.g. SAP APIs flat files)
Build bronze/silver/gold data layers following the defined lakehouse layering architecture & governance
Implement usecase driven dimensional models (star/snowflake schema) tailored to HR PUR & FIN needs
Ensure compatibility with reporting tools (e.g. Power BI) via curated data marts and semantic models
Implement enterpriselevel data warehouse models (domaindriven 3NF models) for HR PUR & FIN data closely aligned with data engineers for other business domains
Develop and apply master data management strategies (e.g. Slowly Changing Dimensions)
Develop automated data validation tests using frameworks
Monitor pipeline health identify anomalies and implement quality thresholds
Establish data quality transparency by defining and implementing meaningful data quality rules with source system and business stakeholders and implementing related reports
Develop and structure pipelines using modular reusable code in a professional IDE
Apply testdriven development (TDD) principles with automated unit integration and validation tests
Integrate tests into CI/CD pipelines to enable failfast deployment strategies
Commit all artifacts to version control with peer review and CI/CD integration
Work closely with Product Owners to refine user stories and define acceptance criteria
Translate business requirements into data contracts and technical specifications
Participate in agile events such as sprint planning reviews and retrospectives
Document pipeline logic data contracts and technical decisions in markdown or autogenerated docs from code
Align designs with governance and metadata standards (e.g. Unity Catalog)
Track lineage and audit trails through integrated tooling
Profile and tune data transformation performance
Reduce job execution times and optimize cluster resource usage
Refactor legacy pipelines or inefficient transformations to improve scalability
Qualifications :
Degree in Computer Science Data Engineering Information Systems or related discipline.
Certifications in software development and data engineering (e.g. Databricks DE Associate Azure Data Engineer or relevant DevOps certifications).
36 years of handson experience in data engineering roles in enterprise environments. Demonstrated experience building productiongrade codebases in IDEs with test coverage and version control.
Proven experience in implementing complex data pipelines and contributing to full lifecycle data projects (development to deployment)
Experience in at least one business domain: HR PUR & FIN or a comparable field
Not required; however experience mentoring junior developers or leading implementation workstreams is a plus
Experience working in international teams across multiple time zones and cultures preferably with teams in India Germany and the Philippines.
Additional Information :
The wellbeing of our employees is important to us. Thats why we offer exciting career prospects and support you in achieving a good worklife balance with additional benefits such as:
- Training opportunities
- Mobile and flexible working models
- Sabbaticals
and much more...
Sounds interesting for you Click here to find out more.
Diversity Inclusion & Belonging are important to us and make our company strong and successful. We offer equal opportunities to everyone regardless of age gender nationality cultural background disability religion ideology or sexual orientation.
Ready to drive with Continental Take the first step and fill in the online application.
Remote Work :
No
Employment Type :
Fulltime
Employment Type
Full-time
