Job Summary:
We are seeking a highly skilled Lead Azure Databricks Developer to support our data modernization initiative by analyzing existing ETL mappings and workflows from legacy systems (e.g. SSIS) understanding their complexity and designing optimal migration strategies to Azure Databricks. The ideal candidate will also provide expert guidance on data modeling transformation logic and performance optimization in the target cloud environment.
Key Responsibilities:
> Review and analyze source system mapping documents ETL workflows and data flow diagrams.
> Understand business logic transformation rules and data dependencies in existing systems.
> Evaluate complexity and design patterns of legacy ETL processes (e.g. SSIS).
> Recommend and design equivalent or improved solutions in Azure Databricks using PySpark and Delta Lake.
> Provide data modeling recommendations for the target architecture ensuring scalability and performance.
> Collaborate with data architects business analysts and migration teams to define migration strategy and execution plan.
> Develop reusable components and frameworks for data ingestion transformation and validation.
> Document migration approach technical specifications and data lineage.
> Mentor junior developers and ensure adherence to best practices in cloud-native development.
Required Skills & Qualifications:
> 8 years of experience in data engineering including 4 years in Azure Databricks and PySpark.
> Strong understanding of ETL concepts data integration and legacy system analysis.
> Hands-on experience with Azure Data Lake Azure Data Factory Delta Lake and Synapse Analytics.
> Proven ability to interpret and reverse-engineer complex ETL mappings and workflows.
> Expertise in data modeling (dimensional normalized denormalized) and performance tuning.
> Proficiency in SQL and scripting languages.
> Excellent communication and documentation skills.
Preferred Qualifications:
> Experience in SSIS or similar legacy ETL tools.
> Azure certifications (e.g. Azure Data Engineer Associate).
> Familiarity with CI/CD pipelines and DevOps practices.
> Exposure to Agile methodologies and cloud migration programs.
Together as owners lets turn meaningful insights into action.
Life at CGI is rooted in ownership teamwork respect and belonging. Here youll reach your full potential because
You are invited to be an owner from day 1 as we work together to bring our Dream to life. Thats why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our companys strategy and direction.
Your work creates value. Youll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas embrace new opportunities and benefit from expansive industry and technology expertise.
Youll shape your career by joining a company built to grow and last. Youll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.
Come join our teamone of the largest IT and business consulting services firms in the world.
The COMPANY is one of the few end-to-end consulting firms with the scale, reach, capabilities and commitment to meet clients’ enterprise digital transformation needs. Our 77,500 consultants and professionals work side-by-side with clients in 10 industries across more than 400 location ... View more