Senior Data Engineer

Mississauga - Canada

Monthly Salary: Not Disclosed

Posted on: 2 hours ago

Vacancies: 1 Vacancy

Job Summary

Role: Senior Data Engineer

Location: Mississauga ON
Contract

Mandatory Skills: Data Lake Storage Azure Data Factory RDD/DataFrame/SQL

Design develop optimize and maintain scalable data pipelines using Databricks (Spark/Scala) on Azure.
Collaborate with cross-functional teams to deliver robust end-to-end data solutions.
Write efficient SQL queries for ETL processes data validation and reporting.
Optimize performance of complex data processing workflows.
Implement best practices for code quality testing deployment and documentation.
Participate in code reviews and mentor junior engineers.
Work closely with stakeholders to understand business requirements and translate them into technical solutions.
Integrate data solutions with other applications; exposure APIs or React-based UIs is a plus.
Required Skills & Qualifications:
Bachelors or Masters degree in Computer Science or related field.
6 years of hands-on experience in software engineering/data engineering roles.
Strong experience with Databricks platform for big data analytics.
Proficient in Scala programming language; solid understanding of Spark framework (RDD/DataFrame/SQL).
Hands-on experience working on Azure cloud services (e.g. Data Lake Storage Azure Data Factory).
Strong background in Database design & development; excellent SQL skills (T-SQL/PLSQL etc.).
Understanding of CI/CD pipelines for automated deployments on cloud platforms.
Experience working within Agile/Scrum methodologies.
Advanced knowledge of Python for scripting automation and AI/ML development.
Familiarity with Langchain for integrating and orchestrating large language model workflows.
Hands-on experience with Azure OpenAI to leverage generative AI capabilities in applications
Understanding of Vector Databases for efficient storage and retrieval of high-dimensional data embeddings essential for semantic search and recommendation systems.
Knowledge of RAG (Retrieval-Augmented Generation) techniques to combine information retrieval with generative AI models for enhanced application intelligence.

Role: Senior Data Engineer Location: Mississauga ON Contract Mandatory Skills: Data Lake Storage Azure Data Factory RDD/DataFrame/SQL Design develop optimize and maintain scalable data pipelines using Databricks (Spark/Scala) on Azure. Collaborate with cross-functional teams to deliver robus...

Role: Senior Data Engineer

Location: Mississauga ON
Contract

Mandatory Skills: Data Lake Storage Azure Data Factory RDD/DataFrame/SQL

Design develop optimize and maintain scalable data pipelines using Databricks (Spark/Scala) on Azure.
Collaborate with cross-functional teams to deliver robust end-to-end data solutions.
Write efficient SQL queries for ETL processes data validation and reporting.
Optimize performance of complex data processing workflows.
Implement best practices for code quality testing deployment and documentation.
Participate in code reviews and mentor junior engineers.
Work closely with stakeholders to understand business requirements and translate them into technical solutions.
Integrate data solutions with other applications; exposure APIs or React-based UIs is a plus.
Required Skills & Qualifications:
Bachelors or Masters degree in Computer Science or related field.
6 years of hands-on experience in software engineering/data engineering roles.
Strong experience with Databricks platform for big data analytics.
Proficient in Scala programming language; solid understanding of Spark framework (RDD/DataFrame/SQL).
Hands-on experience working on Azure cloud services (e.g. Data Lake Storage Azure Data Factory).
Strong background in Database design & development; excellent SQL skills (T-SQL/PLSQL etc.).
Understanding of CI/CD pipelines for automated deployments on cloud platforms.
Experience working within Agile/Scrum methodologies.
Advanced knowledge of Python for scripting automation and AI/ML development.
Familiarity with Langchain for integrating and orchestrating large language model workflows.
Hands-on experience with Azure OpenAI to leverage generative AI capabilities in applications
Understanding of Vector Databases for efficient storage and retrieval of high-dimensional data embeddings essential for semantic search and recommendation systems.
Knowledge of RAG (Retrieval-Augmented Generation) techniques to combine information retrieval with generative AI models for enhanced application intelligence.