Data Engineer

Title: Data Engineer

Type: Contract

Location: Toronto

Environment: Hybrid

We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will have extensive experience in designing building and maintaining scalable data pipelines and platforms using cloudbased technologies particularly within the Microsoft Azure ecosystem. The role will involve collaborating closely with crossfunctional teams to understand data requirements architect solutions and drive the implementation of best practices in data engineering.

Key Responsibilities:

• Design develop and deploy endtoend data pipelines and ETL workflows using Azure Data Factory (ADF) Azure Synapse Analytics and Databricks.
  
• Collaborate with data scientists and analysts to understand data requirements and translate them into scalable and efficient data solutions.
  
• Implement data governance and security best practices to ensure the integrity and confidentiality of sensitive data.
  
• Automate infrastructure provisioning configuration management and deployment processes using Azure DevOps Ansible and HashiCorp products (Terraform Vault Consul etc.).
  
• Optimize data pipelines for performance reliability and costefficiency leveraging advanced techniques such as partitioning caching and parallel processing.
  
• Monitor and troubleshoot production data pipelines ensuring high availability and uptime.
  
• Stay abreast of emerging technologies and best practices in data engineering continuously evaluating and recommending improvements to our data infrastructure.
  

Qualifications:

• Bachelors or Masters degree in Computer Science Engineering or a related field.
  
• 5 years of experience in data engineering with a focus on building and optimizing data pipelines in a cloud environment.
  
• Strong proficiency in Microsoft Azure cloud services including Azure Data Factory Azure Synapse Analytics (formerly SQL Data Warehouse) and Azure Databricks.
  
• Handson experience with infrastructureascode (IaC) tools such as Azure DevOps Ansible or HashiCorp products (Terraform Vault Consul etc.).
  
• Solid understanding of data modeling relational databases and distributed computing principles.
  
• Experience with containerization technologies (Docker Kubernetes) and microservices architecture is a plus.
  
• Excellent problemsolving skills and ability to thrive in a fastpaced collaborative environment.
  
• Strong communication skills and ability to effectively communicate technical concepts to nontechnical stakeholders.