Data Engineer – Azure & Big Data

Toronto - Canada

Monthly Salary: Not Disclosed

Experience Required: 5years

Posted on: 4 hours ago

Vacancies: 1 Vacancy

Job Summary

Data Engineer Azure & Big Data

Role Description:

Designing and developing data pipelines using Cloud Technologies based on business requirements and technical design.

Collaborating with Stakeholders Business Analysts and Data Architects to assist and translate business requirements to technical solutions.

Develop big data and analytic solutions leveraging new or existing technology to advance all lines of business.

Exploratory data analysis by querying and processing data stores and databases and generate reports.

Design upgrade and implement new data workflows.

Write and maintain technical documentation.

Executes updates patches and other activities required to maintain and enhance the operations of on-premise or cloud-based environments.

Essential Skills:

Experience working as Data Engineer with focus on big data processing and/or relational databases with good understanding of SQL.

Experienced with working on Structured and Semi-Structured datasets.

Experience working with Microsoft Azure Data Platform specifically Azure SQL Database Azure Data Factory ADLS storage account Azure Data Bricks and DevOps.

Experienced in creating data pipelines & developing complex and optimized queries.

Experienced with Workflow Management Tools: Airflow Crontab CA Workload Automation.

Experience with implementing CI / CD pipeline on GitHub Jenkins / JGP.

Experience writing complex SQL and NoSQL jobs to analyze data in traditional DBMS (MS-SQL Oracle) environments.

Experience with integrating to back-end/legacy environments.

Experience integrating business and technology teams.

Experienced in any of the following programming/scripting languages (SQL Python Shell Scala).

Excellent organizational and time management skills strong business presence with ability to multi-task and effectively deal with competing priorities.

Data Engineer Azure & Big Data Role Description: Designing and developing data pipelines using Cloud Technologies based on business requirements and technical design. Collaborating with Stakeholders Business Analysts and Data Architects to assist and translate business requirements to...