DataOpsCloud Data Engineer – Senior

Job Title: RQ11099 DataOps/Cloud Data Engineer Senior
Client: Health Services Cluster
Ministry: Ministry of Public and Business Service Delivery and Procurement

Location: 5700 Yonge St Toronto Ontario
Work Model: 5 Days Onsite

Security Clearance: CRJMC

Responsibilities:

Design develop test implement and troubleshoot:

data pipelines using various tools such as Databricks Azure Data Factory AWS Glue

complex data transformation procedures

data models for data warehouses and lakehouses

Data engineering Experience

Programming & scripting: Python SQL Linux shell PowerShell.

Data manipulation/analysis using pandas and pyspark.

Working with XLSX CSV JSON files relational databases cloud storage structured and unstructured data

40%

Cloud experience

AWS and/or Azure services.

Cloud data warehouse solutions such as AWS Redshift.

Data lakehouse solutions such as Databricks (Delta Lake).

Data processing orchestration/automation

25%

Data Warehouse

ETL tools (such as Azure Data Factory AWS Glue) as well as cloud agnostic tools such as Informatica IDMC.

Data modeling & architecture: relational and dimensional modeling.

Data reporting/visualization.

25%

Other Skills

Full SDLC from requirements gathering design implementation testing to deployment and production support

Project management (agile/scrum and waterfall)

Change and Incident management

Communication presentation and negotiation skills

Consulting problem-solving and decision-making skills

5%

Public Sector

Experience working in OPS and/or public sector in general

5%

MUST HAVES:

Data engineering (pandas pyspark)

Data lakehouse solutions such as Databricks (Delta Lake).

Working with XLSX CSV JSON files relational databases cloud storage structured and unstructured data

Experience using AWS Services such as Glue StepFunctions Lambda S3

Extract/Transform/Load data using tools such as Informatica IDMC