DataOpsCloud Data Engineer

Toronto - Canada

Monthly Salary: Not Disclosed

Posted on: 6 hours ago

Vacancies: 1 Vacancy

Job Summary

Assignment: RQ09822 - DataOps/Cloud Data Engineer - Senior

Job Title: DataOps/Cloud Data Engineer

Requisition (SS): RQ09822

Start Date:

Client: Land & Resources Cluster

End Date:

Office Location: 40 St Clair Ave W Toronto

Organization: Land & Resources Cluster

Ministry: Ministry of Public and Business Service Delivery and Procurement

# Business Days: 90.00

Onsite - This position is listed as Onsite unless otherwise instructed by the Client; the resource is expected to work 7.25 hours per calendar day between the standard working hours of 8:00-5:00 PM (excluding lunch breaks) every Monday to Friday inclusive at the identified OPS office location.

Mandatory Skills and Experience (8 years experience):

Design build automate and optimize complex data ETL/ELT processes using Databricks Delta Live Tables (DLT) and equivalent.
Development of unified data platforms using Databricks and Unity catalog or its equivalent.
Relational databases (Oracle MySQL SQL Server) data modelling (relational & dimensional) advanced SQL query optimization data replication administration.
Advanced SQL skills (PL/SQL TSQL).
Big Data processing Frameworks (PySpark).
Proficient in creating Azure Data Factory pipelines for copy activity and custom data pipelines in Databricks with CI CD (Azure DevOPS Git Integration) .
Data migration and data integration across different platforms Oracle to Azure Data Lake Oracle to Databricks Oracle to Microsoft Fabric.

Description

To respond to the priorities of the Government of Ontario through the passage of the Bill 5 Protect Ontario by Unleashing Our Economy Act 2025 the resource may be partially assigned to Bill 5 related projects or initiatives on an as needed basis.

Responsibilities

Designing and developing data pipelines from source to end user
Optimizing data pipelines
General Skills
Experience with Cloud data platforms data management and data exchange tools and technologies
Experience with Commercial and opensource data and database development and management and specializing in data storage setting and managing cloud Data As a Service (DaaS). application Database as a Service (DBaaS) Data Warehouses as a Service (DWaaS) and other storage platforms (both in the cloud and on-premise).
Experience with Data pipeline and workflow development orchestration deployment and automation and specializing in programming and pipelines to create and managing the dataflow and movement of data.
Experience with Cloud data engineer must be familiar and experienced with different programming languages and be able to integrate with many different platforms to create data pipelines automate tasks and write scripts.
Experience in DataOPS principle best practice and implementation and Agile project development and deployment
Experience in Continuous Integration/Continuous Development/Deployment (CI/CD) and Data provisioning Automation
Experience with digital product data analysis data exchange data provisioning and data security
Extensive expert experience in designing/developing and implementing data conversation and migration of VLD (Very large Data) of Online analytical processing (OLAP) and online transaction processing (OLTP) environments to Cloud Software-as-a-service (SaaS) Platform-as-a-service (PaaS) and Infrastructure-as-a-service (IaaS) environments.
Experience in design development and implementation of fact/dimension model data mapping data warehouse data lake and data lakehouse for enterprise
Experience managing Cloud Data services for project delivery including storage repositories Data Lake Data Lakehouse key vault virtual machine disk etc.
Experience with structured semi-structured unstructured data collection ingestion provisioning and exchange technological development of enterprise data warehouse and data lake and data lakehouse solutions and operational support.
Experience with DataOPS performance monitoring and tuning
Excellent analytical problem-solving and decision-making skills; verbal and written communication skills; presentation skills; interpersonal and negotiation skills

Evaluation Criteria - 100%

Mandatory Skills and Experience - 70%
Desirable Skills and Experience - 25%
Education - 3%
Public Sector Experience - 2%

Desirable Skills and Experience:

Data lake design and development with focus on extracting transforming data coming from various data sources and loading into Lakehouse in medallion architecture.
Advanced skills and hands-on experience with Oracle databases Azure SQL Server and Azure Data Factory.
Cloud technologies (Azure Google AWS).
PL/SQL for data extraction transformation and loading initial setup and other ETL experience management and support such as troubleshooting performance tuning failover and recovery.
Scripting languages (Python Unix shell Scala).

Assignment: RQ09822 - DataOps/Cloud Data Engineer - Senior Job Title: DataOps/Cloud Data Engineer Requisition (SS): RQ09822 Start Date: Client: Land & Resources Cluster End Date: Office Location: 40 St Clair Ave W Toronto Organization: Land & Resources Cluster Ministry: Ministry of Public and Busine...