Senior Azure Data Engineer

Must haves:

• Extensive experience in gathering requirements and business process knowledge in order to design correct and high-quality data transformation.
• Extensive experience in design development and implementation with Azure Data Factory Databricks
• Experience with CI/CD (DevOps) pipelines and concepts
• Experience with Azure DevOps

Description

Responsibilities

• Design implement and continuously expand data pipelines by performing extraction transformation and loading activities.
• Required to translate technical systems specifications into working tested applications.
• This includes developing detailed programming specifications writing and/or generating code compiling data-driven programs maintaining and conducting unit tests.
• Resolves and troubleshoots technical problems which arise during the use and operation of software packages including technical assistance in implementation conversion and migrations.

General Skills

• Experience in programming and analysis; specialized software package support at the specified experience level
• Ability to collaborate with IT Professionals throughout the Software Development Life Cycle
• Experience in structured methodologies for the development design implementation and maintenance of applications
• Experience in design code test debug and document applications
• Experience in the use of object and/or third generation language development tools
• Experience in one or more programming languages
• Experience in application design latest design patters deployment and troubleshooting
• Experience with relational and hierarchical database technologies
• Experience in the use of information retrieval packages using query languages
• Experience with one or more communications protocols
• Experience in structured methodologies for the design development implementation and maintenance of applications
• Experience eliciting and documenting information from diverse business area stakeholders and subject matter experts
• Excellent analytical problem-solving and decision-making skills; verbal and written communication skills; interpersonal and negotiation skills
• A team player with a track record for meeting deadlines

ETL Skills

• Knowledge and experience in gathering requirements and business process knowledge in order to transform the data in a way thats geared towards the needs of end users
• Knowledge and experience maintaining and improving already existing processes.
• Knowledge and experience ensuring that the data architecture is scalable and maintainable
• Knowledge and experience working with the business in designing and delivering correct high quality data
• Experience investigating data to identify potential issues within ETL pipelines notify end-users and propose adequate solutions
• Experience preparing detailed documentation for further reference

Desirable Skills

• Knowledge and experience with programming Internet-ready applications
• Knowledge and experience in rapid application development (RAD) methodologies
• Knowledge and understanding of Information Management principles concepts policies and practices
• Experience with middleware and gateways
• Experience reviewing analyzing and modifying product installation scripts including encoding testing debugging
• Ability to provide post-implementation support and resolve any post-implementation technical issues
• Experience conducting design walkthrough sessions with project team
• Knowledge and understanding of Accessibility for Ontarians with Disability Act (AODA) and related regulations and standards
• Ability to provide user and system documentation as required

Evaluation Table

ETL Development-Data Integration - 40%

• Extensive experience in gathering requirements and business process knowledge in order to design correct and high-quality data transformation.
• Extensive experience in design development and implementation with Azure Data Factory Databricks including:
  • • Self-Hosted Integration Runtime
    • Data movement ETL pipelines
    • Performance tuning on large data volumes
    • Complex data transformations
    • ADF Debugger to validate data transformation.
    • Azure Roles
• Experience with Informatica including:
  • Workflow Manager
  • Repository Manager
  • Designer
  • Workflow Monitor
• Experience with developing solutions using Logic App and Function App.
• Experience in rapid application development (RAD) methodologies.
• Experience in maintaining and improving existing ETL processes.
• Experience in investigating data to identify potential issues within ETL pipelines.
• Experience with Business Intelligence tools IBM Cognos and Microsoft Power BI.

Azure Technologies - 40%

• Experience with CI/CD (DevOps) pipelines and concepts including:
  • Azure Resource Management
  • GitHub Repo
  • Source code version control
  • Branching pull requests build release multiple parallel development repositories merge.
• Experience with Azure DevOps including:
  • • Azure Boards
    • Azure Test Plans
    • Azure Pipelines
    • Azure Repos
    • Azure Artefacts

Oracle PL/SQL Development - 10%

• Extensive experience in designing and developing SQL Server Stored Procedure Oracle PL/SQL programs.

Database Technologies - 10%

• Experience with Oracle Microsoft SQL Server database and tools.
• Experience with relational and hierarchical database technologies.
• Extensive knowledge of Azure SQL DB and Synapse platforms including:
• Performance tuning on large data volume