Azure Data Engineer

Job Summary

Job Description

Key Accountabilities:

Data Architecture & Pipeline Development (35%)

• Design and implement robust scalable and secure data pipelines using Azure Data Factory Azure Synapse Analytics and Microsoft Fabric.
• Develop ETL/ELT processes to ingest data from multiple sources into Azure data storage systems (Data Lake SQL DB Synapse).
• Define and implement data models partitioning strategies and performance optimization for large-scale datasets.
• Design and propose technical solutions develop and enhance SQL stored procedures perform complex transformations using PySpark and maintain notebooks.
• Design criteria and test cases for testing; perform unit and end-to-end tests data validations and document results.
• Define and govern design standards tools best practices and related development for enterprise data models.

Data Integration & Management (25%)

• Connect and integrate data from on-premises and cloud-based sources (e.g. SAP SQL Server REST APIs IoT).
• Maintain and manage Azure Data Lake Storage Gen2 Azure SQL and Delta Lake environments.
• Handle data transformation and enrichment using Spark Python SQL or Data Flows.
• Review the solution requirements support architecture design to ensure selection of appropriate technology efficient use of resources and integration of multiple systems.
• Develop and maintain platform roadmaps for cloud-based data driven products supporting Applications & Analytics solutions.
• Ensure data lineage cataloging and metadata management using Azure Purview.
• Create data profiling data cleansing and data migration strategies and plans.
• Advise stakeholders and translate business requirements into designs for secure scalable and reliable data solutions.
• Develop and maintain architectural documentation including diagrams specifications and standards.
• Deliver actionable solutions based on data needs.
• Participate in the design and development of future state architecture for Data and Analytics platforms.

Data Governance Security & Quality (15%)

• Implement data security policies including encryption access controls and masking (using Azure RBAC Key Vault and Private Endpoints).
• Monitor and improve data quality consistency and accuracy using tools like Azure Data Quality Services or custom validations.
• Manage user access to cloud resources deployed for Data and Analytics like ADLS Azure Synapse Azure Machine Learning and Azure Data Factory.
• Support compliance and governance practices in alignment with enterprise data standards.
• Ensure reliability integrity security and compliance of data assets across workspaces of all business fields and group companies.

Collaboration & Stakeholder Support (15%)

• Work closely with data scientists analysts business stakeholders and application teams to understand requirements and deliver fit-for-purpose data solutions.
• Provide technical support and guidance on data-related issues.
• Assist in Agile/Scrum ceremonies backlog grooming and sprint planning.
• Collaborate with multiple teams within the Material Services business segment as well as vendors and third-party service providers.
• Perform independently under limited supervision or in a team environment.
• Effectively communicate projects to the Manager to avoid scheduling conflicts.
• Possess excellent oral and written communications skills.

Monitoring Optimization & Continuous Improvement (10%)

• Monitor performance and troubleshoot issues in data pipelines and workflows.
• Monitor cost of storage compute and general utilization of the Azure data platform.
• Analyze new and existing requirements to understand impact on applications storage connectivity performance and cost and contribute to optimization strategies.
• Tune query and processing performance in Synapse SQL and Spark.
• Contribute to process automation CI/CD practices and reusable frameworks in Azure DevOps.
• Oversee and govern the expansion of existing data architecture and the optimization of data query performance via best practices.
• Stay current on industry technologies trends and best practices.

Qualifications:

Minimum Requirements:

• Bachelors degree from four-year College or University.
• 3-5 years relevant experience in designing and building data solutions in Azure
• 5-10 years of industry experience and
• SAP Dynamics 365 & Salesforce knowledge preferred
• Azure certifications preferred
• Proficiency in tools services & frameworks: Azure ADLS MS SQL Server Azure Databricks SQL Management Studio Azure Synapse Azure Data Factory KeyVaults DevTest Labs PowerBI Microsoft Purview Azure ML Studio ML frameworks
• Knowledge of DevOps & Agile methodologies
• Strong experience of SQL Python languages

Knowledge Skills and Abilities

• Technical Competencies:
  • Data ETL (Extract Transform Load) processes
  • Data validation rule building with scripting/programming
  • Data Modeling (conceptual logical & physical)
  • Database design & metadata management
  • Problem-solving & critical thinking skills
  • Understanding of SAP & Salesforce object nomenclature and integration artifacts

Job Disclaimer

The above is intended to describe the general content of and requirement for the performance of this job. It is not to be construed as an exhaustive statement of duties responsibilities or requirements.

Benefits Overview

We offer competitive company benefits to eligible positions such as :

• Medical Dental Vision Insurance
• Life Insurance and Disability

• Voluntary Wellness Programs

• 401(k) and RRSP programs with Company Match
• Paid Vacation and Holidays
• Tuition Reimbursement
• And more!

Benefits may vary based on job country union role and/or company segment. Please work with your recruiter or tk representative for applicable benefits information.

Disclaimer

This is to notify the general public that some individuals/entities are using the thyssenkrupp (TK) name trademark domain name and logo without authorization. They are posing as employees representatives or agents of TK and its associated/group companies. These individuals/entities are fraudulently offering jobs online through texts websites telephone calls emails or by issuing fake offer letters. They are also soliciting jobseekers to deposit money in certain bank accounts or providing jobseekers with fraudulent checks to obtain banking information.

TK does not ask solicit or accept any monies in any form from candidates job applicants or potential jobseekers who have applied to or wish to apply to TK whether online or otherwise as a pre-employment requirement. TK bears no responsibility for money being deposited/withdrawn therefrom in response to such fake offers.

TK does not:

job offers from free email services like Gmail Rediffmail Yahoo mail etc.;
payment of any kind from prospective jobseekers or candidates for employment;
anyone to collect money or agree to any monetary arrangement in return for a job at TK;
checks to job seekers; or
job offers through third parties. In the event TK uses professional recruitment services through a third party offers are always made directly by TK and not by any third parties.

PLEASE NOTE:

strongly recommends that potential jobseekers do not respond to such fake solicitations in any manner;
will not be responsible to anyone acting on an employment offer that is not directly made by TK;
making an employment offer in return for money is not authorized by TK; and
reserves the right to take legal action including criminal action against such individuals/entities.

TK follows a formal recruitment process through its own HR department and applications are evaluated by its HR department through pre-defined processes. Please visit our official careers website at to view authentic job openings at TK.

If you receive any unauthorized suspicious or fraudulent offers or interview calls please email us at

We shall not accept any liability towards the representation made in any fraudulent communication or its consequences and such fraudulent communication shall not be treated as any kind of offer or representation by TK or its group companies and affiliates.