Azure Data Engineer

Role/Skills

Minimum 10 years IT experience, with at least 5+ years' experience in Data intensive activities (Data Modeling, ER Diagram, ETL, Data mapping, Data Governance and Data security)

Experience in End to End implementation & maintenance of large Data Lake or Data Warehouse solution

Experience with Data bricks, Unity Catalog implementation, use of IDE and languages Python, R.

Good experience in Azure data lake and Azure Synapse, Purview

Strong knowledge on Data Governance and Data Security models

Hands on experience with ETL tools (ADF) and Spark Pools

Strong knowledge and experience in data modeling

Strong visualization knowledge and experience using Power BI

Strong knowledge on Data Security / Encryption/Monitoring /Masking

Knowledge on DevOps.

Knowledge in AI models (Azure ML)

Analytical and problem-solving skills with a high degree of initiative and flexibility to be available over extended hours

Ability to communicate with Business SME.

Understanding of Design Patterns Data Models.

Experienced with Onsite Offshore Delivery Model.

Technical background ideally within Managed Services and IT outsourcing industry.

Certification on Data Platform Solutions.

Design teams to support their data infrastructure needs while assisting with data-related technical issues

Responsibilities

Develop and design DataBricks solution like unity catalog for data governance

Develop Data models and patterns for Data Platform Solutions

Assembling large, complex sets of data that meet non-functional and functional business requirements

Conceptualize and solution a centralized datalake solution to bring data from disparate sources

Present agreed solutions to business to get buy-in on the strategy and roadmap

Finding meaningful insights from data using AIML models

Work closely with development team to provide technical guidance, review designs and technical deliverables

Should be able to work with Senior Executives of Client Organization and able to write solution architecture strategy documents/white papers

Identifying, designing and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes

Build and maintain required infrastructure for optimal extraction, transformation and loading of data from various data sources

Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition