Azure DataLake Engineer [remote]

Title: Azure Datalake Engineer

Desired Start Date: 2/9

Duration: 12 months or longer

Workplace: Remote

Visa: USC/GC/GC-EAD

Location: Houston TX

Client: Fujitsu

Managing Technical Consultant Job Description:

The Azure Data Lake Engineer is responsible for designing building and maintaining scalable data ingestion storage and processing solutions on Microsoft Azure primarily using Azure Data Lake Storage (ADLS) Azure Data Factory and related services.

Top 3 Must Skills

1. 12 years hands-on experience building scalable ETL/ELT pipelines using Azure Data Factory (mapping data flows pipelines triggers)& Synapse pipelines.

2. Experience building data ingestion frameworks data validation workflows and orchestration logic.

3. Integrate SAP ECC/S4 data into Azure Data Lake using ADF/Synapse pipelines for batch & delta loads.

Requirements

Strong Expertise in DATA Azure data lake Azure Databricks Azure SQL Power BI

Expertise to Implementing Data warehousing Solutions experience as Data Engineer in Azure Big Data Environment

Programming experience in Scala or Python SQL Hands-on experience in Azure stack (Azure Data Lake Azure Data Factory Azure Databricks)

Knowledge in PowerBI and Good understanding of other Azure services like Azure Data Lake Analytics & U-SQL Azure SQL DW

Good understanding of Azure Databricks platform and can build data analytics solutions to support the required performance Scale

Proficient in a source code control system such as GIT

Good knowledge or hands-on of DBT tool

Excellent communication skills

Ability to conduct data profiling cataloguing and mapping for technical design and construction of technical data flows

Experience with different database technologies (e.g. SQL database(s) NoSQL such as Hadoop MongoDB Redis etc.

Experience with exposing data to end-users via Power BI Azure API Apps.

Knowledge of Azure Data & Analytics PaaS Services: Azure Data Factory Azure Data Lake Azure Synapse Analytics Azure Databricks Azure IoT Azure HDInsight Spark Azure Cosmos DB Azure Databricks Azure Stream Analytics and Azure SQL DB

Knowledge of Lambda and Kappa architecture patterns and Mastery of SQL

Experience in turning business use cases and requirements into technical solutions

Experience in business processing mapping of data and analytics solutions

Experience in technology solutions practice development architecture consulting and/or cloud/infrastructure technologies

Experience driving knowledge transfer and/or training programs

Experience with agile delivery methodologies proof of concepts and prototyping

Experience with Azure ARM templates PowerShell and CI/CD using Azure DevOp

Knowledge of Master Data Management (MDM) and Data Quality tools

Experience in managing teams being a technical coach

Experience with preparing data for Data Science and Machine Learning purposes

Experience with Azure data Machine Learning and Artificial Intelligence solutions