Data Engineering Consultant- AbInitio and MS Azure

Data Engineering Consultant AbInitio and MS Azure

Should have Telecom Domain and Smart Metering expert

o Accountable for Designing & Developing Cloud Solution for Smart Metering Strategic Data Reporting which provides the Network Performance Measure reports of service provide s network to Data and Communication Company (DCC) and Smart Metering service users (Energy Suppliers).

o Understanding and knowledge of Reporting Data Store (RDS) which is required to support the reporting of Data and Communication Company (DCC)

Should be able to work in the SCRUM team which delivers the ETL Report and Dashboard solution using MS Azure.

Closely work with Product owner Business Analyst to create user stories required for the sprint.

Create design data flows/pseudo code to enable developers to develop Azure ETL/Power BI Reports based on that

Provide required design input and artefacts to the Data Engineers and Data Scientist.

Responsible for the data model and defining Raw Bronze Silver and Gold storage layers.

Developing code using Azure functions Spark and SQL Stored procedures in Azure SQL Pool

Doing code review of ETL code developed using Azure Function

Capacity planning to understand how and when to scale up and scale down

Developing complex Abinitio graphs for the Smart Metering Reporting.

Performance tuning of poorly performing Ab Initio graphs/graphs handling huge volume of data.

Doing high level/low level impact assessment for new change requests.

Designing/Developing ETL solution considering recovery stability reliability scalability and performance.

Translating the reporting requirements to Detailed Design.

Creating Source to Target mapping for various source feeds to target Dimensional Model.

Producing document/prototype graphs with complex file transformations and handing over to developers to complete development.

Conducting Design Walkthrough meetings/ Requirement Clarification meetings with all the stake holders

Providing inputs such as capacity requirement GDPR compliance related information required for networking IT Platform DBA and security teams.

Coordinating regularly with offshore team and ensure they make progress on the tasks given to them.

Doing Ab Initio code review to check if the code follows best practices and meets required standards and provide performance improvement suggestions.

Expertise in ETL tool Ab initio

Should be Expert in MS Azure

o Azure Analytics

o Azure Function

o Power BI

o Data Factory

o Data Brick

Expertise in Business intelligence

Knowledge of Shop For Data

Knowledge of Data Profiler 3.1.4

Knowledge of XML Manipulation

Software development

Analysis and design

Unix/Linux/Windows

Team Leading capabilities