Data Engineer - Offshore

Hyderabad - India

Monthly Salary: Not Disclosed

Experience Required: 5-6years

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

As a Data Engineer in our team you will be responsible for assessing complex new data sources and quickly turning these into business insights. You also will support the implementation and integration of these new data sources into our Azure Data platform.

Responsibilities:

You are detailed reviewing and analyzing structured semi-structured and unstructured data sources for quality completeness and business value.

You design architect implement and test rapid prototypes that demonstrate value of the data and present them to diverse audiences.

You participate in early state design and feature definition activities.

Responsible for implementing robust data pipeline using Microsoft Databricks Stack

Responsible for creating reusable and scalable data pipelines.

You are a Team-Player collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.

You have strong communication skills to effectively convey complex data insights to non-technical stakeholders.

Skills:

Advanced working knowledge and experience with relational and non-relational databases.

Advanced working knowledge and experience with API data providers

Experience building and optimizing Big Data pipelines architectures and datasets.

Strong analytic skills related to working with structured and unstructured datasets.

Hands-on experience in Azure Databricks utilizing Spark to develop ETL pipelines.

Strong proficiency in data analysis manipulation and statistical modeling using tools like Spark Python Scala SQL or similar languages.

Strong experience in Azure Data Lake Storage Gen2 Azure Data Factory Databricks Event Hub Azure Synapse.

Familiarity with several of the following technologies: Event Hub Docker Azure Kubernetes Service Azure DWH API Azure Azure Function Power BI Azure Cognitive Services.

Azure DevOps experience to deploy the data pipelines through CI/CD.

Qualifications and Experience:

Minimum 5-7 years of practical experience as Data Engineer.

Bachelor s degree in computer science software engineering information technology or a related field.

Azure cloud stack in-production experience.

Azure Data factory , Databricks , Python , Pyspark , SQL , Event hub , relational and non relational database , Docker , Kubernetes , Azure Devops , CI/CD Pipelines