Data Engineer

Role: Data Engineer

Location: Deerfield IL (Onsite)

Duration: Long Term

Contract Role

Must Have:

• Spark/Scala Data Engineer Microsoft Azure Azure Databricks Spark Scala Delta Lake

Role Description:

• Work in Agile development projects for batch and real time data ingestion transformation.
• Work closely with product owners understand requirements lead offshore team on Development and QE of data pipelines.
• Work with upstream/downstream teams to resolve any issues/blockers.
• Work with System Integration/Performance engineering team for integration/performance testing of data pipelines.
• Coordinate with DevOps/RunOps teams for deployment in higher environments.

Competencies:

• Digital : Apache Spark Digital : Scala Digital : Databricks.

Experience (Years):

68

Essential Skills:

• Minimum 46 years of experience in Data Warehousing and Analytics
• Minimum 34 years of experience on Spark with Scala / Azure Databricks
• Hands on experience of building data pipelines using Apache Spark (Azure Databricks) Scala
• Hands on experience working on Databricks batch.
• Hands on experience in Azure Data Factory.
• Hands on experience in designing and delivering solutions using the Azure Data Analytics platform including Azure Storage Azure SQL Data Warehouse Azure Data Lake.
• Experience working on Delta Lakes.
• Experience working in a Dev/Ops environment with tools such as Azure DevOps
• Should be able to work independently on assigned projects with offshore team.

Desirable Skills:

• Knowledge of Talend Tool.
• Exposure to Azure PaaS services is highly desirable.
• Problemsolving and good analytical skills
• Communication teamwork and negotiation skills