Data Engineer

Data Engineer

Location: 3 days Hybrid in Chicago IL

Duration: 6 Month Contract

Interview: 2 video interviews and a final on-site

About:

Data Engineer with hands-on experience in PySpark Databricks and Azure data platforms to design build and support end-to-end data pipelines. The ideal candidate will develop and optimize data transformations build production-grade Python components and maintain cloud-native Azure environments while collaborating with application teams and ensuring high-quality reliable data delivery. This role offers the opportunity to work with large-scale datasets implement ETL/ELT best practices optimize Databricks clusters and leverage modern cloud technologies to support AI/ML initiatives.

Qualifications

Education: Bachelors degree in Computer Science Engineering or a related technical field (or equivalent experience).

Required Skills

Experience: 5 years as a Data Engineer; 3 years in ETL/ELT concepts PySpark and SQL.

SQL: Advanced querying CTEs views joins complex transformations and performance tuning.

Python: 2 years building production-quality modules unit/integration testing logging and CI/CD integration.

Databricks: 1 year working with notebooks jobs workflows external/managed tables Delta Lake and basic cluster configuration.

Azure Data Factory (ADF): 1 year creating and maintaining pipelines including triggers parameterisation monitoring and error handling.

Azure Cloud: Hands-on with ADLS Gen2 Azure SQL Azure App Service and general Azure portal/resource group operations.

Infrastructure as Code: Experience provisioning Azure resources with Pulumi.

ETL/ELT Concepts: Strong understanding of pipeline patterns incremental loads data validation and troubleshooting.

Preferred Skills

Additional Skills (nice-to-have): R for data validation TypeScript for Pulumi pipelines Java/.NET for integration Angular/Spring Boot for minor troubleshooting.