Data Engineer

Malvern, PA - USA

Monthly Salary: Not Disclosed

Posted on: 7 hours ago

Vacancies: 1 Vacancy

Job Summary

Ropes Assessment - ASW Glue - Pyspark - Data pipelines - SFTP (Secure File Transfer Protocol

Role Summary:

The contractor will support the Middle Office Digital Activity team by building and maintaining scalable data products and pipelines on AWS. The focus of this role is transforming and delivering reliable datasets using SQL Python and PySpark with AWS Glue as the primary ETL tool.

Required Skillset:

- SQL: Proficiency in complex performant queries and data transformations.

- Python: Experience building data pipelines reusable modules and structured code.

- PySpark: Hands-on experience with large-scale data processing and distributed concepts.

- AWS Data Services: Strong experience with AWS Glue (required) S3-based data lakes IAM and job monitoring.

- Data Engineering Fundamentals: Solid understanding of ETL/ELT design data modeling quality checks partitioning and schema evolution.

Optional Skillsets:

- Experience with Redshift or other MPP databases.

- Knowledge of Iceberg or other open table formats.

- Exposure to CI/CD for data pipelines and event-based systems (e.g. Kafka).

- Familiarity with data governance lineage or cataloging tools.

Ropes Assessment - ASW Glue - Pyspark - Data pipelines - SFTP (Secure File Transfer Protocol Role Summary: The contractor will support the Middle Office Digital Activity team by building and maintaining scalable data products and pipelines on AWS. The focus of this role is transforming and delive...