Lead Data Engineer

Philadelphia, PA - USA

Monthly Salary: Not Disclosed

Posted on: 3 hours ago

Vacancies: 1 Vacancy

Job Summary

Hi
I hope youre doing well. I had a chance to review your profile and wanted to discuss a full-time hire position with our client a major Systems Integrator.
Please review the JD below and let me know if you would be interested in exploring the opportunity.

Job Title: Lead Data Engineer

Location: Philadelphia PA OR Remote

Duration: Fulltime

Job Description

Must Have Technical/Functional Skills

3 years relevant experience in designing and deploying big data applications and ETL jobs using PySpark APIs/SparkSQL.
Strong experience with AWS services across multiple domains:

o Collection: Kinesis DMS

o Storage: S3 RDS Redshift DynamoDB

o Analytics & ML: Glue EMR Athena SageMaker Bedrock

o Compute: EC2 Lambda ECS

o Security: IAM KMS SSE

Proficiency in SQL and relational databases (Oracle SQL Server Teradata); expert-level query tuning.
Hands-on experience with Python development REST APIs (AWS API Gateway ) and CI/CD pipelines using GitHub.
Familiarity with file formats (JSON Parquet Avro) and Linux/Unix shell scripting.
Exposure to Docker/Kubernetes Delta Lake APIs and data quality frameworks.
AWS certification (Developer Associate or higher) preferred.

Roles & Responsibilities

Architect and maintain data pipelines using AWS native services (Glue Kinesis Lambda S3 Redshift).
Design and optimize data models on AWS Cloud leveraging Redshift RDS and S3.
Implement ETL/ELT workflows and PySpark jobs for data ingestion transformation and storage.
Operationalize self-service data preparation tools (e.g. Trifacta) on AWS.
Conduct performance engineering for large-scale data lakes in production environments.
Participate in design workshops provide trade-offs and recommendations for solution architecture.
Mentor engineers on coding best practices problem-solving and AWS service utilization.
Define code review processes deployment strategies and ensure compliance with security standards.
Collaborate with System Architect and Scrum Master to manage dependencies risks and blockers. Support test strategy defect resolution and root cause analysis during warranty periods.
Maintain documentation in Confluence and ensure team alignment on standards and practices.

Thanks & Regards

Sumit Goyal

Sr. Technical Recruiter

Hi I hope youre doing well. I had a chance to review your profile and wanted to discuss a full-time hire position with our client a major Systems Integrator. Please review the JD below and let me know if you would be interested in exploring the opportunity. Job Title: Lead Data Engineer Location: ...