Senior Platform Engineer
Share
Job Location:
Woodbridge, CA - USA
Monthly Salary:
Not Disclosed
Posted on:
10 hours ago
Vacancies:
1 Vacancy
Job Summary
Location: Rockville MD Tysons Corner VA or Woodbridge NJ or Jersey City NJ (3 days onsite per week)
Duration: 6 months (long-term extensions)
Work Authorization: Any
Interview: 2 rounds; Offer
Notes:
Senior Platform Engineer Big Data (AWS EMR EKS)
- Build and modernize a large scale AWS big data platform (EMR S3 Athena Trino) supporting enterprise analytics
- Help drive platform evolution toward cloud native containerized workloads on AWS EKS (Kubernetes)
- Work at the intersection of software engineering big data and platform engineering - not ETL only
- Design and operate Spark based data workloads optimizing performance reliability and cost
- Implement CI/CD and Infrastructure as Code (Terraform / CloudFormation) for data platforms
- Ideal for engineers with a strong backend or platform background whove grown into big data
Job Description:
Overview
We are seeking a Senior Platform Engineer with deep Big Data experience to help design operate and modernize a large scale data platform on AWS. This role goes beyond traditional ETL or pipeline development - it is focused on building and evolving the underlying data platform that supports analytics reporting and future AI/ML use cases.
The current environment is built primarily on AWS EMR and S3 with a strong query layer using Athena and Trino. The team is actively modernizing the platform and evaluating AWS EKS (Kubernetes) as part of a shift toward more cloud native containerized data workloads.
This role is ideal for an engineer with a software or platform engineering background who moved into big data rather than a pure ETL developer.
Key Responsibilities
- Design build and operate scalable big data platforms on AWS with S3 as the core data lake.
- Develop and optimize Spark based workloads on EMR including performance tuning and cost optimization.
- Support and enhance federated query engines such as Athena and Trino for large scale analytics.
- Contribute to the modernization of the data platform including evaluation and adoption of Kubernetes/EKS for data services and workloads.
- Build and operate data services and platform components using containerized deployments (Docker EKS).
- Implement and maintain Infrastructure as Code using Terraform and/or CloudFormation.
- Design and support CI/CD pipelines for data and platform workloads.
- Partner with data engineers analytics teams and stakeholders to ensure the platform is reliable performant and extensible.
- Monitor and troubleshoot platform issues across clusters pipelines and query engines using CloudWatch and related tooling.
- Continuously evaluate new technologies and propose improvements to the overall data architecture.
Required Qualifications
- 8 years of experience in Big Data Platform Engineering or Data Engineering roles.
- Strong hands on experience with AWS including:
- EMR
- S3
- Athena
- AWS Glue / Glue Data Catalog
- Solid experience with Spark (PySpark or Scala) and distributed data processing.
- Strong SQL skills particularly with large datasets (Athena Trino Presto etc.).
- Experience with Docker and containerized applications.
- Working knowledge of Kubernetes with exposure to AWS EKS strongly preferred.
- Experience implementing CI/CD pipelines (Jenkins GitHub Actions or similar).
- Infrastructure as Code experience using Terraform and/or CloudFormation.
- Strong scripting and programming skills (Python preferred).
- Ability to think at a platform and architecture level not just task execution.
Nice to Have
- Experience running Spark on Kubernetes (EKS).
- Trino/Presto performance tuning experience.
- Experience preparing data platforms for AI/ML workloads.
- Observability tooling experience (CloudWatch Grafana Prometheus).
- Background as a software engineer before moving into big data.
