Lead Cloud Engineer – Spark GCP Platform Engineering
Share
Job Location:
Charlotte, VT - USA
Monthly Salary:
Not Disclosed
Posted on:
7 hours ago
Vacancies:
1 Vacancy
Job Summary
Lead Cloud Engineer Spark / GCP / Platform Engineering
Location: Charlotte NC (Hybrid 3 Days Onsite)
Duration: 12 Months (Extension/Conversion Possible)
Important Notes
- Candidates must be local to Charlotte NC (no relocation)
- Hybrid model: 3 days/week onsite
Role Overview
We are hiring a Lead Cloud Engineer to support a Model Risk Management platform that processes large-scale datasets for financial risk modeling.
This is a hands-on senior individual contributor role with some leadership responsibilities focused on backend platform engineering distributed systems and cloud-native architecture.
Key Responsibilities
Platform Engineering
- Deploy configure and maintain GCP environments / OpenShift clusters
- Support scalable platforms for statistical model execution
Distributed Data Processing
- Design and optimize Apache Spark (PySpark) workflows
- Improve performance using Kubernetes orchestration & auto-scaling
Application Development
- Build backend services using Python (Django) & microservices architecture
- Develop APIs and internal tools for data scientists
CI/CD & Automation
- Implement pipelines using GitHub Actions Helm Harness
- Automate deployments and platform operations
Monitoring & Optimization
- Monitor systems using Prometheus Grafana GCP tools
- Troubleshoot distributed systems and optimize performance
Security & Compliance
- Implement RBAC encryption and secure data handling practices
Collaboration
- Partner with cross-functional teams (Data Platform Engineering)
Required Skills
Must-Have
- Apache Spark / PySpark (10 years production scale)
- Google Cloud Platform (GCP) strong hands-on
- Kubernetes / OpenShift (4 years)
- Python (Django APIs microservices)
- Experience with distributed systems & large datasets
Core Technologies
- Spark (PySpark Scala or Java)
- Docker & containerization
- CI/CD: GitHub Actions Helm Harness
- Storage: S3 GCS HDFS
Nice to Have
- Hadoop GCP migration experience
- AI/ML platform engineering (LLMs GPU-based workloads)
- Financial services domain experience
