SeniorLead Big Data Engineer + AI
SeniorLead Big Data Engineer + AI
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
We are looking for Senior/Lead Big Data Engineer to join our team for a long-term cooperation.
Role Overview:
As a Lead Big Data Engineer you will combine hands-on engineering with technical leadership. Youll be responsible for designing developing and optimizing Spark-based big data pipelines in Palantir Foundry ensuring high performance scalability and reliability. You will also mentor and manage a team of engineers driving best practices in big data engineering ensuring delivery excellence and collaborating with stakeholders to meet business needs. While our project uses Palantir Foundry prior experience with it is a plus but not mandatory.
Key Responsibilities:
- Lead the design development and optimization of large-scale Spark-based (PySpark) data processing pipelines.
- Build and maintain big data solutions using Palantir Foundry
- Ensure Spark workloads are tuned for performance and cost efficiency.
- Oversee and participate in code reviews architecture discussions and best practice implementation.
- Maintain high standards for data quality security and governance.
- Manage and mentor a team of Big Data Engineers providing technical direction
- Drive continuous improvement in processes tools and development practices.
- Foster collaboration across engineering data science and product teams to align on priorities and solutions.
Requirements:
- Bachelors or Masters degree in Computer Science Engineering or related field.
- 6 years in Big Data Engineering with at least 1-2 years in a lead (tech/team lead) role.
- Deep hands-on expertise in Apache Spark (PySpark) for large-scale data processing.
- Proficiency in Python and distributed computing principles.
- Experience designing implementing and optimizing high-volume low-latency data pipelines.
- Strong leadership communication and stakeholder management skills.
- Experience with Palantir Foundry is a plus but not required.
- Familiarity with CI/CD and infrastructure as code (Terraform CloudFormation) is desirable.
We offer*:
- Flexible working format - remote office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program tech talks and trainings centers of excellence and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
*not applicable for freelancers
Required Experience:
Senior IC
Employment Type
Full Time
