As part of our team you will enhance thousands of compute and big data pipelines to deliver greater scalability reliability and efficiency. By leveraging innovative approaches with machine learning and large language models you will improve pipeline quality optimize Spark and Kubernetes resource utilization and create automation that accelerates developer agility.
Bachelors degree in Computer Science Computer Engineering or a related field.
3 years of experience with large-scale data processing and pipelines.
Proficiency in Scala Python and scripting languages.
Experience in and solid understanding of distributed systems performance tuning and resource optimization.
Strong hands-on expertise with Apache Spark and the Hadoop ecosystem.
Experience developing or applying machine learning techniques or LLM-based agentic workflows for data pipeline optimization and data quality improvements.
Knowledge of cost optimization strategies for big data infrastructure.
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.