Algorithm Evaluation Manager

We are looking for an experienced and highly motivated Engineering Manager to lead a dynamic team focused on machine learning algorithms or Large Language Model (LM) this role you will guide a team responsible for the critical data and evaluation pipelines that ensure our models are accurate robust and performant. The ideal candidate will bring a strong mix of technical leadership expertise in data curation and annotation processes and deep analytical skills. You will collaborate closely with cross-functional research and engineering teams requiring exceptional communication and strategic thinking.

As the Engineering Manager for this team you will be at the forefront of our AI/ML development lifecycle. Your day-to-day responsibilities will include:n- Team Leadership: Lead mentor and grow a team of engineers and data specialists. Foster a culture of innovation rigorous analysis and continuous learning.n- Evaluation Strategy: Define and execute the evaluation strategy for both CV and LM models. Build robust scalable evaluation pipelines that accurately reflect real-world performance.n- Data Pipeline Management: Oversee the end-to-end data lifecycle. This includes establishing data curation guidelines managing data quality and optimizing large-scale annotation workflows with external vendors or internal teams.n- Analytical Deep Dives: Guide the team in performing rigorous data analysis to troubleshoot model regressions uncover data quality issues and identify opportunities for algorithmic improvements.n- Strategic Alignment: Act as the primary point of contact for your team communicating progress bottlenecks and strategic data needs to leadership and partner teams.

Education u0026 Experience: BS and a minimum of 10 years relevant industry experiencenManagement Experience: 2 years of direct people management experience with a track record of hiring mentoring and leading high-performing technical Expertise: Proven experience in model evaluation benchmarking and A/B testing methodologies for machine learning models (Computer Vision or Foundation Models).nInference Infrastructure: Familiarity with the design and architecture of machine learning inference pipelines and underlying u0026 Annotation: Hands-on experience designing and managing data curation strategies and human-in-the-loop annotation Analysis: Strong analytical skills with the ability to dive deep into datasets to identify trends biases and areas for model : Excellent verbal and written communication skills with the ability to translate complex technical concepts to both technical and non-technical stakeholders.

Advanced Degree: PhD in Computer Science Machine Learning or a related Domain Knowledge: Expertise in both Computer Vision (CV) algorithms and Large Language Model (LM) evaluation methodologies (e.g. RLHF prompt evaluation).nScale u0026 Operations: Experience scaling large data operations managing complex annotation workflows and working directly with external data Stack: Familiarity with Python SQL and ML frameworks (e.g. PyTorch) to effectively review technical work and guide engineering -Functional Leadership: Demonstrated ability to drive strategic alignment across downstream product teams ML researchers and platform engineers in a highly matrixed environment.