Data Scientist
Sunnyvale, CA - USA
Job Summary
As a Data Scientist focused on Algorithm Evaluation you will serve as a technical leader responsible for driving end-to-end evaluation strategy for complex algorithmic systems. You will develop rigorous methodologies to assess algorithm quality identify failure patterns and quantify system behavior across large-scale datasets and real-world will lead deep dives into algorithm performance uncover insights through advanced statistical analysis and establish scalable frameworks to improve evaluation efficiency and confidence in product decisions. You will also help shape how agentic solutions and AI-assisted tooling are integrated into day-to-day workflows to accelerate data analysis failure investigation annotation quality improvement root-cause discovery and evaluation role requires strong technical depth exceptional analytical rigor and the ability to influence cross-functional teams in highly ambiguous environments.
Define and drive evaluation strategy methodologies and success metrics for machine learning and algorithmic systems across multiple product scalable frameworks for measuring algorithm quality robustness reliability and customer deep technical investigations into algorithm performance model behavior regression trends and failure patterns using large-scale data quantitative metrics and evaluation standards to assess algorithm quality across precision recall latency robustness edge cases and real-world root-cause analysis of algorithm failures and partner with engineering teams to identify optimization opportunities and performance algorithm roadmap and product decisions through rigorous experimentation statistical analysis and actionable and implement automated evaluation pipelines benchmarking systems visualization tools and scalable reporting opportunities to leverage agentic systems LLM-based workflows and AI-assisted tooling to improve efficiency and quality in evaluation data analysis annotation and failure intelligent workflows that utilize AI agents for tasks such as failure pattern clustering anomaly detection data curation evaluation synthesis experiment analysis and large-scale adoption of AI-assisted solutions to reduce manual effort in data deep dives regression triage and annotation quality closely with algorithm engineers ML researchers QA and annotation teams to improve evaluation coverage data quality and operational cross-functional efforts to establish best practices for algorithm validation regression analysis evaluation governance and AI-assisted evaluation team members and raise technical standards in evaluation methodologies statistical rigor and analytical best practices.
BS and a minimum of 10 years relevant industry experiencen7 years of experience in data science machine learning evaluation algorithm analysis or related technical experience driving technical initiatives in ambiguous cross-functional expertise in statistical analysis experimentation methodologies and large-scale data experience evaluating machine learning computer vision or AI systems through quantitative metrics and performance programming experience in Python with hands-on experience building scalable analytics and automation conducting algorithm deep dives failure analysis and model performance with AI-assisted analysis workflows foundation models agentic systems or intelligent automation approaches for technical problem understanding of algorithm evaluation concepts including precision/recall tradeoffs confusion analysis robustness measurement regression detection and benchmarking problem-solving skills with ability to translate ambiguous technical problems into measurable frameworks.
Experience evaluating machine learning computer vision multimodal or foundation model systems in production designing or deploying agentic workflows to improve engineering productivity data analysis evaluation efficiency or annotation with LLM-based systems retrieval pipelines structured reasoning or AI-assisted analytics defining quality frameworks and evaluation methodologies for large-scale intelligent building automated benchmarking systems and large-scale performance monitoring of A/B experimentation causal inference and advanced statistical understanding of the ML lifecycle model validation and continuous evaluation communication skills with proven ability to influence technical decisions through data-driven insights.
Required Experience:
IC
About Company
Ask Siri to name the most successful company in the world and it might respond: Apple. And it's not just out of familial pride. Apple consistently ranks highly in profit, revenue, market capitalization, and consumer cachet. In 2018, the company became the first reach a trillion dollar ... View more