AIML Machine Learning Engineer, Evaluation Responsible AI and Safety
Cambridge, MA - USA
Job Summary
We are seeking a visionary Senior Machine Learning Engineer to drive Responsible AI Evaluation efforts for global generative AI this role you will provide critical technical execution for operationalizing policy and risk assessments into measurable criteria; designing datasets; developing LLM- and human-in-the-loop methods for data analysis; and implementing robust data pipelines to produce reliable scalable insights that inform ship/no-ship product - and post-launch you will be deeply hands-onanalyzing evaluation results conducting failure investigations and leveraging human and automated feedback to continuously improve feature safety and build user trust. As a senior technical leader on the team you will champion technical excellence empower leadership with data-driven safety recommendations and systematically solve complex engineering roadblocks. This role heavily draws on applied data science scientific investigation cross-functional collaboration and defending safety metrics to stakeholders and decision-makers.
MS or PhD in Computer Science Linguistics Cognitive Science HCI Psychology Mathematics Physics or a similar science or technology field with a strong basis in scientific data collection and analysis at least 4 years of relevant work experience or BA/BS with 8 years of relevant work experiencenExperience with LLM-Judges as well as human-in-the-loop techniques including designing human annotation projects writing guidelines and dealing with highly multi-labeled nuanced and often conflicting datanProficiency in data science machine learning analytics programming with Python u0026 Pandas and AI-assisted coding tools; experience with one or more plotting u0026 visualization librariesnCross-functional collaboration skills to prioritize competing projects needs set and maintain schedules for project milestones and communicate with all levels of team members as well as internal and external stakeholdersnA rigorous approach to metric development; interpretation of experiments and evaluations; and presentation to executivesnComfortable working with sensitive or potentially offensive content
Experience working in the Responsible AI spacenA passion for building outstanding and innovative products. This position involves a wide variety of interdisciplinary skillsnCuriosity about fairness and bias in generative AI systems and a strong desire to help make the technology more equitablenA Social Science or Humanities background or other demonstrable interest in socio-technical issuesnPrior scientific research and publication experiencenProven track record of contributing to diverse teams in a collaborative environmentnExperience working with multimodal models agentic systems and/or diverse languages and localesnExperience with compliance and regulatory needs in the AI space specifically regarding child safety clinical/health applications or companionship chatbotsnExperience conducting large scale benchmarking across models and capabilities
Required Experience:
IC
About Company
Ask Siri to name the most successful company in the world and it might respond: Apple. And it's not just out of familial pride. Apple consistently ranks highly in profit, revenue, market capitalization, and consumer cachet. In 2018, the company became the first reach a trillion dollar ... View more