As a Senior Principal Applied Scientist on OCIs AI Innovation team youll shaping Oracles next generation AI infrastructure. You will bring foresight into concrete requirements for training and serving models at massive scale. Your evidence-backed insights will guide where we invest across the stack so OCIs AI Infra offerings are ready for what is coming in the future.
This role requires a deep understanding of machine learning techniques including foundation model evaluation traditional NLP/CV metrics human evaluation confidence estimation agentive application evaluation and RAG application evaluation. The successful candidate will have experience in conducting in-depth research producing production-ready code and collaborating with cross-functional teams to integrate evaluation capabilities into various products. A strong background in programming languages such as Python and experience with machine learning frameworks such as TensorFlow or PyTorch is also required.
In this role you will lead deep research and rigorous evaluation of LLMs and related systems designing benchmarks that capture quality latency cost and reliability in real production settings. Expect to prototype new training and inference methods and run large-scale experiments that reveal scaling behavior and bottlenecks.
A PhD in Computer Science Mathematics Statistics Physics Linguistics or a related field with a dissertation or thesis centered on machine learning techniques is preferred although a Masters or Bachelors degree with relevant experience will also be considered.
Responsibilities
Research and Development: Conduct in-depth research on foundation model evaluation including traditional NLP/CV metrics LLMaaJ techniques human evaluation confidence estimation agentive application evaluation and RAG application evaluation. Produce production-ready code for handoff of POC applications to counterparts in Engineering.
Collaboration: Work closely with cross-functional teams to integrate evaluation capabilities into various applications and products.
Innovation: Identify new opportunities for evaluation and explore emerging technologies.
Stay Updated: Maintain a deep understanding of industry trends and advancements in evaluation.
Qualifications and Experience:
PhD Computer Science Mathematics Statistics Physics Linguistics or a related field (with a dissertation thesis or final project centered in Machine Learning/Deep Learning/Generative AI/Computational Linguistics) and a minimum of 4 years work experience. Candidates without a PhD but with 5 additional years experience will be considered.
Strong publication record including as a lead author or reviewer in top-tier journals or conferences.
Extensive experience in generative AI and model/application evaluation.
Strong understanding of machine learning algorithms and architectures
Excellent problem-solving and analytical skills
Strong leadership and communication abilities
Career Level - IC5
Required Experience:
Staff IC
As a world leader in cloud solutions, Oracle uses tomorrow’s technology to tackle today’s challenges. We’ve partnered with industry-leaders in almost every sector—and continue to thrive after 40+ years of change by operating with integrity. We know that true innovation starts when eve ... View more