Principal AI Engineer
Share
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
Principal Engineer AI Quality & Evaluation Architecture
Role Overview
Vocera (Stryker) is seeking a Principal Engineer to own end-to-end AI quality across the lifecycle data models prompts evaluation deployment and monitoring. This role will define and scale reliable measurable production-grade AI systems across speech NLP and GenAI in healthcare.
Key Responsibilities
AI Quality Ownership
Own AI quality across the full lifecycle
Define SLAs KPIs release gates and production readiness decisions
Evaluation & Reliability
Build evaluation frameworks for ASR (WER latency) NLP (intent/entity) and LLMs/RAG (hallucination safety groundedness)
Develop benchmarking regression pipelines and golden datasets
Drive adversarial testing edge case handling and failure analysis
AI Testing Platform
Architect scalable evaluation platforms (offline regression A/B shadow testing)
Integrate with CI/CD and MLOps pipelines
Implement monitoring observability and drift detection
Data Governance
Define standards for data curation annotation and versioning
Ensure reproducibility and feedback loops from production
Maintain healthcare data compliance
MLOps & Continuous Quality
Establish AI MLOps standards for evaluation retraining and deployment
Enable continuous evaluation and performance monitoring at scale
Leadership
Act as AI quality authority across the organization
Mentor teams and align with product and business goals
Qualifications
12 years in software/AI engineering; 5 years in LLMs NLP RAG or speech
Experience building scalable AI evaluation frameworks
Expertise in:
LLM evaluation (hallucination safety groundedness)
Golden datasets regression testing adversarial testing
Prompt validation Python data analysis automation
CI/CD MLOps distributed systems
Nice to Have
RAG evaluation & retrieval benchmarking
Speech/ASR evaluation
Azure ML / OpenAI / AI Search
Responsible AI & compliance
Required Experience:
Staff IC
Key Skills
About Company
Stryker is a global leader in medical technologies and, together with its customers, is driven to make healthcare better. The company offers innovative products and services in MedSurg, Neurotechnology, Orthopaedics and Spine that help improve patient and healthcare outcomes. Alongsid ... View more
