Principal AIML Engineer Post Deployment Governance

As the Principal AI/ML Engineer Post-Deployment Governance within AI Validation & Monitoring (AVM) you will serve as the enterprise technical and methodological authority for post-deployment monitoring and reporting measurement lifecycle evidence and Post Deployment Monitoring (PDM) and Post Deployment Reporting Summary (PDRS) governance. You will define risk-proportionate AIA Governance requirements and standards for monitoring readiness; performance and functionality; patient safety; adoption and fidelity; outcomes; change and retesting; metrics formulas baselines targets and thresholds; subgroup interpretation; uncertainty; and evidence confidence. You will apply data science AI/ML engineering statistical and systems expertise to determine whether evidence is traceable appropriately interpreted proportionate to risk and decision-ready.

Within AIA Governance you will review drafted monitoring reporting measurement and PDRS content; direct corrections and alternate approaches; consult on complex cases; establish precedent; and escalate unresolved technical or policy issues.

• Provide strategic and technical leadership for enterprise post-deployment governance measurement monitoring and reporting and PDM and PDRS standards.
• Define risk-proportionate requirements across pilot full implementation post-deployment change recurring PDRS and legacy-product pathways.
• Establish standards for signals metrics formulas baselines targets thresholds uncertainty evidence confidence outcomes and subgroup interpretation.
• Define monitoring-readiness expectations for sources owners collection methods cadence versions limitations lineage Data Cards Model Cards handoffs and sustainable ownership.
• Provide authoritative SME review of Governance Operations Product Lead assessment content and evidence for policy alignment sufficiency traceability methodological adequacy and decision readiness.
• Apply data science statistical AI/ML engineering and systems methods to assess metric validity source fitness threshold logic analyses limitations and conclusions.
• Review observability logging telemetry workflow signals version context change detection and monitoring and reporting continuity through significant changes.
• Own complex or precedent-setting questions involving monitoring thresholds evidence insufficiency vendor limitations significant change revalidation continuity lifecycle action PDRS or CAIO escalation.
• Recommend corrections alternate methods interim controls additional evidence action plans re-review retesting or revalidation.
• Set precedent issue final AVM direction and escalate policy clinical cross-domain or enterprise impasses.
• Lead PDRS templates and rubrics evidence-confidence and escalation methods metric libraries executive presentation standards and governance acceptance criteria.
• Convert recurring gaps into policy playbooks standard findings rubrics examples training calibration and Product Lead enablement.
• Define enterprise requirements for TRex workflows evidence objects traceability dashboards portfolio visibility and reusable governance capabilities.
• Coordinate with product teams vendors platforms legal committees and enterprise groups on methods tooling specifications and ownership.
• Provide clear complex-case findings that communicate limitations confidence required actions and escalation triggers to technical and non-technical audiences.
• Mentor and calibrate engineers analysts and Product Leads; foster consistent methods and cross-lane coordination with Validation & Evaluation.
• Support audit sampling quality assurance enterprise learning and continuous improvement while preserving AVMs review-and-consultation boundary.
• Provide mentorship guidance and technical leadership to junior engineers. May have supervisory responsibilities.

• A masters degree in engineering computer science mathematics health science or a related field with 7 years of relevant experience or a bachelors degree with 9 years of relevant experience.
• Extensive (7 years) experience applying AI and machine learning in production healthcare environments or similar highly regulated or technology focused industries showcasing an acute understanding of healthcare technology.
• Demonstrated leadership in managing complex projects with a proven ability to navigate intricate project requirements and deliver successful outcomes
• Proven success in fostering collaboration across diverse teams and effectively communicating complex technical concepts to non-technical stakeholders.
• Demonstrated expertise in cloud infrastructure environment and software development tools.
• Experience working with large complex and heterogeneous data sets preferably in healthcare.
• Strong skills in AI/ML techniques and frameworks.
• Expertise with best practices in data engineering data science AI Engineering and the MLOps communities.
• In-depth knowledge of healthcare domain including clinical workflows electronic health records medical terminologies regulatory requirements and industry standards.
• Demonstrated leadership in administration education software development and technical reporting.

• Experience mentoring and training less-experienced team members coupled with strong interpersonal communication and time management skills.

  Preferred Qualifications:

• A Ph.D. or other doctorate is preferred.
• Experience with healthcare industry informatics standards best practices and common data models. Participation in national or international standards organizations or other domain-specific professional organizations or extensive implementation experience with common data development and deployment standards.
• Excellent communication collaboration and stakeholder management skills with the ability to effectively engage with diverse stakeholders and translate complex technical concepts and results to non-technical audiences.
• Demonstrated experience leading technical/quantitative teams in a regulated environment.
• Familiarity with systems or quality engineering best practices regulatory standards and compliance frameworks with the ability to adapt these effectively to different project scenarios.
• Demonstrated experience creating risk management files and verification/validation strategies for digital health technology products within the healthcare industry.
• Demonstrated expertise in user-centered design human factors engineering usability testing methodologies and evaluation across AI product development. Ability to lead expert reviews using established usability practices and methods. Presents findings in easy-to-understand terms for the business or clinical practice.
• Strong problem-solving abilities critical thinking skills and a passion for driving innovation and positive change in healthcare through AI technology.
• Demonstrated hands-on leadership using the TRex assessment application to govern AI tools deployed in EPIC ANIMATE and comparable clinical environments including post-deployment standards metric thresholds evidence confidence significant-change review revalidation and executive escalation.