Principal Manager AI Quality Engineering
Bala Cynwyd, PA - USA
Job Summary
Marketing Statement:
Tokio Marine North America Services (TMNAS) provides professional support services to Tokio Marine Group companies in the United States including Tokio Marine America First Insurance Company of Hawaii and Philadelphia Insurance Companies. Customer service is our focus and our passion! Tokio Marine North America Services adds value in all that we do providing the best professional service to our business partners allowing them to achieve their goals. Our talented team delivers actuarial audit corporate communications financial and investment human capital services information technology legal and payroll expertise to our customers.
Job Summary:
In support of the VP of Quality Assurance this role works closely with IT and business leaders to formulate and implement comprehensive QA Automation strategies and policies. The Principal Manager Quality Assurance provides strategic leadership and hands-on technical expertise in Automation and Performance engineering. This pivotal role is responsible for building and advancing intelligent Java/Python-based automation frameworks and performance testing solutions while driving AI-augmented testing practices. The role combines strong leadership strategic planning technical oversight and active hands-on contribution to drive innovation in quality engineering practices across the organization.
Essential Job Functions:
Performs essential Quality Engineering functions as applicable to the specific position including:
Automation Strategy & Execution
- Heads the Automation Engineering and Performance Engineering functions for the QA organization providing strategic direction technical leadership and hands-on guidance to build scalable efficient and intelligent automation frameworks.
- Owns and drives the Enterprise business process automation coverage target across critical applications establishing baseline metrics gap analysis and phased delivery roadmaps.
- Architects and maintains end-to-end process automation that spans multiple integrated systems (policy administration claims billing document management data reporting) validating complete business transactions.
AI/ML-Driven Testing Innovation
- Drives innovation in test automation by actively evaluating piloting and implementing AI/ML-powered testing capabilities including intelligent test case generation self-healing test scripts AI-assisted defect prediction and automated root cause analysis.
- Evaluates and integrates AI coding assistants (GitHub Copilot Claude Code Cursor) into QA development workflows to accelerate script development code review framework and process modernization.
- Designs and implements AI/ML model testing frameworks covering data quality validation model performance testing (accuracy precision) bias and fairness testing and production drift monitoring critical for insurance industry AI applications in insurance domain applications.
- Leverages Large Language Models (LLMs) and Generative AI for test case generation from requirements synthetic test data creation automated test documentation and intelligent test prioritization.
- Implements Retrieval-Augmented Generation (RAG) solutions for building QA knowledge bases and test asset repositories.
Hands-On Technical Contribution
- Serves as a hands-on technical contributor in Python/Java-based automation development. Leverages strong proficiency in Python along with key automation packages including Selenium WebDriver Playwright pytest RestAssured Appium Pandas NumPy and AI/ML libraries (scikit-learn TensorFlow/PyTorch LangChain SHAP Fairlearn) to develop scalable automation solutions.
- Leads the migration of legacy automation scripts (UFT/QTP) to modern Automation frameworks supporting ReactJS Unqork and modern web application architectures.
- Designs and implements Page Object Model (POM) frameworks data-driven testing architectures and API testing solutions (RestAssured Postman) for comprehensive multi-layer validation (UI API database integration).
- Establishes CI/CD pipeline integration for automated test execution using Azure DevOps Jenkins or GitHub Actions with parallel execution capabilities.
- Implements test data management strategies including synthetic data generation data masking and environment provisioning.
Quality Strategy & Process
- Formulates and implements QA strategies and policies to ensure product quality and compliance with industry standards and regulations applicable to P&C insurance.
- Supervises the development and execution of test plans including manual and automated testing to identify defects and ensure product quality.
- Works closely with Application Development Engineering Data Operations and Customer Service to ensure quality standards are integrated throughout the product lifecycle
- Identifies potential risks related to product quality develops mitigation strategies and ensures these are communicated and managed effectively.
- Drives continuous improvement initiatives within the QA department leveraging industry best practices emerging technologies and data-driven methodologies to enhance quality and efficiency.
- Establishes AI/ML governance frameworks for model validation explainability and regulatory compliance in collaboration with Risk Compliance and Actuarial teams.
- Ensures conversion of business requirements into verifiable use cases/test cases and maintenance tasks.
- Provides guidance for the planning and execution of User Acceptance Testing (UAT).
- Ensures that products meet all relevant regulatory and industry standards.
Leadership & Team Development
- Leads and mentors a team of QA engineers fostering a culture of continuous learning innovation and technical excellence.
- Designs and executes team upskilling programs for AI/ML testing modern automation frameworks and cloud-native testing practices.
- Builds high-performance teams through effective training mentoring delegating and providing constructive feedback.
- Manages vendor relationships for QA tools AI testing platforms and contractor augmentation.
Other Functions
- Complies with proper internal controls as necessary to conduct job functions and/or carry out responsibilities and/or administrative activities.
- Performs special duties and other projects as may be assigned.
- Establishes and builds strong working relationships and partnerships with IT domain units and senior management.
- Leads change initiatives and operations.
- Sets Service Level Agreements ensures SLAs are met; when SLAs are not met leads improvement efforts.
- Serves as focal point of communication for major service outages/escalations.
Qualifications:
Required
- 12 years of progressive Dev/QA experience with a proven track record of leadership and management of teams for at least 5 years.
- 5 years of hands-on experience building and maintaining automation frameworks in Python and/or Java.
- Strong proficiency in Selenium WebDriver pytest/TestNG and at least one API testing tool (RestAssured Postman ReadyAPI).
- Experience with CI/CD tools (Azure DevOps Jenkins GitHub Actions) and version control (Git).
- Demonstrated experience leading large-scale automation initiatives with measurable coverage and ROI outcomes.
- Working knowledge of AI/ML concepts including supervised/unsupervised learning model evaluation metrics data quality and bias detection.
- Experience using AI coding assistants (GitHub Copilot Claude Code or equivalent) for development productivity.
- Experience with data analysis libraries (Pandas NumPy) and SQL for test data management and validation.
- Excellent communication skills with ability to present technical strategies to executive leadership.
Preferred
- Bachelors Degree in Computer Science Engineering Information Technology or related field.
- 3 years of experience working in P&C insurance or financial services.
- Experience with AI/ML testing frameworks and tools (Great Expectations Evidently AI Fairlearn SHAP).
- Experience with LLM/GenAI applications (prompt engineering LangChain RAG architectures).
- Experience with cloud platforms (Azure AWS GCP) and containerization (Docker).
- Experience with performance testing tools (JMeter Gatling LoadRunner).
- Experience migrating legacy automation (UFT/QTP) to modern frameworks.
- Familiarity with MLOps practices (MLflow model monitoring data drift detection).
- Understanding of P&C insurance business processes (policy lifecycle billing claims underwriting).
Certifications (Preferred)
- AWS Certified Machine Learning Specialty
- Microsoft Certified: Azure AI Engineer Associate
- TensorFlow Developer Certificate (Similar hands-on coding certification)
- (Any) Industry recognized AI certification
Salary range$130000 to $180000. Ultimate salary offered will be based on factors such as applicant experience and geographic location. Our company offers a competitive benefits package and bonus eligibility on top of base.
EEO Statement:
Tokio Marine Group of Companies (including but not limited to the Philadelphia Insurance Companies Tokio Marine America Inc. TMNA Services LLC TM Claims Service Inc. and First Insurance Company of Hawaii Ltd.) is an Equal Opportunity order to remain competitive we must attract develop motivate and retain the most qualified employees regardless of age color race religion gender disability national or ethnic origin family circumstances life experiences marital status military status sexual orientation and/or any other status protected by law.
Required Experience:
Manager
About Company
Tokio Marine North America Services provides a wide range of services across the U.S. as a member of the Tokio Marine Group - one of the premier global insurance companies in the world.