Testing - DET - Manager
Testing - DET - Manager
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
At EY were all in to shape your future with confidence.
Well help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.
Join EY and help to build a better working world.
Job Title: AI Test Manager
Job Summary:
We are seeking a skilled AI Test Manager who will be responsible in leading a team of QE professionals in designing implementing and executing testing strategies for AI-based applications and systems. This role requires a strong understanding of AI/ML concepts testing methodologies and project management skills to ensure the delivery of high-quality products. The AI Test Manager will collaborate with cross-functional teams including data scientists software engineers and product managers to ensure robust testing processes and practices are in place.
Key Responsibilities:
- Leadership and Team Management:
- Lead and mentor a team of QA engineers focused on AI and machine learning projects.
- Foster a collaborative and innovative team culture.
- Conduct performance evaluations provide feedback and support career development for team members.
- Testing Strategy Development:
- Design and implement comprehensive testing strategies for AI applications including functional integration regression and performance testing.
- Develop and maintain test plans test cases and test scripts tailored to AI-specific challenges.
- Ensure compliance with industry standards and best practices in AI testing.
- Collaboration with Cross-Functional Teams:
- Work closely with data scientists developers and product managers to understand requirements and provide insights on testing needs.
- Participate in design reviews and provide input on potential testing challenges and considerations.
- AI Model Validation:
- Define and execute methodologies for validating AI models including bias detection performance evaluation and accuracy testing.
- Establish metrics and KPIs to assess the effectiveness of AI systems and ensure they meet specified requirements.
- Experience in testing LLM models
- Test Automation and Tools:
- Oversee the implementation and maintenance of automated testing frameworks and tools specific to AI applications.
- Stay updated on emerging tools and technologies in AI testing and recommend their adoption as appropriate. Knowledge on explainability testing tools (e.g. LIME SHAP etc.)
- Experience in any of the AI testing platforms and tools (e.g. Functionize Applitools Testim etc.)
- Knowledge on cloud platforms & services for ML Ops / LLM Ops
- Documentation and Reporting:
- Maintain detailed documentation of testing processes results and lessons learned.
- Prepare and present reports to stakeholders on testing progress risks and outcomes.
- Continuous Improvement:
- Identify opportunities for process improvement in testing and development methodologies.
- Advocate for best practices in AI testing across the organization.
- Risk Management:
- Identify potential risks in AI models and applications and develop mitigation strategies.
- Ensure timely communication of testing issues and risks to project stakeholders.
Qualifications:
- Education: Bachelors degree in Computer Science Engineering or a related field; Masters degree preferred.
Experience:
- 10 years of experience in software testing / development with exposure to Python
- 1-2 years of experience in defining test strategy and plan for AI/ML applications
- 2 years in a managerial or leadership role.
Technical Skills:
- Strong understanding of AI/ML concepts algorithms and frameworks (e.g. TensorFlow PyTorch).
- Knowledge on ML / LLM frameworks & libraries
- Proficiency in programming languages commonly used in AI testing (e.g. Python R).
- Experience with test automation tools (e.g. Selenium TestNG) and frameworks.
- Familiarity with data validation model evaluation and statistical analysis.
Soft Skills:
- Excellent communication and interpersonal skills.
- Strong problem-solving abilities and analytical thinking.
- Ability to work collaboratively in a fast-paced dynamic environment.
EY Building a better working world
EY is building a better working world by creating new value for clients people society and the planet while building trust in capital markets.
Enabled by data AI and advanced technology EY teams help clients shape the future with confidence and develop answers for the most pressing issues of today and tomorrow.
EY teams work across a full spectrum of services in assurance consulting tax strategy and transactions. Fueled by sector insights a globally connected multi-disciplinary network and diverse ecosystem partners EY teams can provide services in more than 150 countries and territories.
Required Experience:
Manager
Employment Type
Full Time
