Testing - DET - Senior

Job Title: Senior AI Test Engineer

Job Summary:

We are seeking a skilled Senior AI Test Engineer to join our dynamic team. The ideal candidate will be responsible for developing and executing test plans and test cases for AI models and systems ensuring their performance reliability and accuracy. You will work closely with data scientists software engineers and product managers to identify testing requirements and deliver high-quality AI products.

Key Responsibilities:

• Test Planning and Design:
• • Collaborate with cross-functional teams to define testing strategies for AI applications.
  • Develop comprehensive test plans and test cases based on functional and non-functional requirements.
  • Good understanding about the various testing touch points in a model development lifecycle
  • Ability to define test cases to validate input data Model functional performance Model response etc

Test Execution:

• • Execute manual and automated tests for AI models and systems.
  • Monitor and document the results of tests identifying any defects or areas for improvement.
  • Prior experience in performing AI specific testing methods like Pairwise testing Metamorphic testing Back-to-Back testing Bias Testing Drift testing etc
  • Knowledge on responsible AI testing
  • Knowledge on explainability testing tools (e.g. LIME SHAP etc.)

• Model Validation:
• • Good understanding about the various performance benchmarks across the different ML models
  • Validate AI models against performance benchmarks and metrics (e.g. accuracy precision recall).
  • Conduct exploratory testing to uncover edge cases and potential failure modes.
  • Exposure to testing LLM models

• Data Management:
• • Work with data engineers to ensure quality and consistency of training and validation datasets.
  • Implement data validation checks to assess the integrity of input data used for AI models.
  • Prior experience in performing various data validation activities like data collection/generation data augmentation Exploratory data analysis data bias and privacy etc

Automation Development:

• • Develop and maintain automated testing frameworks and scripts for AI applications.
  • Utilize tools and libraries (e.g. TensorFlow PyTorch scikit-learn) for testing AI models.

• Defect Tracking and Reporting:
• • Track and manage defects using issue-tracking tools (e.g. JIRA Bugzilla).
  • Prepare detailed reports on test results defect status and overall quality of AI systems.

• Continuous Improvement:
• • Participate in post-mortem analyses of testing processes and results to identify areas for improvement.
  • Stay up-to-date with industry best practices in AI testing and testing methodologies.

• Collaboration and Communication:
• • Communicate effectively with technical and non-technical stakeholders regarding testing progress and outcomes.
  • Provide training and support to team members on AI testing tools and techniques.

Education:

• Bachelors degree in Computer Science Engineering or a related field; Masters degree preferred.

Experience:

• 5-9 years of experience in software testing / development with exposure to Python
• 1-2 years in testing on AI/ML applications.

Technical Skills:

• Strong understanding of AI/ML concepts algorithms and frameworks (e.g. TensorFlow PyTorch).
• Knowledge on ML / LLM frameworks & libraries
• Proficiency in programming languages commonly used in AI testing (e.g. Python R).
• Experience with test automation tools (e.g. Selenium TestNG) and frameworks.
• Working knowledge on AI testing platforms and tools (e.g. Functionize Applitools Testim etc.)
• Familiarity with data validation model evaluation and statistical analysis.

Qualifications:

• Bachelors degree in Computer Science Engineering Data Science or a related field.
• Proven experience in software testing with a focus on AI/ML applications preferred.
• Strong understanding of AI/ML concepts and algorithms.
• Proficiency in programming languages such as Python Java or C.
• Experience with testing frameworks and tools (e.g. Selenium PyTest unittest).
• Familiarity with version control systems (e.g. Git) and continuous integration/continuous deployment (CI/CD) pipelines.
• Knowledge of data preprocessing feature engineering and model evaluation techniques.
• Excellent analytical and problem-solving skills.
• Strong attention to detail and commitment to quality.

Preferred Qualifications:

• Experience with cloud platforms (e.g. AWS Azure Google Cloud) and their AI services.
• Familiarity with big data technologies (e.g. Hadoop Spark).
• Knowledge of Agile methodologies and experience working in Agile teams.