Gen AI Testing

TalentOla

Job Location:

Bengaluru - India

Monthly Salary: Not Disclosed

Posted on: 7 days ago

Vacancies: 1 Vacancy

Job Summary

Seeking a Lead Software Engineer Automation Testing with 6 8 years of experience in designing developing and executing automation frameworks for AI/ML-driven applications.

The ideal candidate brings hands-on experience with GenAI Testing LangSmith PyTest Agentic frameworks and strong AI/ML evaluation expertise.

Strong programming expertise in Python.
Hands-on expertise with LangSmith Agentic frameworks PyTest BDD/V&V frameworks.

AI / GenAI Testing

Design and execute V&V strategies for classical ML and GenAI/ agentic systems.
Define test plans tests cases and execute them
Evaluate AI models for accuracy performance safety robustness hallucinations bias and correctness.
For GenAI rely on LangSmith for LLM observability tracing monitoring and evaluation for offline and oline evaluation
For classical ML UC ensure V&V tests are done during the whole life cycle of the model (ML monitoring)

Test Automation

Build scalable automation frameworks using PyTest integrated with CI/CD pipelines.
Develop automated test suites for APIs microservices and AI pipelines.
Perform sanity integration functional regression and UAT testing across QA UAT and Prod environments.
Implement automation strategies: Keyword-driven Data-driven BDD Hybrid frameworks.

Good understanding of Data Science related metrics (for both classical ML and GenAI UC)Experience with AI/Chatbot products and testing AI applications.

Experience with Git GitHub Copilot and CI/CD pipelines.

Knowledge of Azure/AWS cloud environments.

Familiarity with MySQL PostgreSQL Oracle SQL Server NoSQL databases.

Seeking a Lead Software Engineer Automation Testing with 6 8 years of experience in designing developing and executing automation frameworks for AI/ML-driven applications. The ideal candidate brings hands-on experience with GenAI Testing LangSmith PyTest Agentic frameworks and strong AI/ML eva...