Gen AI Tester

Bengaluru - India

Monthly Salary: Not Disclosed

Posted on: 5 hours ago

Vacancies: 1 Vacancy

Job Summary

Job Title: Gen AI Tester / AI QA Engineer

Experience: 6 10 Years
Location: Add Location
Employment Type: Full-time

Job Overview

We are looking for a skilled Gen AI Tester to validate evaluate and ensure the quality and reliability of Generative AI applications and LLM-based solutions. The ideal candidate will have experience in AI/ML testing prompt testing model performance evaluation automation and validating RAG and LLM workflows.

Key Responsibilities

Test and validate LLM-based applications Gen AI workflows and chatbot interactions
Perform prompt testing scenario testing hallucination detection and response validation
Validate RAG pipelines including data ingestion embeddings vector search and retrieval accuracy
Execute functional non-functional and usability testing for Gen AI systems
Evaluate model performance using precision recall BLEU ROUGE and other metrics
Create test plans test cases and defect reports for AI applications
Test APIs microservices and backend integrations involving AI models
Work closely with data engineers ML engineers and developers to improve model output
Perform automation testing using Python PyTest or similar frameworks
Validate guardrails content safety and ethical AI compliance

Must-Have Skills

Strong experience in AI/ML application testing
Hands-on knowledge of LLMs Gen AI tools (OpenAI Azure OpenAI Gemini etc.)
Experience with prompt engineering and prompt evaluation
Understanding of RAG architecture embeddings vector DBs (FAISS Pinecone Chroma)
Strong knowledge of Python for automation/testing
Experience in testing REST APIs
Ability to identify hallucinations bias toxicity and response inconsistencies

Good to Have

Knowledge of Azure ML AWS Sagemaker or GCP AI Platform
Experience with MLOps tools
Familiarity with test automation frameworks
Understanding of NLP concepts
AI/ML related certifications

Education

Bachelors degree in Computer Science IT or related field

Job Title: Gen AI Tester / AI QA Engineer Experience: 6 10 Years Location: Add Location Employment Type: Full-time Job Overview We are looking for a skilled Gen AI Tester to validate evaluate and ensure the quality and reliability of Generative AI applications and LLM-based solutions. The ideal ...