INSenior Associate QAGenAITestingEmerging BusinessesAdvisoryGurgaon

Responsibilities:

• Design and implementautomated test frameworks(PythonPytest/Robot Framework) for chatbot and AI applications.

• ConductAPI testingfor chatbot backends RAG pipelines and third-party integrations (REST/GraphQL).

• Validatechatbot workflowsincludingflow-based conversations contextual continuity fallback handling and error reportingfor end-to-end customer scenarios.

• ImplementRAG testingusingRAGADeepEval and LLM-as-a-Judgeto evaluateretrievalaccuracy and hallucination risks.

• Build anAI-driven Testcase Generator(using GPT-4 Omni) to process enterprise documents (doc pdf xml jpg excel) and auto-generate Q&A test datasets.

• Define and trackcomprehensive AI evaluation metrics:

• Quantitative Metrics Accuracy Latency Throughput.

• Qualitative MetricsAnswer Relevancy Faithfulness Hallucination Contextual Relevancy.

• Business MetricsCustomer feedback (thumbs up & thumbs down) resolution rateRacialBiasand Diversity (using embedding models to detect bias/fairness issues).

• Conductperformance load and stress testingusingLocust Grafana and K6ensuring scalability under peak citizen traffic.

• Ensureethical AI complianceby testing fairness inclusivity transparency and privacy.

• Integrate automated test suites intoCI/CD pipelines(Jenkins GitHub Actions GitLab CI).

• Collaborate with AI/ML engineers developers and stakeholders to ensureresponsible AI deployment.

• Generate structured reports/dashboards (Allurepytest-html Grafana) capturingquality performance and ethical AI insights.

Mandatory skill sets:

• Strong experience inQA Automationwith Python (Pytest Robot Framework).

• Handson knowledge in Mobile testing.

• ExpertiseinAPI testing(RESTGraphQL) with tools like PostmanPytest or Rest Assured.

• Hands-on withRAG testing tools(RAGADeepEval LLM-as-a-Judge).

• Proficiencyinautomated Q&A generationusing GPT-based models (GPT-4 Omni).

• Strong knowledge ofperformance/load testing tools(Locust Grafana K6).

• Familiarity withAI evaluation metrics(Quantitative Qualitative Business).

• Experience inchatbot/NLP testing(flow-based conversations contextual relevance).

• Knowledge ofbias/fairness validationusing embedding models for diversity/racial bias detection.

• Strong debugging log analysis and test reporting skills.

Preferred skill sets:

• Exposure toGenerative AI evaluation frameworksfor hallucination and relevance scoring.

• Experience withCI/CD automationin AI testing workflows.

• Awareness ofdata privacybestpracticesin test data handling.

• Domain knowledge incitizen services banking or telecom.

Years of experience:

4 to 6 years of relevant job experience

Education qualification:

Bachelors degree in Computer Science Engineering ora relatedfield or equivalent work experience.

Minimum of 4 years of professional experience in Java development.