Generative AI- QA Engineer-Senior Associate

Industry/Sector

Specialism

Management Level

Job Description & Summary

Exp 4 yrs

PwC US - Acceleration Center isseekinga highly skilled and experiencedGenAITest Automation QA at the Senior Associate level. Asa QApersonnel you will play a critical role in ensuring the quality and reliability of AI andGenAIprojects. You will work closely with cross-functional teams to design develop and execute test automation strategies and frameworks.

Responsibilities:

- Design develop andmaintaintest automation frameworks for AI and GenAI projects.

-Evaluate generative AI systems for adherence to Responsible AI standards including fairness transparency and ethical bias detection and mitigation testing ensuringequitableperformance across robustness explainability and user safety.

- Develop ground truth datasets for evaluating generative outputs across diverse case creation and result evaluation for iterative AI model refinement.

- Implement and customize frameworks like RAGAS for thorough evaluation of retrieval-augmented generation (RAG) systems. Analyze model performance on metrics such as accuracy relevance coherence and response completeness.

- Collaborate with developers product owners and stakeholders to define test automation requirements and strategies.

- Create and execute test cases scripts and scenarios for functional integration and regression testing.

- Perform API test automation to ensure the seamless integration of AI and GenAI solutions.

-UtilizeAzure Test Automation (or equivalent framework) to automate test cases andvalidatesystem functionality.

- Work with Azure Kubernetes to deploy and manage test environments.

- Collaborate with the development team toidentifyand resolve defects and issues.

- Ensure test coverage andtraceability andprovide test reports and metrics to stakeholders.

- Collaborate with Agile teams to integrate test automation into the CI/CD pipeline using Azure DevOps.

- Stayup to datewith industry best practices and advancements in test automation and AI technologies.

Requirements:

- 4 years of relevant experience as a Test Automation QA or Software Engineer in Test.

- Strong experience inpythonprogramming.

- Experience working on AI/GenAI projects is essential.

- Experience in LLMEvaluation Ground truthpreparationand GenAI Solution evaluation.

- Strong experience with Azure Test Automation (or equivalent framework) for test automation.

-Proficiencyin Azure Kubernetes for test environment management.

- Experience with API test automation using tools such as Postman orRestAssured.

- Familiarity with Azure DevOps for test management and CI/CD integration.

- Strong understanding of software testing principles methodologies and best practices.

- Excellent problem-solving and analytical skills with the ability toidentifyand troubleshoot issues.

-Strong communicationand collaboration skills with the ability to work effectively in cross-functional teams.

- Ability to work in a fast-paced dynamic environment and manage multiple priorities.

Educational Background:

• BE / MCA / Degree/MBA/ Any degree

Travel Requirements

Job Posting End Date