Generative AI Tester - Senior Associate
Generative AI Tester - Senior Associate
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Line of Service
AdvisoryIndustry/Sector
Not ApplicableSpecialism
Data Analytics & AIManagement Level
Senior AssociateJob Description & Summary
At PwC our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights enabling informed decisionmaking and driving business growth.In data engineering at PwC you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines data integration and data transformation solutions.
Years of Experience: Candidates with 4 years of hands on experience
Responsibilities:
Design develop and maintain test automation frameworks for AI and GenAI projects.
Evaluate generative AI systems for adherence to Responsible AI standards including fairness transparency and ethical considerations. Conduct bias detection and mitigation testing ensuring equitable performance across demographic groups.Assess system robustness explainability and user safety.
Develop ground truth datasets for evaluating generative outputs across diverse scenarios. Automate test case creation and result evaluation for iterative AI model refinement.
Implement and customize frameworks like RAGAS for thorough evaluation of retrievalaugmented generation (RAG) systems. Analyze model performance on metrics such as accuracy relevance coherence and response completeness.
Collaborate with developers product owners and stakeholders to define test automation requirements and strategies.
Create and execute test cases scripts and scenarios for functional integration and regression testing.
Perform API test automation to ensure the seamless integration of AI and GenAI solutions.
Utilize Azure Test Automation (or equivalent framework) to automate test cases and validate system functionality.
Work with Azure Kubernetes to deploy and manage test environments.
Collaborate with the development team to identify and resolve defects and issues.
Ensure test coverage and traceability and provide test reports and metrics to stakeholders.
Collaborate with Agile teams to integrate test automation into the CI/CD pipeline using Azure DevOps.
Stay up to date with industry best practices and advancements in test automation and AI technologies.
Required Skills:
Strong experience in python programming.
Experience working on AI/GenAI projects is essential.
Experience in LLM Evaluation Ground truth preparation and GenAI Solution evaluation.
Strong experience with Azure Test Automation (or equivalent framework) for test automation.
Proficiency in Azure Kubernetes for test environment management.
Experience with API test automation using tools such as Postman or RestAssured.
Familiarity with Azure DevOps for test management and CI/CD integration.
Strong understanding of software testing principles methodologies and best practices.
Professional and Educational Background:
- BE / B.Tech / MCA / M.Sc / M.E / M.Tech / MBA
Excellent problemsolving and analytical skills with the ability to identify and troubleshoot issues.
Strong communication and collaboration skills with the ability to work effectively in crossfunctional teams.
Ability to work in a fastpaced dynamic environment and manage multiple priorities.
Education (if blank degree and/or field of study not specified)
Degrees/Field of Study required:Degrees/Field of Study preferred:Certifications (if blank certifications not specified)
Required Skills
API Testing Azure Automation Azure Kubernetes Service (AKS) Generative AI Large Language Models (LLMs)Optional Skills
Accepting Feedback Accepting Feedback Active Listening Agile Scalability Amazon Web Services (AWS) Analytical Thinking Apache Hadoop Azure Data Factory Communication Creativity Data Anonymization Database Administration Database Management System (DBMS) Database Optimization Database Security Best Practices Data Engineering Data Engineering Platforms Data Infrastructure Data Integration Data Lake Data Modeling Data Pipeline Data Quality Data Transformation Data Validation 18 moreDesired Languages (If blank desired languages not specified)
Travel Requirements
Not SpecifiedAvailable for Work Visa Sponsorship
NoGovernment Clearance Required
NoJob Posting End Date
Required Experience:
Senior IC
Employment Type
Full-Time
