QA Automation with LLM AI

San Jose, CA - USA

Monthly Salary: Not Disclosed

Posted on: 6 hours ago

Vacancies: 1 Vacancy

Job Summary

QA Automation with LLM AI

San Jose CA

Job Description:

Design and run intelligent test strategies for LLM and agent-based systems combining classic QA and test automation skills with a deep understanding of agentic automationand conversational AI

Responsibilities

Design and execute end-to-end test plans for UX / Uber orchestrator/ Sub Agents . Build and maintain agentic AI test suites validating autonomous . Develop test data and synthetic scenarios for edge cases ambiguous queries and multilingual requests.
Evaluate agent behavior through reasoning traces tool-use decisions and memory usage.
Implement and improve automated testing frameworks leveraging AI for conversational flows APIs and integrations.
Collaborate with Product management / cross-functional teams to refine requirements and acceptance criteria.
Document defects test results and insights clearly for stakeholders.

Skill Set

Minimum 5 years of experience in software QA / test engineering. Strong understanding of testing principles: functional regression integration performance.
Hands-on experience testing AI or ML systems such as LLM chatbots or recommendation engines.
Familiarity with key concepts of multi-agentic systems agentic frameworks (e.g. LangChain) AWS AgentCore.
Scripting skills in Python and experience with REST API testing
Hands on experience with AI and Machine Learning (ML) techniques to enhance test automation such as generating test cases predicting potential defects or updating test code for user interface changes.
Hands on experience with test automation frameworks and tools like Selenium Playwright or Cucumber using scripting languages like Python Java or JavaScript and leveraging AI to improve productivity .
Excellent analytical documentation and communication skills
Collaborate with cross-functional teams for requirements clarifications and acceptance criteria.
Document defects test results and insights clearly for stakeholders.

QA Automation with LLM AI San Jose CA Job Description: Design and run intelligent test strategies for LLM and agent-based systems combining classic QA and test automation skills with a deep understanding of agentic automationand conversational AI Responsibilities Design and execute end-to-e...