AI Agentic Tester

AI Agentic Tester

Denver MO (Remote)

Must-Have Skills

* AI Agentic Testing

* Functional Testing

* Generative AI Testing

* Large Language Models (LLMs)

* AI Agents & Agentic AI

* Retrieval-Augmented Generation (RAG)

* Prompt Engineering Validation

* AI Model Validation & Evaluation

* API Testing (Postman REST APIs Swagger)

* Python

* Test Automation (Selenium / Playwright / PyTest)

* AI Evaluation Metrics (Accuracy Hallucination Relevance Consistency)

* End-to-End Workflow Testing

* Multi-Agent Testing

* Azure OpenAI / AWS Bedrock / Gemini

Core Responsibilities

* Perform functional testing of AI agent workflows and end-to-end AI applications.

* Validate LLM responses for accuracy relevance consistency and hallucination rates.

* Test RAG pipelines prompt engineering and AI agent orchestration.

* Execute API testing using Postman Swagger and REST APIs.

* Develop and maintain automation scripts using Python Selenium Playwright or PyTest.

* Validate multi-agent interactions tool calling and workflow execution.

* Apply AI testing methodologies and evaluation metrics to ensure response quality.

* Work with Azure OpenAI AWS Bedrock Gemini or similar AI platforms to validate AI solutions.