Enter a job title or keyword
Validation Engineer, AI & Data
Job Location:
Job Location:
Indianapolis, IN - USA
Monthly Salary:
Not provided by the employer
Monthly Salary:
Experience Required:
10years
Experience Required:
Posted:
12 November 2026 (3 hours ago)
Posted:
Application Deadline:
9 February 2027
Application Deadline:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
Validation Engineer AI & Data
Location: Indianapolis IN
Summary: Validate the accuracy reliability completeness and consistency of business context and AI generated responses against approved business requirements and ground truth answers.
Responsibilities:
- Design and implement the testing strategy for the Domain Context Model Business Context Repository ontology mappings and AI response evaluation.
- Develop representative ground truth question and answer test sets with defined expected results and tolerances validated by data owners.
- Build and maintain automated evaluation tests for AI responses generated through the clients Context Injection toolkit.
- Evaluate AI responses with and without domain context to establish baseline accuracy and measure improvements.
- Validate numeric answers SQL and query results business logic aggregations join relationships and narrative responses.
- Identify and classify failures including hallucinations stale information incorrect schemas faulty joins missing business context and persona mismatches.
- Develop repeatable evaluation scripts scoring rubrics regression tests and accuracy reporting dashboards or scorecards.
- Verify context repository data completeness classification consistency access control behavior and changes introduced through automated context updates.
- Document test evidence defects remediation results AI readiness assessments and validation outcomes.
- Support release readiness weekly demonstrations and handover of reusable evaluation frameworks and documentation.
Requirements
Required Technical Expertise: Data quality engineering SQL Python automated testing PyTest or equivalent frameworks ETL and data validation AI and LLM evaluation ground truth testing regression testing and test automation.
Required Skills:
Required Technical Expertise: Data quality engineering SQL Python automated testing PyTest or equivalent frameworks ETL and data validation AI and LLM evaluation ground truth testing regression testing and test automation.
