AI Solutions Analyst
Share
Job Location:
Monthly Salary:
Posted on:
12 days ago
Vacancies:
1 Vacancy
Job Summary
Role Overview:
We are seeking an AI Solutions Analyst who will bridge the gap between business intelligence data engineering and AI application teams. The ideal candidate will design and refine how Large Language Models (LLMs) such as GPT interact with enterprise datasets to deliver accurate contextual and compliant business insights.
Responsibilities:
Serve as the link between business data and AI teams ensuring effective interaction between the LLM and modeled datasets.
Define and curate context and knowledge bases to enable GPT to provide relevant precise and compliant outputs.
Collaborate with Data Analysts Engineers and SMEs to identify structure and tag data for AI usage.
Design test and refine prompt strategies and context frameworks aligned with business objectives.
Conduct LLM evaluation and performance testing (evals) to validate accuracy completeness and contextual relevance.
Develop and maintain an LLM Interaction Design Framework documenting context injection prompt templates and retrieval logic.
Partner with IT Compliance and Governance teams to ensure ethical secure and responsible AI deployment.
Produce reports and dashboards summarizing AI performance usage trends and continuous improvement metrics.
Key Deliverables:
LLM Interaction Design Framework: Documentation of GPT connections context injection and prompt architecture.
Knowledge Base Configuration: Curated domain and business rules repository for GPT accuracy.
Evaluation Scripts & Reports: Test datasets scoring criteria and LLM output analysis.
Prompt Library & Usage Guidelines: Standardized templates for consistent and effective AI interactions.
AI Performance Dashboard: Visual summary of LLM accuracy response quality and improvement metrics.
Governance & Compliance Documentation: Inputs to bias prevention ethical use and AI control frameworks.
Required Skills:
LLM EvaluationTesting
Key Skills
- Organizational Management
- Presentation Skills
- Agile
- SAFe
- AWS
- Solution Architecture
- Conflict Management
- Data Management
- Scrum
- Team Management
- Pre-sales
- Management Consulting
