Product Analyst AI Solutions

The Product Analyst AI Solutions is responsible for shaping optimizing and maintaining advanced AI-driven features and user experiences within our product suite. This role sits at the intersection of business analysis AI tuning and user experiencecombining expertise in software requirements gathering prompt engineering and AI model evaluation. The Analyst is heavily focused on refining how AI systems behave in real-world scenarios ensuring outputs are accurate trustworthy and closely aligned with client needs.

You partner with AI Engineers Product Managers UX/UI Designers QA and Subject Matter Experts (SMEs) to extract user requirements translate those into actionable User Stories and evaluation metrics and continuously tune AI systems for best-in-class performance and usability.

This role is pivotal in ensuring our AI solutions provide maximum business value aligning advanced technology with end-user needs while driving continuous improvement and reliability. The Analyst produces high-quality deliverables using systematic processes with impact across multiple projects deadlines and teams.

Principal Duties

AI Tuning & Output Quality

• Design evaluate and refine prompts templates and conversation flows for reliability cost effectiveness and user alignment.
• Maintain prompt libraries track changes and manage versioning.
• Systematically test AI model outputs for accuracy consistency compliance and tone applying rerankers guardrails and validation layers to ensure business rules are met.
• Collaborate with engineering and QA to promptly integrate feedback and resolve model drift or new scenarios.

Requirements Gathering & Documentation

• Proactively gather requirements of moderate scope and complexity from diverse sources (clients market analysis SMEs) to drive product development and AI enhancements.
• Document software and system requirements including user stories program functions test cases and steps required for new features or modifications.
• Ensure all documentation supports future maintainability and growth of AI-enabled features.

Evaluation Metrics & Continuous Improvement

• Establish systematic evaluation frameworks for AI model outputs leveraging quantitative and qualitative benchmarks.
• Run experiments A/B tests and structured evaluations to compare tuning strategies and track performance metrics (accuracy coverage hallucination rates latency).
• Translate business and user feedback into actionable improvements owning continuous iteration for released solutions.

Collaboration & UX Alignment

• Act as a bridge between technical and non-technical stakeholders ensuring clear effective communication and alignment.
• Work closely with product development and UX/UI teams to ensure AI-driven features fit seamlessly into business workflows are intuitive and user-friendly and meet client needs.
• Help shape the overall user experience for AI interactions promoting explainability and transparency.

Education Experience and Special Skills

• Bachelors degree in Computer Science Management Information Science Business or related field OR equivalent work experience.
• 1 years experience in software AI/ML or related product analysis roles.
• Experience with AI/ML applications (LLMs NLP machine learning pipelines).
• Strong understanding of prompt engineering and generative AI interaction design.
• Basic knowledge of data modeling relational database concepts and SQL.
• Familiarity with model evaluation techniques and performance metrics.
• Proficiency with Python or JavaScript for prototyping and evaluation.
• Experience with Agile SCRUM development processes.
• Excellent planning organization and interpersonal communication skills.
• Ability to work independently and collaboratively across functional groups.

• Experience in ERP systems enterprise software or complex business domains.
• Familiarity with RAG (retrieval-augmented generation) embeddings MLOps and responsible AI practices.
• Experience with fine-tuning methods (LoRA QLoRA PEFT) running structured evaluation series or human-in-the-loop systems.