Job Summary:

Key Responsibilities:

• Lead LLM infrastructure efforts across multiple engineering teams ensuring scalable secure and efficient delivery of AI-powered features.
• Design build and maintain production-grade systems for deploying and managing LLMs including versioning A/B testing and rollback strategies.
• Collaborate with the AI team to implement prompt management systems prompt versioning and token optimization strategies.
• Monitor and optimize inference latency throughput caching strategies and multi-provider cost management (OpenAI Anthropic AWS Bedrock etc.).
• Develop observability pipelines including quality metrics evaluation workflows error monitoring and user feedback loops.
• Implement and maintain Retrieval-Augmented Generation (RAG) systems embedding pipelines and vector database operations.
• Support fine-tuning workflows and manage model registries for both proprietary and open-source models.
• Implement AI safety guardrails content filtering and compliance measures to ensure responsible deployment.
• Support general DevOps initiatives 10% of the time including CI/CD improvements and cloud infrastructure updates.
• Maintain thorough documentation of all LLM infrastructure processes and best practices.

Business Problem the LLMOps Engineer Will Solve:

Ideal Candidate Demographics:

• 3 years of experience in LLMOps MLOps or similar production-focused AI/ML roles.
• Strong Python programming skills and familiarity with LLM libraries and frameworks.
• Hands-on experience with LLM providers (OpenAI Anthropic AWS Bedrock Azure Vertex Databricks).
• Experience with vector databases such as Pinecone Weaviate Qdrant or Chroma.
• Knowledge of model serving tools (vLLM TGI Ray Serve).
• Proficiency with Docker Kubernetes and cloud environments (AWS preferred).
• Familiarity with prompt engineering token optimization chain-of-thought approaches and evaluation metrics.
• Experience with LLM-specific tooling (LangSmith Weights & Biases Phoenix MLflow).
• Ability to troubleshoot LLM issues such as latency improvements hallucination mitigation and context window strategies.
• Strong communication skills with both technical and non-technical stakeholders.

• Experience with open-source LLMs (Llama Mistral etc.).
• Knowledge of advanced RAG techniques including hybrid search and re-ranking.
• Exposure to agent frameworks and real-time LLM applications.
• Background in traditional MLOps data engineering or multimodal models.
• Experience with Ruby on Rails.
• Understanding of AI safety and alignment principles.

Our Hiring Process:

Life at Thrive:

• Fast-paced high-trust environment with significant ownership.
• Opportunity to shape the foundation of Thrives AI infrastructure from day one.
• Strong career progression and mentorship opportunities.

Total Rewards Package:

• 3 weeks paid vacation 1-week holiday shutdown
• Health insurance & wellness coverage
• Yearly Learning & Development Allowance
• Yearly Workspace Allowance