Senior LLMOps AI Platform Engineer

• This role owns the operational foundation for George and other generative AI applications. The person should make the AI platform reliable observable cost-aware secure and release-ready across development QA and production environments.

Role Area Expected Ownership

• Azure AI / Azure OpenAI Model deployments quotas TPM/RPM planning rate-limit troubleshooting deployment configuration model upgrade support fallback planning and cost/performance monitoring.
• Azure AI Search / RAG Infrastructure Index indexer skillset embedding semantic search hybrid search retrieval quality support and performance troubleshooting for George knowledge sources.
• CI/CD and Release Engineering Jenkins pipelines build promotion environment readiness automated gates release checklists rollback planning and deployment validation.
• Kubernetes / Argo CD Argo CD sync health manifest drift Kubernetes pod health environment promotion scaling config maps secrets and rollback operations.
• Langfuse / Observability Trace ingestion dashboards prompt/version visibility datasets support experiment troubleshooting latency token usage cost and failure analysis.
• Production Support Incident triage root-cause analysis operational runbooks cross-team issue resolution and executive-ready status reporting.

George-Specific Responsibilities

• Azure resource configuration: Maintain Azure OpenAI deployments model capacity deployment names environment configuration quotas private networking needs and service-level access. Microsoft documents Azure OpenAI quota as scoped by region subscription model and deployment type which makes active quota planning part of production operations.
• Model operations: Coordinate model upgrades fallback behavior model routing deployment comparisons cost control latency monitoring and model-specific release risk.
• RAG platform support: Support Azure AI Search indexes indexers skillsets chunking strategy vectorization semantic ranking hybrid retrieval document ingestion failures and search performance. Azure AI Search positions RAG as a pattern for grounding LLM responses in proprietary content.
• Langfuse ownership: Own Langfuse uptime trace ingestion dashboards prompt/version visibility dataset support experiment logging scoring ingestion and troubleshooting when runs do not persist.
• Jenkins pipeline ownership: Maintain build test package promotion release and rollback workflows. Jenkins should be used as the automation backbone for CI/CD activities where applicable.
• Argo CD / Kubernetes ownership: Resolve sync errors drift bad manifests image deployment issues pod failures config map/secrets issues autoscaling and rollback scenarios. Argo CD continuously compares desired Git state to live Kubernetes state and surfaces OutOfSync conditions.
• Operational observability: Own dashboards and alerting for p50/p95 response latency LLM latency search latency tool-call latency token usage error rate trace ingestion health cost trends and failed user journeys.
• Incident management: Create runbooks and RCA templates for failed LLM calls quota pressure failed traces broken eval ingestion indexer failures retrieval misses bad releases and gateway or service errors.

Responsibilities for Broader Agentic Applications

• Capability What This Role Should Cover
• Agent runtime operations Support LangGraph LangChain OpenAI Agents SDK Microsoft Agent Framework Semantic Kernel MCP servers and other future orchestration patterns.
• Tool-call observability Track tool selection parameters latency retries failures confidence source system errors and downstream business impact.
• Prompt/model release operations Version prompts compare prompt changes coordinate release promotion roll back bad prompt versions and document model behavior changes.
• Evaluation gates in CI/CD Partner with QA to make eval results latency cost safety and tool-call quality part of release approval.
• Security and governance Support identity secrets private endpoints least-privilege access environment isolation and auditability for AI systems.

Candidate Requirements

• Must-Have Requirement Reason It Matters
• Azure cloud experience George depends on Azure AI/OpenAI Azure AI Search networking monitoring and environment configuration.
• Kubernetes experience Needed for pod health scaling deployment troubleshooting secrets/configuration and service reliability.
• Jenkins or comparable CI/CD experience Needed to own build release validation rollback and promotion workflows.
• Argo CD / GitOps experience Needed to manage environment drift declarative deployments sync issues and auditable release promotion.
• Observability and incident response Needed to debug logs traces metrics dashboards alerts production incidents and performance regressions.
• LLM application understanding Must understand prompts tokens embeddings RAG retrieval context windows hallucinations model latency and rate limits.
• Python/scripting ability Needed for automation eval support Langfuse integrations operational scripts and troubleshooting utilities.
• Cross-functional communication Must translate technical AI platform risks into business-readable status release risk and recommended action.

Strongly Preferred Why It Helps

• Langfuse Directly relevant to George tracing prompt versions datasets eval runs and production behavior analysis.
• Azure AI Foundry / Azure OpenAI Useful for model deployment evaluation monitoring tracing and quota/capacity operations.
• Azure AI Search Required to deeply support George RAG retrieval indexes skillsets and search quality diagnostics.
• Dynatrace / App Insights / OpenTelemetry Helps bridge LLM telemetry with normal distributed system telemetry.
• LangGraph / MCP / agent frameworks Important as George evolves from RAG chatbot into agentic workflow platform.
• IaC: Bicep Terraform Helm Kustomize Makes AI infrastructure repeatable reviewable and environment-consistent.

Expected Deliverables

Deliverable Description

• LLMOps runbook Troubleshooting guide for model failures latency spikes quota limits Langfuse ingestion issues indexer failures Argo CD sync failures and rollback.
• Environment readiness checklist Dev/QA/Prod checklist for Azure resources model deployments secrets pods indexes Langfuse monitoring and routing.
• Release readiness gate Pre-release sign-off proving deployment health eval completion traces captured acceptable latency/cost rollback readiness and no critical incidents.
• Observability dashboard set Dashboards for LLM latency total latency token usage cost search latency tool-call success error rates trace health and feedback trends.
• Incident RCA template Standard format for capturing root cause customer impact detection gap mitigation permanent fix and prevention plan.
• Model/prompt deployment process Documented approach for promoting comparing monitoring and rolling back model prompt and routing changes.

30 / 60 / 90 Day Expectations

• Timeframe Expected Outcomes
• First 30 days Understand George architecture Azure resources model deployments Jenkins Argo CD Langfuse environments release process and current operational gaps. Produce an initial LLMOps assessment.
• First 60 days Improve dashboards runbooks release checklists Langfuse health checks Azure quota monitoring and Argo CD/Jenkins troubleshooting documentation.
• First 90 days Implement or formalize AI release gates improve observability coverage reduce production troubleshooting time and establish repeatable deployment/rollback process for model prompt index and backend changes.