Senior LLM Engineer (Prompting & Evaluation)
Senior LLM Engineer (Prompting & Evaluation)
Posted on :
21-08-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Summary (List Format):
- Design test and deploy LLM (Large Language Model) workflows to power Klarions feedback-intelligence engine.
- Own and improve prompt engineering for state-of-the-art models (OpenAI Anthropic etc.) using techniques like few-shot learning chain-of-thought and context-tuning.
- Develop and automate evaluation strategies including precision recall cost and downstream impact for CI and production environments.
- Build and manage a versioned prompt library with automated rollout/rollback and context-injection patterns.
- Monitor model drift and performance ensuring robustness against context or model shifts.
- Implement context injection using retrieval-augmented generation (RAG) vector search and context management patterns (MCP).
- Stay updated on latest LLM research and best practices translating learnings into code that improves key metrics.
- Collaborate closely with product platform and dashboard engineers to rapidly ship and measure new features.
- Utilize strong Python skills to build automated evaluation pipelines data viewers and model-selection logic.
- Communicate technical concepts and trade-offs clearly to both engineering and product teams.
- (Bonus) Apply NLP and data science skills for text analytics pattern detection and causal analysis.
- Design test and deploy LLM (Large Language Model) workflows to power Klarions feedback-intelligence engine.
- Own and improve prompt engineering for state-of-the-art models (OpenAI Anthropic etc.) using techniques like few-shot learning chain-of-thought and context-tuning.
- Develop and automate evaluation strategies including precision recall cost and downstream impact for CI and production environments.
- Build and manage a versioned prompt library with automated rollout/rollback and context-injection patterns.
- Monitor model drift and performance ensuring robustness against context or model shifts.
- Implement context injection using retrieval-augmented generation (RAG) vector search and context management patterns (MCP).
- Stay updated on latest LLM research and best practices translating learnings into code that improves key metrics.
- Collaborate closely with product platform and dashboard engineers to rapidly ship and measure new features.
- Utilize strong Python skills to build automated evaluation pipelines data viewers and model-selection logic.
- Communicate technical concepts and trade-offs clearly to both engineering and product teams.
- (Bonus) Apply NLP and data science skills for text analytics pattern detection and causal analysis.
Employment Type
Full-time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
