Prompt Engineer
New York City, NY - USA
Job Summary
Prompt Engineer at COMPANY NAME
We are seeking an experienced Prompt Engineer to design develop and optimize prompts and prompt-driven systems for large language models. This role will collaborate closely with product managers researchers and software engineers to build robust safe and high-performing conversational and generative applications. The ideal candidate combines technical expertise with strong communication and experimentation skills to translate product requirements into repeatable prompt strategies and evaluation frameworks.
- Design prototype and refine prompts templates and prompt chains to achieve product goals and user intents.
- Develop and run systematic experiments to evaluate prompt performance using quantitative and qualitative metrics.
- Create and maintain automated evaluation pipelines benchmarks and datasets for ongoing monitoring.
- Integrate prompts with model APIs and application backends; collaborate with engineers to productionize prompt workflows.
- Implement prompt tuning and retrieval-augmented generation patterns where applicable.
- Document prompt patterns best practices and failure modes; create reusable prompt libraries and templates.
- Work with cross-functional teams to ensure outputs meet quality accessibility and safety standards.
- Identify and mitigate risks related to bias hallucination and misuse; implement guardrails and content filters.
- Bachelor’s or Master’s degree in Computer Science Linguistics Cognitive Science or a related field or equivalent practical experience.
- Proven experience working with large language models and generative AI in a product or research setting.
- Strong Python skills and experience building evaluation and data pipelines.
- Hands-on experience with model APIs and tools (for example OpenAI API Hugging Face LangChain) and integrating them into applications.
- Experience designing experiments A/B tests and quantitative evaluation metrics for language outputs.
- Excellent written and verbal communication skills; ability to translate technical findings into actionable product recommendations.
- Demonstrated understanding of AI ethics model safety and bias mitigation strategies.
- Familiarity with version control (Git) containerization (Docker) and basic data querying (SQL) is a plus.
- Prompt Engineering
- Large Language Models
- Natural Language Processing
- Machine Learning
- Python
- Data Analysis
- API Integration
- OpenAI API
- Hugging Face
- LangChain
- Prompt Tuning
- Model Evaluation
- A/B Testing
- Experimentation
- Instructional Design
- Creative Writing
- Critical Thinking
- Ethics in AI
- Model Safety
- Version Control (Git)
- Docker
- SQL
Required Skills:
Prompt Engineering Large Language Models Natural Language Processing Machine Learning Python Data Analysis API Integration OpenAI API Hugging Face LangChain Prompt Tuning Model Evaluation A/B Testing Experimentation Instructional Design Creative Writing Critical Thinking Ethics in AI Model Safety Version Control (Git) Docker SQL