Computational Linguist, Generative AI Sr. Associate

Were looking for a versatile Computational Linguist to join our team focused on evaluating and supporting Generative and Agentic AI systems. This role combines linguistic expertise data analysis and hands-on experimentation with large language models.
This role is ideal for someone who can move between qualitative language analysis and quantitative evaluation. Youll work cross-functionally with Machine Learning Engineers Analytics team and annotators to design innovative rigorous and scalable evaluation processes for LLM-powered workflows.

Core Responsibilities
Serve as a subject matter expert engaging with various stakeholders throughout the product lifecycle and maintain a strong understanding of the Chase Digital Assistants model from both customer and technical perspectives.
Manage monitor and evaluate and version the Chase Digital Assistants intent and entity taxonomy and the model training; Enforce taxonomy versioning practices to ensure traceability and rollback capability
Work closely on the adaptation to LLM-driven workflows ensuring seamless integration of LLMs with existing conversational AI architectures and event tracking systems.
Implement and evolve metrics and KPIs across Model Correctness Customer Experience AI Assurance and Business Metrics and ensure evaluation is transparent repeatable and release-decision-ready
Maintain established metrics and introduce new guardrail metrics for LLM and generative use cases
Manage full artifact suite for LLM models including descriptions prompts evaluation rubrics LLM-as-judge prompts guidelines calibration data data statistics & reliability measures.
Apply ontology design principles to improve semantic reasoning and data integration aligned to business standards.
Design frameworks to incorporate knowledge graphs within classification and extraction model architectures.
Align knowledge models with RAG pipelines and agent orchestration to enhance AI functionality.
Work with data scientists software engineers and business stakeholders to translate requirements into robust solutions.
Identifying optimization opportunities across teams supporting continuous improvement across model performance data quality and feature coverage for improving customer experience.

Required Qualifications Capabilities and Skills
Masters degree in Computational Linguistics NLP Linguistics or related field
2 experience in Computational Linguistics or NLP applied to chatbot or conversational AI development
Hands-on experience with Generative AI and Agentic AI frameworks and evaluation (e.g. AutoGen LangGraph CrewAI Sierra)
Linguistic background in discourse & pragmatics
Advanced knowledge of conversational AI product development lifecycle training design conversation analysis
Hands-on experience with LLM integration prompt engineering evaluation and performance monitoring
Proficient in Python Git Linux and Bash scripting
Fluent with NLP/data science libraries: pandas numpy scikit-learn NLTK
Experience with transformer-based models (e.g. BERT GPT) fine-tuning and application
Experience with generative AI SDKs and frameworks (e.g. OpenAI Google Anthropic LangChain)
Comfortable with JSONL CSV and Jupyter notebook workflows
Experience with ontologies/taxonomies and knowledge graphs
Solid understanding of evaluation methodologies including human-AI comparison and red teaming
Direct experience with financial institutions financial products and customer-facing queries
Chase customer service experience highly desirable
Strong written communication for documenting experiments results and processes

Preferred Qualifications
Experience with hybrid conversational architectures generative AI and LLM-driven flow design
Familiarity with LLM safety bias and compliance
Demonstrated success in a highly matrixed organization
Awareness of current GenAI trends and evaluation challenges in subjective NLP tasks
Solve the cold start problem via synthetic data generation for new intents flows and low-resource scenarios.