Member of Technical Staff

Our client an AI healthcare software startup is seeking a Member of Technical Staff (Research Scientist) to join their this role youll be at the forefront of developing benchmarks and evaluation methodologies for large language models helping shape how the most advanced AI systems are tested and validated before deployment in enterprise environments.

Key Responsibilities:

- Evaluate new AI models as theyre released (e.g. DeepSeek Gemini)

- Create new benchmarks from scratch including hiring labelers constructing datasets and writing white papers

- Improve underlying methods for auto-evaluation of generated text

- Work closely with engineering to implement and scale evaluation methodologies

- Collaborate with top AI labs and enterprise customers to understand evaluation needs

Qualifications:

- 0-3 years of research experience in AI/ML particularly in benchmarking evaluation methodologies or language models

- Experience working in team environments including development sprints Git best practices and pull request review

- Previous experience at AI startups research labs (e.g. Anthropic DeepMind) or as a founder/early employee at your own company is highly valued

- Published research (especially in NLP or benchmarking) is valued but not required; applied work is prioritized over purely academic publications

- Experience building new benchmarks and evaluation methodologies preferred

- Academic background in Computer Science Machine Learning or a related field (Masters or PhD preferred); candidates with a Bachelors degree and 0-3 years of relevant experience will also be considered

- Background from top tech/AI programs preferred

- NLP research experience with publications in reputable journals preferred

- Strong Python programming skills with the ability to build clean maintainable code

- Experience with deep learning frameworks (PyTorch TensorFlow) and techniques such as diffusion models and language modeling

- Interest in and familiarity with LLM infrastructure

- Experience with Django Flask or other Python-based HTTP servers is a plus

- Strong communication skills; able to give input to others and receive/integrate feedback effectively

Job Reference: X985X343

Package Details: Salary & equity

Visa Sponsorship: Open to visa transfers (e.g. OPT H1B transfers) NOT new sponsorship. Will support relocation or transportation as needed.