Senior AI Agent Engineer Voice AI

Job Description

The Agentic Tribe is revolutionizing the chatbot and voice assistance landscape with Gen3 a cutting-edge AI Agent system thats pushing the boundaries of conversational AI. Gen3 is not your typical chatbot; its a goal-oriented dynamic and truly conversational system capable of reasoning planning and adapting to user needs in real time. Leveraging a multi-agent architecture and advanced language models Gen3 delivers personalized and engaging user experiences going far beyond scripted interactions to handle complex tasks and off-script inquiries with ease.

We are seeking a passionate and experienced Senior Voice AI Agent Engineer with a strong focus on Voice AI to join our this role You will be dedicated to innovating at the forefront of conversational AI engineering intelligent autonomous agents that can listen understand and speak with human-like fluidity.

You will build the cognitive architecture for our voice applications creating systems that can reason plan and execute complex tasks through seamless low-latency spoken dialogue. A key part of your role will be to effectively communicate complex technical concepts to both technical and non-technical stakeholders.

What you will do:

• Design and develop robust stateful and scalable voice-first AI agents using Python specifically optimized for real-time voice interactions managing turn-taking interruptions and low-latency responses.

• Integrate best-in-class real-time Speech-to-Text (STT) Text-to-Speech (TTS) and Voice Activity Detection (VAD) services to create a seamless conversational flow.

• Connect voice agents with existing enterprise systems databases and third-party APIs to create powerful end-to-end automated workflows initiated and managed through voice.

• Establish and own the Evals for voice agent performance and behavior and Iterate over time and systematically improve performance reliability and the overall user experience

• Build end-to-end conversational flows with reasoning planning and dynamic tool use beyond pre-scripted voice experiences.

• Work cross-functionally with product managers ML Scientists and engineers to deeply understand user needs and voice interaction goals.

• Implement fallback recovery and error-handling strategies to deal with noisy audio input or speech recognition inaccuracies.

• Define and track voice-specific evaluation metrics (e.g. word error rate latency conversational naturalness).

• Develop observability tools and guardrails to monitor performance ensure safety and handle edge cases in spoken interactions.

• Document your development architecture decisions and research findings to share knowledge across the team.
  

Requirements:

• LLM-Oriented System Design: Strong experience building multi-step tool-using agents (LangChain Autogen). Familiar with prompt engineering context management and reasoning strategies like Chain-of-Thought and ReAct.

• Voice AI Expertise:

  • Experience building low-latency streaming voice applications. Expertise in integrating and managing real-time STT/TTS models and APIs. Proficient with techniques for Voice Activity Detection (VAD) noise suppression and implementing robust barge-in/interruption logic.

  • Experience with integrating third-party voice AI APIs including Speech-to-Text (STT) and Text-to-Speech (TTS) services from providers like OpenAI Deepgram ElevenLabs etc

  • Understanding of latency timing and streaming audio constraints.

• Tool Integration & APIs: Comfortable connecting agents to external APIs tools databases in secure environments.

• RAG (Retrieval-Augmented Generation): Building pipelines with vector stores chunking strategies and hybrid retrieval.

• Evaluation & Observability: Implementing and Using monitoring tools and evaluation frameworks (Braintrust) to score our AI Agents

• Safety & Reliability: Familiarity with techniques for prompt injection defense guardrails (Rebuff Guardrails AI) and failover logic.

• Performance Optimization: Token budget and latency management using caching model routing etc.

• Programming & Deployment: Expert in Python FastAPI and LLM SDKs. Experience deploying AI apps to cloud platforms (AWS GCP Azure) using CI/CD best practices.

Nice-to-have:

• M.S. / Ph.D. in Computer Science NLP Machine Learning or related field

• Background in spoken dialogue systems or conversational UX design.

• Familiarity with real-time streaming architecture (e.g. WebRTC gRPC ).

• Multilingual ASR/TTS pipeline experience

About Zendesk

Zendesk builds software for better customer relationships. It empowers organizations to improve customer engagement and better understand their customers. Zendesk products are easy to use and implement. They give organizations the flexibility to move quickly focus on innovation and scale with their growth.

More than 100000 paid customer accounts in over 150 countries and territories use Zendesk products. Based in San Francisco Zendesk has operations in the United States Europe Asia Australia and South America.

Interested in knowing what we do in the community Check out the Zendesk Neighbor Foundation to learn more about how we engage with and provide support to our local communities.

Zendesk is an equal opportunity employer and were proud of our ongoing efforts to foster diversity & inclusion in the workplace. Individuals seeking employment at Zendesk are considered without regard to race color religion national origin age sex gender gender identity gender expression sexual orientation marital status medical condition ancestry physical or mental disability military or veteran status or any other characteristic protected by applicable law.

By submitting your application you agree that Zendesk may collect your personal data for recruiting global organization planning and related purposes. Zendesks Candidate Privacy Notice explains what personal information Zendesk may process where Zendesk may process your personal information its purposes for processing your personal information and the rights you can exercise over Zendesks use of your personal information.

#LI-MK12

Hybrid: In this role our hybrid experience is designed at the team level to give you a rich onsite experience packed with connection collaboration learning and celebration - while also giving you flexibility to work remotely for part of the week. This role must attend our local office for part of the week. The specific in-office schedule is to be determined by the hiring manager.

The intelligent heart of customer experience

Zendesk software was built to bring a sense of calm to the chaotic world of customer service. Today we power billions of conversations with brands you know and love.

Zendesk believes in offering our people a fulfilling and inclusive experience. Our hybrid way of working enables us to purposefully come together in person at one of our many Zendesk offices around the world to connect collaborate and learn whilst also giving our people the flexibility to work remotely for part of the week.

As part of our commitment to fairness and transparency we inform all applicants that artificial intelligence (AI) or automated decision systems may be used to screen or evaluate applications for this position in accordance with Company guidelines and applicable law.

Zendesk is an equal opportunity employer and were proud of our ongoing efforts to foster global diversity equity & inclusion in the workplace. Individuals seeking employment and employees at Zendesk are considered without regard to race color religion national origin age sex gender gender identity gender expression sexual orientation marital status medical condition ancestry disability military or veteran status or any other characteristic protected by applicable law. We are an AA/EEO/Veterans/Disabled employer. If you are based in the United States and would like more information about your EEO rights under the law please click here.

Zendesk endeavors to make reasonable accommodations for applicants with disabilities and disabled veterans pursuant to applicable federal and state law. If you are an individual with a disability and require a reasonable accommodation to submit this application complete any pre-employment testing or otherwise participate in the employee selection process please send an e-mail to with your specific accommodation request.