Backend AI Platform Engineer

We are seeking a Backend AI Platform Engineer to design and build core backend services that power our Agentic AI Platform enabling reasoning-driven agents multi-agent orchestration dialog management and outcome-based workflows.

You will work closely with the Principal AI Agentic Architect to implement high-performance APIs orchestration runtimes agent lifecycle management and integration with frameworks like LangChain LangGraph and LangSmith. This role is hands-on and ideal for someone passionate about building robust scalable backend systems in a fast-paced startup-like environment. 

Key Responsibilities: 

Platform Backend Development

• Experience building agent runtime systems orchestration layers or reasoning engines. 
• Implement agent runtime orchestration services for reasoning planning and tool invocation (integrating LangChain/LangGraph). 
• Develop APIs and SDKs for building deploying and managing agent workflows and tools. 
• Build stateful dialog management and memory services to support multi-turn context-aware agents. 
• Implement multi-agent communication (A2A) protocols and shared context for collaborative agent behavior. 

System Architecture & Scalability

• Work with the Architect to build cloud-native event-driven backend services using microservices or service mesh architecture.
• Design task scheduling and workflow orchestration (e.g. Temporal/Airflow) for agent planning and long-running operations.
• Develop real-time message buses (Kafka/Pub-Sub) for agent communication and event propagation.
• Optimize LLM orchestration performance and cost efficiency with caching batching and token usage monitoring. 

Integrations & Data Services

• Implement integrations with enterprise systems APIs and knowledge sources (REST GraphQL gRPC). 
• Work with vector DBs (e.g. ElasticSearch Pinecone Weaviate FAISS) and RAG pipelines for knowledge-augmented reasoning agents. 
• Support metadata and telemetry pipelines for LangSmith-based evaluation and observability. 

Security Compliance & Governance

• Build secure multi-tenant backend services with RBAC API authentication and tenant isolation. 
• Ensure agent action auditability and reproducibility for compliance and governance. 
• Collaborate on implementing platform guardrails (rate limiting content filtering safe tool usage). 

Testing Observability & Reliability

• Write automated test suites (unit integration load testing) for backend services.
• Implement logging tracing and metrics (OpenTelemetry Prometheus Grafana) for full observability. 
• Participate in on-call rotations and improve platform resiliency and fault tolerance. 

Please note: This is a hybrid role that will be based in San Mateo CA or Bellevue WA and requires an in-office presence three days per week (Tuesday - Thursday). 

Qualifications :

Required Qualifications: 

• 7 years of backend engineering experience with at least 2 years in AI/ML platform development. 
• Strong programming skills in Java and Python
• Hands-on experience with LangChain LangGraph LangSmith LLM/agent frameworks
• Expertise in distributed systems event-driven architectures and microservices
• Solid experience with cloud platforms (AWS) Kubernetes and IaC (Terraform/Helm)
• Familiarity with databases (PostgreSQL Redis MongoDB) and vector databases (ElasticSearch Pinecone FAISS Weaviate)
• Understanding of workflow orchestration (Temporal/Airflow) and messaging systems (Kafka RabbitMQ)
• Knowledge of AI/LLM (OpenAI Anthropic Azure OpenAI etc.) and prompt/tool orchestration

Preferred Qualifications: 

• Exposure to conversation orchestration dialog state tracking and AI-driven planning systems
• Familiarity with enterprise security standards (RBAC SSO/OAuth2 data encryption)
• Prior experience in startup or high-growth environments building SaaS platforms
• Contributions to open-source AI agentic frameworks or backend libraries

Additional Information :

The annual base salary range for this position is $238000 - $342000. 

Compensation is based on a variety of factors including but not limited to location experience job-related skills and level. Bonus/equity may be available.

Freshworks offers multiple options for dental medical vision disability and life insurances. Equity ESPP flexible PTO flexible spending commuter benefits and wellness benefits are also offered. Freshworks also offers adoption and parental leave benefits.

At Freshworks we are creating a global workplace that enables everyone to find their true potential purpose and passion irrespective of their background gender race sexual orientation religion and ethnicity. We are committed to providing equal opportunity for all and believe that diversity in the workplace creates a more vibrant richer work environment that advances the goals of our employees communities and the business.

Remote Work :

No

Employment Type :

Full-time