Backend AI Platform Engineer

Backend AI Platform Engineer

Organizations everywhere struggle under the crushing costs and complexities of "solutions" that promise to simplify their lives. To create a better experience for their customers and employees. To help them grow. Software is a choice that can make or break a business. Create better or worse experiences. Propel or throttle growth. Business software has become a blocker instead of ways to get work done.

There's another option. Freshworks. With a fresh vision for how the world works.

At Freshworks, we build uncomplicated service software that delivers exceptional customer and employee experiences. Our enterprise-grade solutions are powerful, yet easy to use, and quick to deliver results. Our people-first approach to AI eliminates friction, making employees more effective and organizations more productive. Over 72,000 companies, including Bridgestone, New Balance, Nucor, S&P Global, and Sony Music, trust Freshworks' customer experience (CX) and employee experience (EX) software to fuel customer loyalty and service efficiency. And, over 4,500 Freshworks employees make this possible, all around the world.

Fresh vision. Real impact. Come build it with us.

Job Description

We are seeking a Backend AI Platform Engineer to design and build core backend services that power our Agentic AI Platform, enabling reasoning-driven agents, multi-agent orchestration, dialog management, and outcome-based workflows.

You will work closely with the Principal AI Agentic Architect to implement high-performance APIs, orchestration runtimes, agent lifecycle management, and integration with frameworks like LangChain, LangGraph, and LangSmith. This role is hands-on and ideal for someone passionate about building robust, scalable backend systems in a fast-paced, startup-like environment.

Key Responsibilities:

• Experience building agent runtime systems, orchestration layers, or reasoning engines.
• Implement agent runtime orchestration services for reasoning, planning, and tool invocation (integrating LangChain/LangGraph).
• Develop APIs and SDKs for building, deploying, and managing agent workflows and tools.
• Build stateful dialog management and memory services to support multi-turn, context-aware agents.
• Implement multi-agent communication (A2A) protocols and shared context for collaborative agent behavior.

System Architecture & Scalability:

• Work with the Architect to build cloud-native, event-driven backend services using microservices or service mesh architecture.
• Design task scheduling and workflow orchestration (e.g., Temporal/Airflow) for agent planning and long-running operations.
• Develop real-time message buses (Kafka/Pub-Sub) for agent communication and event propagation.
• Optimize LLM orchestration performance and cost efficiency with caching, batching, and token usage monitoring.

Integrations & Data Services:

• Implement integrations with enterprise systems, APIs, and knowledge sources (REST, GraphQL, gRPC).
• Work with vector DBs (e.g., ElasticSearch, Pinecone, Weaviate, FAISS) and RAG pipelines for knowledge-augmented reasoning agents.
• Support metadata and telemetry pipelines for LangSmith-based evaluation and observability.

Security, Compliance & Governance:

• Build secure multi-tenant backend services with RBAC, API authentication, and tenant isolation.
• Ensure agent action auditability and reproducibility for compliance and governance.
• Collaborate on implementing platform guardrails (rate limiting, content filtering, safe tool usage).

Testing, Observability & Reliability:

• Write automated test suites (unit, integration, load testing) for backend services.
• Implement logging, tracing, and metrics (OpenTelemetry, Prometheus, Grafana) for full observability.
• Participate in on-call rotations and improve platform resiliency and fault tolerance.

Please note: This is a hybrid role that will be based in San Mateo, CA or Bellevue, WA and requires an in-office presence three days per week (Tuesday - Thursday).

Qualifications

Required Qualifications:

• 7+ years of backend engineering experience, with at least 2+ years in AI/ML platform development.
• Strong programming skills in Java and Python
• Hands-on experience with LangChain, LangGraph, LangSmith LLM/agent frameworks
• Expertise in distributed systems, event-driven architectures, and microservices
• Solid experience with cloud platforms (AWS), Kubernetes, and IaC (Terraform/Helm)
• Familiarity with databases (PostgreSQL, Redis, MongoDB) and vector databases (ElasticSearch, Pinecone, FAISS, Weaviate)
• Understanding of workflow orchestration (Temporal/Airflow) and messaging systems (Kafka, RabbitMQ)
• Knowledge of AI/LLM (OpenAI, Anthropic, Azure OpenAI, etc.) and prompt/tool orchestration

Preferred Qualifications:

• Exposure to conversation orchestration, dialog state tracking, and AI-driven planning systems
• Familiarity with enterprise security standards (RBAC, SSO/OAuth2, data encryption)
• Prior experience in startup or high-growth environments building SaaS platforms
• Contributions to open-source AI agentic frameworks or backend libraries

Additional Information

The annual base salary range for this position is $238,000 - $342,000. Compensation is based on a variety of factors including but not limited to location, experience, job-related skills, and level. Bonus/equity may be available. Freshworks offers multiple options for dental, medical, vision, disability and life insurances. Equity + ESPP, flexible PTO, flexible spending, commuter benefits and wellness benefits are also offered. Freshworks also offers adoption and parental leave benefits.

At Freshworks, we are creating a global workplace that enables everyone to find their true potential, purpose, and passion irrespective of their background, gender, race, sexual orientation, religion and ethnicity. We are committed to providing equal opportunity for all and believe that diversity in the workplace creates a more vibrant, richer work environment that advances the goals of our employees, communities and the business.