Software Engineer- Product Recommendations

As the Software Engineer Product Recommendations at Klaviyo youll help build the machine learningpowered systems that decide which products to show to whom and when across our platform. Youll work on large-scale backend and data systems that turn billions of behavioral events into real-time personalized product recommendations that drive revenue for merchants of all sizes.

Youll join the Product Recommendation team partnering closely with Machine Learning Engineers AI Engineers other engineers Product Managers and Designers to design build and operate services and data pipelines that power our recommendation features end to endfrom data ingestion and feature generation to ranking models and APIs exposed in product. This is a hands-on backend role with a strong focus on building scalable systems and data processing frameworks with prior ML system experience as a plus (not a hard requirement).

How youll make a difference

Design build and operate backend services that power product recommendations across Klaviyo experiences (email SMS KAgent onsite etc.) with a focus on reliability performance and clear APIs.

Build and maintain large-scale data processing pipelines (e.g. using Apache Spark or similar frameworks) that transform raw events and catalog data into high-quality features and inputs for recommendation models.

Collaborate with ML engineers to productionize recommendation modelsdefining interfaces feature contracts and deployment patterns for batch and/or real-time inference.
Build ML/AI systems such as vector search that power recommendation semantic search and agentic use cases.

Implement and evolve data and service observability (metrics logging tracing dashboards) to ensure recommendations are correct fast and available when customers need them.

Contribute to and improve shared data frameworks libraries and patterns that make it easier to build new recommendation use cases and iterate quickly.

Work with product managers to break down complex recommendation initiatives into clear milestones helping balance experimentation speed with reliability and technical soundness.

Partner on data-driven decision making and A/B testingensuring recommendation systems are instrumented with the right metrics and helping interpret results to guide future iterations.

Participate in on-call and incident response for the systems you own driving follow-ups that improve the resilience and operability of our recommendation stack.

Transform workflows by putting AI at the center building smarter systems and ways of working from the ground upfor example using AI to accelerate development automate tests or better monitor and debug recommendation behavior.

Share knowledge and mentor other engineers on working with large-scale data frameworks distributed systems and best practices for integrating ML into production systems.

Who you are

3 years of software engineering experience including building and operating backend services in production.

Strong focus on backend and distributed systems at scale; youve worked on high-throughput or highly available services and care about latency reliability and operability.

Proficient in Python and comfortable working in at least one modern language used for backend/data work (e.g. Java or Scala).

Proficient with big data frameworks such as Apache Spark (or similar technologies like Flink Beam etc.) for building batch or streaming pipelines.

Comfortable with cloud-native architectures (AWS preferred) and container orchestration (e.g. Kubernetes); able to work with infrastructure and CI/CD pipelines as part of your day-to-day development.

Comfortable with data-driven decision making and A/B testingyou understand how to instrument experiments read results and fold learnings back into the system.

Comfortable designing and querying data models in relational or analytical datastores (e.g. Postgres MySQL data warehouses).

Familiarity with modern DevOps practices (CI/CD monitoring alerting) and how they apply to large-scale data and recommendation systems.

Proven track record of owning projects end-to-endfrom design and implementation through rollout monitoring and iterationideally across multiple components or services.

Excellent collaborator and communicator: you can explain tradeoffs to technical and non-technical partners and work effectively with ML Engineers Software Engineers PMs and other teams.

Youve already experimented with AI in work or personal projects and youre excited to dive in and learn fast. Youre hungry to responsibly explore new AI tools and workflows finding ways to make your work smarter and more efficient.

Nice to haves

Previous experience working on product recommendation systems or adjacent ML-powered features (ranking personalization search or similar).

Experience in AI/ML systems and products such as integrating models into production systems or building features powered by ML.

Experience training machine learning models (e.g. for ranking prediction or personalization) even if you dont consider yourself a full-time ML engineer.

Experience with ML and distributed compute frameworks such as Ray or similar tools.

Experience partnering with data science or ML teams to productionize models (feature stores offline/online parity model deployment and monitoring).

Experience with additional data technologies (e.g. Kafka Kinesis Redis feature stores or vector databases).

Background in e-commerce marketing tech or consumer personalization products

We use Covey as part of our hiring and / or promotional process. For jobs or candidates in NYC certain features may qualify it as an AEDT. As part of the evaluation process we provide Covey with job requirements and candidate submitted applications. We began using Covey Scout for Inbound on April 3 2025.

Please see the independent bias audit report covering our use of Covey here

Required Experience:

IC