Staff Software Engineer, Data Movement

About Stripe

Stripe is a financial infrastructure platform for businesses. Millions of companiesfrom the worlds largest enterprises to the most ambitious startupsuse Stripe to accept payments grow their revenue and accelerate new business opportunities. Our mission is to increase the GDP of the internet and we have a staggering amount of work ahead. That means you have an unprecedented opportunity to put the global economy within everyones reach while doing the most important work of your career.

About the team

The Data Movement team is responsible for building and operating a very large scale read-only ecosystem extending the primary online database in Stripe providing much needed complements to the primary DB in terms of read and query capabilities at massive scale.

We build innovative solutions for real-time ingestion of terabytes of data and billions of events into our data warehouse for analysis and business-specific processing. Our team operates some of Stripes highest volume data pipelines processing over 1 GB/second for many business critical use cases are successful for Stripe including:

• user-facing reporting products (Sigma)
• user-facing dashboards
• Risk and Fraud Detection
• Machine Learning
• Data Scientists
• Payments (pay-in and payout)
• Ledger
• Regulatory Reporting
• Financial Data Reconciliation and
• the list goes on

There are about 10000 downstream jobs built by hundreds of internal teams based on this read-only online ecosystem that the Data Movement team owns and operates.

We adopt a combination of open source technologies and in-house built software to ensure high scalability reliability and near-realtime freshness of data. Key example technologies used include: Flink Spark (and structured streaming) Iceberg (and associated table compaction) Trino Airflow Kafka CDC and Kubernettes etc. And we operate these at a non-trivial scale across multiple regions.

Its a high stake and high visibility role especially as a Staff Engineer you will be one of the most senior members in the org. Not only will you provide strong technical directions for the team we also look up to you as the culture north star for the organization. We really value strong engineers who have a team-first mentality a strong bias for action and a solution-focused mindset.

What youll do

As a Staff Software Engineer on our team you will do the following:

• Design build and maintain innovative next-generation or first-generation versions of solutions and products with an emphasis on reliability scalability and efficiency.
• Define and design opinionated architectural strategies policies and abstractions that build a long term sustainable solution for internal Stripes that will in turn enhance the experience of millions of Stripe users.
• Ensure operational excellence and enable a highly available & reliable Data Movement platform across streaming and batch workloads delivering a world-class data and query ecosystem.
• Collaborate nimbly with high-visibility teams and their stakeholders to support their key initiatives - navigating various types of organizational complexity with the support of your own leadership team.
• Connect your work with improvements in the usability and reliability of Open Source Software (OSS) like Apache Airflow Iceberg Spark and contribute back to the OSS community.

Who you are

Were looking for someone who:

Minimum requirements

• Has very strong operational experience with large-scale high-availability data pipelines from design to execution and safe change management. Expertise in Flink Spark (especially structured streaming) Airflow Python Java SQL Trino and API design is a plus.
• Has experience developing maintaining and debugging distributed systems built with open source tools
• Has experience building infrastructure-as-a-product with a strong focus on users needs
• Has strong collaboration and communication skills and can comfortably interact with both technical and non-technical participants.
• Has the curiosity to continuously learn about new technologies and business processes.
• Is energized by delivering effective user-first solutions through creative problem-solving and collaboration.

Preferred qualifications

• Has experience writing production-level code in expertise in Scala Spark Flink Iceberg Airflow Python Java and SQL is a plus.
• Experience packaging and deploying code into cloud-based environments (AWS GCP Azure) with tools including Bazel Docker Containers etc

• Has experience designing APIs or building developer platforms
• Has experience optimizing the end to end performance of distributed systems
• Has experience with scaling distributed systems in a rapidly moving environment
• Has experience working with data pipelines
• Genuine enjoyment of innovation and a deep interest in understanding how things work

Required Experience:

Staff IC