Senior Data Engineer ETL & Pipelines (Remote)

Our client operates a high-volume B2C creator platform in the entertainment industry serving more than 200000 active users and 30000 creators with payment events flowing through the platform every second.

Data is one of the companys most valuable competitive this role you will work closely with an experienced data analyst to transform a continuous flow of events into clean reliable and query-ready data that supports product payments and growth decisions.

We are looking for a Data Engineer who will take end-to-end ownership of the companys ETL pipelines - the systems responsible for moving data from the transactional platform including MongoDB payment streams and behavioral events into the analytics environment where it can be turned into actionable insights.

This is not simply a dashboard integration role.

You will design and maintain pipelines that remain accurate reliable and timely while millions of events are processed and financial transactions occur every second.

Data integrity is critical. A missing payment event or duplicated transaction is not merely a technical issueit can have a direct financial impact.

• Own the pipelines: Design build and maintain ETL and streaming pipelines that ingest high-volume event and payment data and deliver it to the analytics warehouse reliably and accurately.

• Model the data: Partner closely with the Lead Data Analyst to create warehouse schemas that answer essential business questions related to creator earnings payment throughput retention lifetime value and cohort behavior.

• Engineer for failure: Build idempotent replayable and observable jobs that can withstand traffic spikes retries and partial failures without compromising downstream data.

• Process high-volume events: Move data through queue infrastructure including BullMQ and Redis AWS SQS and RabbitMQ while instrumenting pipelines through OpenTelemetry.

• Protect data integrity: Implement reconciliation checks deduplication backfills and reliable handling of late or out-of-order events.

• Protect financial transactions: Collaborate with payments and platform teams to ensure that every transaction is captured once accurately and on time.
  

• Languages: and TypeScript as the primary technologies with and C#

• Data stores: MongoDB as the primary transactional database and Redis

• Queues and streaming: BullMQ AWS SQS and RabbitMQ

• Analytics and warehouse: PostHog its data warehouse layer and SQL data modeling

• Platform: AWS including S3 and SNS Docker Kubernetes and GitHub Actions

• Observability: OpenTelemetry tracing across every pipeline

You are a strong engineer with substantial experience building high-volume data and ETL pipelines.

You will ideally bring:

• Approximately 510 years of relevant engineering experience

• Strong experience with batch and/or streaming data pipelines

• Deep knowledge of event-driven architectures and message queues

• A strong understanding of idempotency at-least-once versus exactly-once processing retries and out-of-order event delivery

• Professional experience with TypeScript and

• Solid SQL and data-modeling capabilities

• The ability to collaborate with analysts and design schemas that make business queries efficient and intuitive

• A strong focus on data integrity particularly when working with payments and financial transactions

• Experience implementing reconciliation auditing monitoring and data-quality controls

• Previous experience in a high-volume B2C environment such as gaming gambling fintech payments or another real-money platform

• Experience or C# would be considered an advantage.
  

• Experience working with payment or transaction data and financial reconciliation

• Experience with MongoDB change streams or change data capture

• Familiarity with data warehouse tooling and dbt-style modeling

• Experience working with Kubernetes and infrastructure as code

We are a team of builders and problem solvers united by a shared commitment to delivering excellence.

This commitment has shaped a company culture focused on creating solutions and products that consistently exceed expectations. Our drive to learn evolve and solve meaningful business challenges has taken us on a journey that goes #beyondsoftware.

As part of that journey we partner with ambitious companies to help them identify and hire exceptional technology professionals.

We are currently managing the recruitment process for a client seeking an experienced Data Engineer ETL & Pipelines to join its team.

This is a direct-hire opportunity. The successful candidate will be employed or engaged directly by the client while will coordinate the recruitment and selection process.