Senior Engineer I - Observability - Pipelines & Visualization

About the role:

The Observability Pipelines and Visualization team manages the doors both entry and exit to the telemetry databases that are managed by the Observability Department. The Pipelines and Visualization team ensures that CoreWeave engineers can effectively produce and consume telemetry for their services. This involves building pipes to ingest and route data in predictable composable ways as well as visualizing that data after the fact.

We are seeking a Senior Engineer who has experience across observability stacks understands the unique problems that come with moving cloudlevel volumes of telemetry data at scale and is excited at the prospect of ensuring both CoreWeavers as well as CoreWeave customers have eyes into the same underlying telemetry data to run their services as efficiently as possible. Knowledge of and experience running Grafana at scale serving a wide variety of customers with varying degrees of access is a strong plus. As you work closely with a team of 36 engineers focused on pipelines and visualization you will have the opportunity to:

• Collaborate with Logging Tracing and Metrics SMEs at a cloud scale.
• Ensure CoreWeavers from teams across the organization have easy access to the telemetry they need for compliance security billing and myriad other specialized use cases.
• Build lowtouch automated tooling that enables CoreWeave engineers to clearly articulate which pieces of telemetry visualizations and dashboards should be presented to customers.
• Develop and enforce best practices on alerting and tracking the health of telemetry ETL pipelines.
• Improve the performance security reliability and scalability of observability services while participating in the teams oncall rotation.
• Enable and evangelize the best practices around alerting. Collaborate with teams to establish observability standards.
• Work with telemetry data at enormous scale ingesting data from industryleading GPU clusters.
• Grow change invest in your teammates be investedin share your ideas listen to others be curious have fun and above all be yourself.

Wondering if youre a good fit We believe in investing in our people and value candidates who can bring their own diversified experiences to our teams even if you arent a 100% skill or experience match. Here are some qualities weve found compatible with our team. If a portion of this resonates with you wed love to talk.

• You have six or more years of experience in a software or infrastructure engineering industry.
• You enjoy helping your colleagues achieve more with less effort.
• You are customer obsessed ecstatic to provide infrastructure as a service and default to adopting a product lens when evaluating platform scale problems.
• You have experience with observability platforms in production and at scale and are versed in reliability engineering concepts such as the different types of testing progressive deployments error budgets the role observability plays and faulttolerant design.
• You have experience using Kubernetes a conceptual understanding of its major components and/or have operated Kubernetes clusters at scale for both eventdriven and stateful orchestration.
• You have experience using Temporal as an eventbased workflow management tool.
• Youre familiar with various logging and metrics systems like ClickHouse Elastic Loki Victoria Metrics Prometheus Thanos and Grafana. You have experience with designing and operating these systems at scale.
• You are familiar with PromQL any other querying language and enjoy understanding the data model for observability systems.
• You are excited at the prospect of zerodowntime transparent migrations of major production services.
• Youre comfortable with the idea of using Go as your primary programming language but dont shy away from templating yaml as circumstances call for it.
• You know your way around a Linux distro shell scripting and/or the Linux storage and networking stacks.
• You can transform problems in elastic solutions decompose them into achievable tasks and socialize both to your teammates.
• Youre excited about being part of a team of diverse perspectives and backgrounds that believe in tackling challenges growing hand in hand and winning together.

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $175000$225000. Pay is based on a number of factors including market location and may vary depending on jobrelated knowledge skills and experience. This position also includes a discretionary bonus equity and a comprehensive benefits package.