Advanced Associate, Observability Engineering

Function: Observability Engineering / SRE / Cloud Engineering

About the Role

We are looking for an Observability Engineer to join our Observability Engineering team and help build standardize and operate a modern enterprise observability platform across cloud and application environments.

The ideal candidate will have strong hands-on engineering experience with New Relic Grafana and modern observability technologies combined with solid foundations in cloud engineering automation DevOps and SRE practices.

This role goes beyond dashboard creation and monitoring operations. You will be responsible for engineering scalable observability solutions defining telemetry standards building reusable monitoring capabilities improving application and infrastructure visibility and enabling engineering teams to adopt observability as part of their software delivery lifecycle.

You will work closely with SRE Cloud Engineering Platform Engineering Application Engineering and Operations teams to build a consistent and scalable observability experience across the organization.

Key Responsibilities

1. Observability Engineering

• Design implement and maintain enterprise-grade observability solutions across applications infrastructure cloud platforms and services.
• Build and maintain monitoring alerting dashboards service health views and operational telemetry.
• Develop standardized observability patterns for metrics logs traces events and application performance monitoring.
• Implement observability solutions using New Relic Grafana and other industry-standard tools.
• Develop reusable dashboards alerts instrumentation patterns and observability components.
• Establish observability standards and best practices across engineering teams.
• Continuously improve signal quality by reducing alert noise false positives and non-actionable alerts.

2. New Relic Engineering

• Hands-on engineering experience with New Relic APM Infrastructure Monitoring Browser Monitoring Synthetic Monitoring Logs Distributed Tracing NRQL Alerts Workloads and Dashboards.
• Design and implement New Relic monitoring and alerting strategies for enterprise applications.
• Develop complex NRQL queries alert conditions dashboards and operational views.
• Configure and optimize New Relic agents and integrations.
• Implement application and infrastructure instrumentation.
• Develop reusable New Relic configurations and automation using APIs/IaC where appropriate.
• Participate in New Relic platform governance licensing optimization and standardization.
• Evaluate and implement emerging New Relic capabilities to improve engineering productivity and reliability.

3. Grafana & Visualization

• Build and maintain operational dashboards using Grafana.
• Integrate Grafana with multiple telemetry and data sources.
• Design effective dashboards for application health infrastructure SRE NOC and executive operational visibility.
• Develop visualization standards and reusable dashboard templates.
• Understand the difference between visualization monitoring alerting and observability and apply each appropriately.

4. OpenTelemetry & Modern Observability

• Experience with OpenTelemetry and modern telemetry architectures.
• Implement and manage telemetry collection for metrics logs and traces.
• Understand distributed tracing and service dependency mapping.
• Work with telemetry pipelines collectors agents exporters and integrations.
• Experience with technologies such as Prometheus Loki Elastic Splunk Datadog Dynatrace AppDynamics or similar observability platforms is desirable.
• Evaluate new observability technologies and recommend solutions based on scalability cost reliability and engineering value.

5. Cloud Engineering

Strong cloud engineering fundamentals are expected including experience with one or more major cloud platforms:

• AWS
• Microsoft Azure
• Google Cloud Platform

Experience should include:

• Compute networking storage databases containers and cloud-native services.
• Cloud monitoring and logging.
• IAM and security fundamentals.
• Infrastructure automation.
• Cloud-native architecture and operational best practices.
• Troubleshooting distributed cloud environments.

6. Automation & Infrastructure as Code

• Automate repetitive observability and operational activities.
• Develop scripts and tools using Python Bash Go or similar languages.
• Use Terraform / OpenTofu or equivalent Infrastructure as Code technologies.
• Build reusable automation for dashboards alerts instrumentation integrations and configuration management.
• Integrate observability capabilities into CI/CD pipelines.

7. DevOps & CI/CD

• Experience with modern CI/CD practices and tools.
• Hands-on experience with GitHub Actions Jenkins GitLab CI Azure DevOps or similar platforms.
• Integrate observability and quality gates into deployment pipelines.
• Implement deployment markers and release health monitoring.
• Enable automated validation of application and infrastructure health following deployments.

8. SRE & Reliability Engineering

• Apply SRE principles to improve system reliability and operational maturity.
• Define and monitor SLIs SLOs and error budgets.
• Participate in incident investigation and root-cause analysis.
• Develop proactive monitoring and reliability solutions.
• Identify reliability gaps and engineer solutions to eliminate recurring incidents.
• Support capacity performance availability and resilience engineering.

Required Skills & Experience

Must Have

• 3 years of experience in Observability SRE DevOps Cloud Engineering Platform Engineering or a related engineering discipline.
• Strong hands-on experience with New Relic.
• Strong hands-on experience with Grafana.
• Experience building production-grade dashboards monitoring and alerting solutions.
• Strong understanding of APM infrastructure monitoring logging metrics tracing and distributed systems.
• Experience with NRQL and New Relic alerting.
• Experience with OpenTelemetry is highly desirable.
• Strong cloud engineering experience in AWS Azure or GCP.
• Strong scripting/programming experience in Python Bash Go or similar.
• Experience with Terraform/OpenTofu or another Infrastructure-as-Code technology.
• Understanding of CI/CD and DevOps practices.
• Understanding of SRE principles SLIs SLOs incident management and reliability engineering.
• Strong troubleshooting and analytical skills.

Good to Have

• New Relic certifications or equivalent hands-on expertise.
• Grafana/Prometheus experience.
• OpenTelemetry implementation experience.
• Kubernetes and container observability.
• Experience with Prometheus Loki Elastic Splunk Datadog Dynatrace AppDynamics or similar platforms.
• Experience designing enterprise observability architectures.
• Experience with observability platform migrations or consolidation.
• Experience with observability cost optimization and licensing governance.
• Experience developing observability-as-code.
• Experience integrating observability with ServiceNow Jira PagerDuty Opsgenie or similar ITSM/incident platforms.
• Experience with AI-assisted observability AIOps anomaly detection or automated incident investigation.

What Success Looks Like

In this role you will be successful when you can:

• Engineer rather than simply operate monitoring.
• Build scalable observability solutions that can be reused across hundreds of applications.
• Turn raw telemetry into meaningful engineering and operational insights.
• Reduce alert noise and improve signal quality.
• Standardize New Relic and Grafana adoption across engineering teams.
• Automate observability configuration and reduce manual operational work.
• Improve application reliability through better telemetry and proactive detection.
• Enable engineering teams to own more of their operational health.
• Help establish Observability as a Platform/Capability rather than simply another monitoring tool.

Behavioral & Leadership Skills

• Strong ownership and accountability.
• Ability to work across engineering application infrastructure and operations teams.
• Strong communication and stakeholder management skills.
• Ability to explain complex technical concepts to both engineers and non-technical stakeholders.
• Strong problem-solving and analytical mindset.
• Comfortable working in a fast-paced enterprise engineering environment.
• Passion for automation engineering excellence reliability and continuous improvement.
• Ability to challenge existing approaches and introduce better engineering practices.