Observability Engineer

OBSERVABILITY:

No H1B

Client : Penny Mac

Location : Carrolton TX

Rate : 65

The Sr Observability Engineer will:

• Design & Implement Observability Solutions: Architect build and scale comprehensive monitoring solutions using the New Relic platform including APM Infrastructure Logs Synthetics and custom instrumentation (NRQL).
• Automate with IaC: Develop manage and maintain observability configurations-including alerts dashboards and synthetic checks-using Infrastructure as Code (IaC) tools such as Terraform/OpenTofu.
• Develop Dashboards & Alerts: Create and refine insightful dashboards and actionable alerting policies in New Relic to provide real-time visibility into infrastructure and application health.
• Promote Best Practices: Act as a subject matter expert on observability guiding teams on best practices for logging metrics and tracing to improve system reliability and reduce mean time to resolution (MTTR).
• Troubleshoot & Optimize: Analyze performance data and telemetry to identify bottlenecks troubleshoot production issues and drive performance optimization efforts across the stack.

Qualifications:

• Bachelors degree in Computer Science Engineering or a related field (or equivalent experience).
• 7 years of experience in a Cloud Engineering role (Observability DevOps SRE etc).
• Proven New Relic Expertise: 3 years of hands-on experience with the New Relic platform including deep knowledge of Dashboards NRQL APM and setting up effective alerting.
• Strong IaC Proficiency: 3 years of experience managing infrastructure and configurations with IaC tools like Terraform/OpenTofu (preferred) AWS CDK CloudFormation Chef or Ansible.
• Cloud Platform Experience: Extensive hands-on experience working with major cloud providers such as AWS (preferred) GCP or Azure.
• Scripting Skills: Proficiency in a scripting language such as Python Go or Bash for automation and tooling.
• System Knowledge: Strong understanding of cloud architecture networking principles Windows/Linux Server administration microservices in a SaaS context containerization (Docker Kubernetes) and CI/CD principles.
• Security & Compliance experience: Deep understanding of security best practices and their implementation in cloud infrastructure and CI/CD pipelines. Experience working in complexly regulated environments.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.

Intake Call Notes:

Observability Engineer

Core purpose

This role focuses on building out the companys observability function. It is still a very important role but the team is smaller today and still growing.

What they work on

• Observability coverage across systems
• Monitoring and visibility strategy
• Likely helping mature the observability function as it scales

Skill set

• New Relic is the most important tool mentioned
• Infrastructure as Code
• AWS preferred
• Broad monitoring/observability experience is valuable

What they want in the person

• Ideally someone with direct New Relic depth
• They are open to someone with lighter New Relic experience if the candidate is strong overall
• If someone has strong observability background with other tools and only 1 2 years of New Relic they may still consider them

Flexibility

• New Relic is not a hard stop
• If candidate is good across the board theyll validate through a quick technical panel

Why this role matters

• Small but growing function
• There is currently only one person directly focused here so they need reinforcement