Senior Site Reliability Engineer - Logging Metrics and Monitoring

Join us as we work to create a thriving ecosystem that delivers accessible highquality and sustainable healthcare for all.

We are looking for a Senior Site Reliability Engineerto join ourCloud Infrastructure Engineering & Operationsdivision. Ultimately your work will focus on improving the performance and efficiency of our teams by joining our quest to continue building a worldclass observability platform and contribute to the success of our business.

The Team:

The Logging Metrics and Monitoring team is responsible for building and providing observability services and tools for engineering teams within the Cloud Engineering & Operations and Research & Development zones. Our services are highly visible and used every day to develop monitor troubleshoot and scale our web services. The team is responsible for collecting and hosting large volumes of metrics and log data; we do this by running large scale distributed fault tolerant systems to collect and host all this data.

Our team has a big impact on productivity of hundreds of developers across athenaNation.

In a typical week our engineers work on problems ranging from tuning performance and scaling services to debugging hard problems. Were responsible for delivering new features and partnering with development teams to solve their pressing monitoring and logging issues. We work in an agile sprintbased schedule running daily standups and work in both the private and public cloud

Job Responsibilities

Automate the deployment of logging metrics and monitoring services through configuration management utilizing Puppet.

Address and resolve production incidents by applying Linux administration and engineering expertise.

Lead projects from inception to completion including designing technical solutions managing timelines and executing deliverables.

Design and implement metrics dashboards and alert criteria to effectively monitor and scale services.

Participate in a weeklong oncall rotation in collaboration with team members.

Assist development teams in enhancing their logging and metrics collection processes.

Demonstrate the ability to manage oncall rotations every few weeks.

Typical Qualifications

Possess 5 to 8 years of prior experience in a production environment exhibit strong system administration and DevOps skills for managing services within a Linux environment.

Demonstrate handson experience with configuration management tools such as Puppet or Ansible.

Strong experience troubleshooting production services in a Linux environment and participating in oncall rotations.

Proficient in programming with experience writing and maintaining scripts in the following languages: Bash Ruby Python Perl C Java and Golang.

Experience developing Infrastructure as Code utilizing Terraform and CloudFormation.

Display adaptability and flexibility in response to changing environmental and business demands.

Additional Qualifications

Demonstrated experience in managing production server fleets at a scale of thousands.

Subject matter expertise in relevant technologies including FluentD Kafka Elasticsearch Graphite Clickhouse Prometheus Grafana Graylog Terraform CloudFormation Docker Jenkins and Git.

Exposure to Amazon Web Services (AWS) for deploying managing and scaling applications with a foundational understanding of AWS services architecture and best practices.

Proficient in using protocol analyzers such as tcpdump and Wireshark.