drjobs Lead Cloud Engineer Kafka

Lead Cloud Engineer Kafka

Employer Active

1 Vacancy
drjobs

Job Alert

You will be updated with latest job alerts via email
Valid email field required
Send jobs
Send me jobs like this
drjobs

Job Alert

You will be updated with latest job alerts via email

Valid email field required
Send jobs
Job Location drjobs

Calgary - Canada

Monthly Salary drjobs

Not Disclosed

drjobs

Salary Not Disclosed

Vacancy

1 Vacancy

Job Description

About the Role:

Grade Level (for internal use):

11

About the Role:

The Team:


The mission of the DSE Messaging team is to provide a reliable scalable and realtime distributed messaging backbone that enables seamless data streaming and eventdriven communication across services systems and application of the Market Intelligence division. We are currently using software products like Apache Kafka MSK and Informatica technologies to link Business Units across MI. Additionally the team owns a sizable infrastructure that is set up on the AWS cloud or onpremise servers with multiple datacenters.

Responsibilities and Impact:

As Lead engineer you will help to ensure that the messaging platform functions successfully safely and efficiently. You will also work to increase productivity decrease downtime and support the mission of the Market Intelligence division. Giving client teams a dependable application infrastructure and helping them to succeed with their unique business use cases are important functions of this role. You will also be working on fixing technical issues maintaining infrastructure providing oncall support and making enhancements as needed by business clients.

You will be posted as a lead level contributor or assigned to collaborate with user groups or client teams in the following areas.

  • Kafka Platform Engineering:
  • Install configure and maintain Kafka clusters and associated infrastructure across multiple regions and accounts.
  • Monitor & troubleshoot technical issues and take corrective action as required.
  • Develop and implement backup and disaster recovery plans for Kafka clusters.
  • Configure and manage Kafka topics partitions and consumer groups.
  • Manage access controls security and authentication protocols for Kafka clusters.
  • Cloud Architecture & Infra Management:
  • Design develop and deploy scalable reliable and secure AWS solutions.
  • Optimize AWS platform performance scalability and costefficiency.
  • Develop and maintain blueprints and design documents for platform architecture.
  • Process Automation:
  • Automate cloud operations using Infrastructure as Code (IaC) tools like Terraform Ansible or AWS CloudFormation.
  • Use scripting languages (Python Bash PowerShell) to optimize and automate workflows.
  • Governance & Security:
  • Implement security best practices to safeguard cloud resources and ensure compliance with organizational and regulatory standards.
  • Develop risk assessments disaster recovery plans and support during disaster recovery exercises.
  • Collaboration & Stakeholder Engagement:
  • Partner with product and platform owners to deliver innovative solutions to empower business applications with scalable infrastructure.
  • Work with crossfunctional teams to evaluate solutions and recommend strategies for modernizing and consolidating legacy platforms.
  • Production Support:
  • Provide operations support for event streaming platforms ensuring Kafka cluster health and stability. Handle incident management request fulfilment and escalation.
  • Analyze logs to debug issues and performance bottlenecks. Collaborate with client teams to troubleshoot platform issues consumer lag and data replication inconsistencies.
  • Perform routine maintenance tasks including log retention management and partition rebalancing.

What Were Looking For:

  • 8yearsofrelevantexperiencecombinedwithabachelorsdegree. Mastersdegreeispreferred.
  • 35 years of experience working with Messaging Platforms in a production environment.
  • Strong knowledge of Kafka architecture including brokers topics partitions and replicas.
  • Proficiency in configuring deploying and managing Kafka clusters in cloud and onpremises environments.
  • Experience with AWS services such as EC2 S3 RDS Elastic Beanstalk Elastic Load Balancer Route 53 VPC IAM CloudFront CloudWatch etc.
  • Experience with automation tools such as CloudFormation Ansible and Terraform
  • Proficiency in Java Scala or Python for Kafkarelated development tasks will be a plus.
  • Familiarity with DevOps practices including CI/CD pipelines monitoring and logging.
  • Strong problemsolving skills and the ability to troubleshoot complex issues in a distributed environment.
  • Excellent communication and collaboration skills to work effectively with crossfunctional teams and stakeholders.

Additional Preferred Qualifications:

  • Working experience in an agile environment and methodologies.
  • Work experience with Observability tools such as Grafana Prometheus and Splunk.
  • Work experience with any of the Linux products. Good knowledge on Shell or Python scripting.
  • Familiarity with containerization and orchestration tools (e.g. Docker Kubernetes)

Right to Work Requirements:

This role is limited to persons with indefinite right to work in the United States.

Return to Work:

Have you taken time out for caring responsibilities and are now looking to return to work As part of our Return to Work initiative Restart we are encouraging enthusiastic and talented returners to apply and will actively support your return to the workplace.

Include for roles that are bonus plan eligible including sales commission plans.

About S&P Global Market Intelligence

At S&P Global Market Intelligence a division of S&P Global we understand the importance of accurate deep and insightful information. Our team of experts delivers unrivaled insights and leading data and technology solutions partnering with customers to expand their perspective operate with confidence andmake decisions with conviction.

For more information visit In It For You

Our Purpose:

Progress is not a selfstarter. It requires a catalyst to be set in motion. Information imagination people technologythe right combination can unlock possibility and change the world.

Our world is in transition and getting more complex by the day. We push past expected observations and seek out new levels of understanding so that we can help companies governments and individuals make an impact on tomorrow. At S&P Global we transform data into Essential Intelligence pinpointing risks and opening possibilities. We Accelerate Progress.


Our People:

Were more than 35000 strong worldwideso were able to understand nuances while having a broad perspective. Our team is driven by curiosity and a shared belief that Essential Intelligence can help build a more prosperous future for us all.

From finding new ways to measure sustainability to analyzing energy transition across the supply chain to building workflow solutions that make it easy to tap into insight and apply it. We are changing the way people see things and empowering them to make an impact on the world we live in. Were committed to a more equitable future and to helping our customers find new sustainable ways of doing business. Were constantly seeking new solutions that have progress in mind. Join us and help create the critical insights that truly make a difference.

Our Values:

Integrity Discovery Partnership


At S&P Global we focus on Powering Global Markets. Throughout our history the worlds leading organizations have relied on us for the Essential Intelligence they need to make confident decisions about the road ahead. We start with a foundation of integrity in all we do bring a spirit of discovery to our work and collaborate in close partnership with each other and our customers to achieve shared goals.

Benefits:

We take care of you so you cantake care of business. We care about our people. Thats why we provide everything youand your careerneed to thrive at S&P Global.

Our benefits include:

For more information on benefits by country visit: Hiring and Opportunity at S&P Global:

At S&P Global we are committed to fostering an inclusive workplace where all individuals have access to opportunities based on their skills experience and contributions. Our hiring practices emphasize fairness transparency and equal opportunity ensuring that we attract and retain top talent. By valuing different perspectives and promoting a culture of respect and collaboration we drive innovation and power global markets.

Employment Type

Full-Time

About Company

Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.