Lead Kafka Engineer
Share
Job Location:
Monthly Salary:
Posted on:
1 hour ago
Vacancies:
1 Vacancy
Job Summary
Job Title: Lead Kafka Engineer / Confluent Platform Architect (Real-Time Streaming GCP)
Location:Bengaluru Chennai Hyderabad Pune Kolkata Lucknow Madurai Nagpur Vijayawada
Experience: 5 Years
Work Mode: Hybrid
Job Summary
We are seeking an experienced Kafka / Confluent Platform Architect to lead the design development and deployment of enterprise-grade real-time data streaming solutions. The ideal candidate will architect scalable streaming pipelines manage Kafka clusters across on-prem and GCP environments and establish best practices for governance security and performance. This role also involves technical leadership of offshore teams and automation using CI/CD and Infrastructure as Code.
Key Responsibilities
- Lead the design development and deployment of solutions using Apache Kafka and Confluent Platform
- Architect and implement real-time data pipelines and event-driven streaming applications
- Install configure manage and optimize Kafka clusters (on-premises and Google Cloud Platform)
- Ensure high availability fault tolerance scalability and performance tuning of Kafka environments
- Implement and manage Kafka Connect Schema Registry and ksqlDB for stream processing
- Define and enforce Kafka best practices for usage governance data standards and security
- Monitor cluster health and troubleshoot issues related to latency consumer lag and throughput
- Collaborate with cross-functional teams (onshore/offshore) to deliver robust streaming solutions
- Provide technical leadership mentor offshore team members and conduct code/design reviews
- Automate deployments and infrastructure using CI/CD pipelines and Terraform
Required Skills & Experience
- 8 years of experience in data engineering or platform engineering roles
- Strong hands-on expertise with Apache Kafka and Confluent Platform
- Experience building real-time/event-driven data pipelines
- Kafka cluster administration capacity planning and performance tuning
- Hands-on with Kafka Connect Schema Registry ksqlDB and stream processing
- Experience with GCP Kubernetes and Docker-based deployments
- Proficiency in Infrastructure as Code (Terraform) and CI/CD automation
- Strong understanding of Kafka security (SSL ACLs RBAC)
- Experience working in distributed onshore/offshore team models
- Excellent troubleshooting and problem-solving skills
