Job Overview:
We are seeking an experienced Senior Devops Engineer to lead our Data DevOps team in building managing and optimizing high-scale secure and reliable big data platforms. The ideal candidate will combine strong technical expertise with proven leadership skills driving best practices in cloud infrastructure automation CI/CD and big data technologies. This role involves managing cross-functional priorities mentoring engineers and ensuring delivery of high-quality scalable data solutions.
Key Responsibilities:
Design and implementation of large-scale cloud and data infrastructure ensuring scalability performance and security.
Collaborate closely with Data Engineering Data Science Analytics and Product teams to deliver efficient and reliable data platforms.
Oversee operations and optimization of AWS-based infrastructure including VPC EC2 S3 EMR EKS SageMaker Lambda CloudFront CloudWatch and IAM. Manage and scale big data platforms leveraging Kafka Hive HMS Apache Ranger Apache Airflow EMR Spark Trino Looker and Jupyter Notebooks.
Implement and maintain CI/CD pipelines and infrastructure automation using Terraform Ansible and CloudFormation.
Ensure system observability proactive incident management and SLA adherence.
Champion cloud security best practices including API security TLS/HTTPS and access control policies.
Required Qualifications
Experience: 3-6 years in DevOps including at least 4 years in a leadership role.
Proven track record in managing high-scale big data infrastructure
Strong hands-on experience with AWS services and infrastructure automation tools (Terraform Ansible CloudFormation).
Deep knowledge and hands-on experience with Kafka Hive HMS Apache Ranger Apache Airflow EMR Spark Trino Jupyter Notebooks and Looker.
Proficiency in Kubernetes/EKS Docker ECS and CI/CD tools.
Strong understanding of networking cloud security and compliance requirements.
Excellent communication stakeholder management and decision-making skills.
Exposure to SQL and data query optimization is an advantage.