Data Engineer-3
Data Engineer-3
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Company Name: Cashfree Payments Pvt Ltd
Job Title: Data Engineer III
Location: Bengaluru
Experience: 6 years
Employment Type: Fulltime
About the Role
We are looking for an experienced Senior Data Engineer (Data Engineer III) to join our data engineering team. The ideal candidate will have deep expertise in Redshift ClickHouse and other modern data technologies with a strong background in data lake architectures Apache Hudi Spark Flink and Airflow. This role requires handson experience in data modeling building scalable data systems from the ground up and mentoring junior engineers.
Key Responsibilities
Design & Develop: Architect and implement scalable data pipelines using Spark Flink Airflow and Apache Hudi to process largescale data efficiently.
Database & Storage Optimization: Work extensively with Redshift ClickHouse and other databases optimizing query performance storage and cost efficiency.
Data Lake & ETL Development: Build and manage data lakes and ingestion frameworks using AWS services such as S3 Glue and EMR.
Data Modeling & System Design: Design and implement robust data models for structured and semistructured datasets ensuring performance scalability and maintainability.
Realtime & Batch Processing: Develop and maintain both realtime and batch data pipelines using technologies like Flink Spark Streaming and Kafka.
AWS & Cloud Infrastructure: Leverage AWS services (S3 Redshift Lambda ECS etc. to build a scalable and costefficient data ecosystem.
Automation & Orchestration: Manage workflow automation and orchestration using Apache Airflow.
Mentorship: Guide and mentor junior engineers conduct code reviews and contribute to best practices for data engineering.
Monitoring & Optimization: Implement observability (monitoring alerting logging) for data pipelines to ensure reliability and performance.
Required Qualifications
6 years of experience in data engineering big data processing and cloudbased data architectures.
Strong expertise in Redshift ClickHouse and other distributed databases.
Experience with Apache Spark Flink Apache Hudi and Airflow.
Deep knowledge of data modeling schema design and database optimization.
Handson experience in building data pipelines on AWS (S3 Glue Redshift EMR Lambda etc..
Proficiency in Python SQL and Scala for data engineering tasks.
Experience in mentoring and leading junior engineers in best practices.
Strong problemsolving skills and ability to work in fastpaced environments.
Preferred Qualifications
Experience with CI/CD pipelines for data engineering workflows.
Familiarity with streaming data technologies (Kafka Kinesis Pulsar etc..
Prior experience with Terraform Kubernetes or serverless architectures.
Exposure to data governance security and compliance best practices.
Employment Type
Full-Time
