Data Engineer – AWS + Hadoop
Share
Job Location:
Monthly Salary:
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
Job Title: Data Engineer AWS Hadoop
Location: Bangalore
Experience: 7 Years
About the Role
Were looking for a seasoned Data Engineer with hands-on expertise in AWS data services and the Hadoop ecosystem. You will design build and optimize batch/streaming data pipelines enable reliable data ingestion/processing and support analytics ML and BI use cases at scale.
Key Responsibilities
- Design and implement scalable ETL/ELT pipelines for batch and streaming workloads.
- Build data ingestion frameworks using Kafka/Kinesis and process data with Spark (PySpark/Scala).
- Develop and optimize data lakes and data warehouses on AWS (S3 Glue EMR Athena Redshift).
- Manage and tune Hadoop ecosystem components (HDFS Hive Spark Oozie/Airflow Sqoop).
- Model data (star/snowflake) manage schemas partitioning and metadata; ensure data quality (DQ checks).
- Implement data governance security and access controls (IAM Lake Formation encryption key management).
- Set up orchestrations and CI/CD for data jobs (Airflow/AWS Step Functions Jenkins/GitHub Actions).
- Monitor pipelines and optimize cost performance and reliability (CloudWatch logs metrics).
- Collaborate with Analytics/ML/BI teams; provide high-quality curated datasets and APIs/Views.
- Document solutions conduct code reviews and enforce engineering best practices.
Required Skills & Qualifications
- 7 years in Data Engineering with large-scale distributed data systems.
- Strong experience with AWS data stack: S3 Glue EMR Athena Lambda Redshift IAM CloudWatch.
- Hands-on with Hadoop ecosystem: HDFS Hive Spark (PySpark/Scala) Kafka Oozie/Airflow.
- Expertise in SQL (complex queries performance tuning) and data modeling.
- Practical knowledge of streaming (Kafka/Kinesis Spark Streaming/Structured Streaming).
- Experience with Python or Scala for data pipelines; Shell scripting.
- Familiarity with Orchestration (Airflow/AWS Step Functions) and CI/CD for data jobs.
- Strong understanding of security & governance (encryption PII handling RBAC Lake Formation).
- Proficient with version control (Git) and containers (Docker) for reproducible jobs.
- Excellent problem-solving communication and collaboration skills.
SYNECHRONS DIVERSITY & INCLUSION STATEMENT
Diversity & Inclusion are fundamental to our culture and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity Equity and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger successful businesses as a global company. We encourage applicants from across diverse backgrounds race ethnicities religion age marital status gender sexual orientations or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements mentoring internal mobility learning and development programs and more.
All employment decisions at Synechron are based on business needs job requirements and individual qualifications without regard to the applicants gender gender identity sexual orientation race ethnicity disabled or veteran status or any other characteristic protected by law.
Required Experience:
IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Chez Synechron, nous croyons en la puissance du numérique pour transformer les entreprises en mieux. Notre cabinet de conseil mondial combine la créativité et la technologie innovante pour offrir des solutions numériques de premier plan. Les technologies progressistes et les stratégie ... View more
