Senior / Lead - Data Engineer - AWS
Share
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
We are looking for an experienced Senior Data Engineer with a strong foundation in Python SQL and Spark and hands-on expertise in AWS Databricks. In this role you will build and maintain scalable data pipelines and architecture to support analytics data science and business intelligence initiatives. Youll work closely with cross-functional teams to drive data reliability quality and performance
Qualifications :
Responsibilities:
- Design develop and optimize scalable data pipelines using Databricks in AWS such as Glue S3 Lambda EMR Databricks notebooks workflows and jobs.
- Building data lake in WS Databricks.
- Build and maintain robust ETL/ELT workflows using Python and SQL to handle structured and semi-structured data.
- Develop distributed data processing solutions using Apache Spark or PySpark.
- Partner with data scientists and analysts to provide high-quality accessible and well-structured data.
- Ensure data quality governance security and compliance across pipelines and data stores.
- Monitor troubleshoot and improve the performance of data systems and pipelines.
- Participate in code reviews and help establish engineering best practices.
- Mentor junior data engineers and support their technical development.
Additional Information :
Qualifications
Requirements
- Bachelors or masters degree in computer science Engineering or a related field.
- 5 years of hands-on experience in data engineering with at least 2 years working with AWS Databricks.
- Strong programming skills in Python for data processing and automation.
- Advanced proficiency in SQL for querying and transforming large datasets.
- Deep experience with Apache Spark/PySpark in a distributed computing environment.
- Solid understanding of data modelling warehousing and performance optimization techniques.
- Proficiency with AWS services such as Glue S3 Lambda and EMR.
- Experience with version control Git or Code commit
- Experience in any workflow orchestration like Airflow AWS Step funtions is a plus.
Remote Work :
No
Employment Type :
Full-time
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Blend360 is an award-winning provider of data, analytics, and talent solutions for Fortune 500 companies. The company has made the Inc. 5000 list of Fastest Growing Companies every year they have been in business and has been awarded a world-class ranking in client satisfaction for th ... View more
