EY - GDS Consulting - AI and DATA - AWS Data Engineer - Senior
EY - GDS Consulting - AI and DATA - AWS Data Engineer - Senior
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
At EY youll have the chance to build a career as unique as you are with the global scale support inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better too. Join us and build an exceptional experience for yourself and a better working world for all.
AWS Senior Data Engineer
Experience Required:
- Minimum 5 years
Job Summary:
We are seeking a skilled Data Engineer with a strong background in data ingestion processing and storage. The ideal candidate will have experience working with various data sources and technologies particularly in a cloud environment. You will be responsible for designing and implementing data pipelines ensuring data quality and optimizing data storage solutions.
Key Responsibilities:
- Design develop and maintain scalable data pipelines for data ingestion and processing using Python Spark and AWS services.
- Work with on-prem Oracle databases batch files and Confluent Kafka for data sourcing.
- Implement and manage ETL processes using AWS Glue and EMR for batch and streaming data.
- Develop and maintain data storage solutions using Medallion Architecture in S3 Redshift and Oracle.
- Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
- Monitor and optimize data workflows using Airflow and other orchestration tools.
- Ensure data quality and integrity throughout the data lifecycle.
- Implement CI/CD practices for data pipeline deployment using Terraform and other tools.
- Utilize monitoring and logging tools such as CloudWatch Datadog and Splunk to ensure system reliability and performance.
- Communicate effectively with stakeholders to gather requirements and provide updates on project status.
Technical Skills Required:
- Proficient in Python for data processing and automation.
- Strong experience with Apache Spark for large-scale data processing.
- Familiarity with AWS S3 for data storage and management.
- Experience with Kafka for real-time data streaming.
- Knowledge of Redshift for data warehousing solutions.
- Proficient in Oracle databases for data management.
- Experience with AWS Glue for ETL processes.
- Familiarity with Apache Airflow for workflow orchestration.
- Experience with EMR for big data processing.
Mandatory:
- Strong AWS data engineering skills.
Good Additional Skills:
- Familiarity with Terraform for infrastructure as code.
- Experience with messaging services such as SNS and SQS.
- Knowledge of monitoring and logging tools like CloudWatch Datadog and Splunk.
- Experience with AWS DataSync DMS Athena and Lake Formation.
Communication Skills:
- Excellent verbal and written communication skills are mandatory for effective collaboration with team members and stakeholders.
EY Building a better working world
EY exists to build a better working world helping to create long-term value for clients people and society and build trust in the capital markets.
Enabled by data and technology diverse EY teams in over 150 countries provide trust through assurance and help clients grow transform and operate.
Working across assurance consulting law strategy tax and transactions EY teams ask better questions to find new answers for the complex issues facing our world today.
Employment Type
Full Time
