EY GDS Consulting AI And DATA Data Engineer-Staff

Staff Data Engineer

The Opportunity:

Join our dynamic team as a Senior Data Engineer where you will play a critical role in designing building and optimizing scalable data solutions that drive business insights and innovation. You will work with cutting-edge technologies in a collaborative Agile environment contributing to the full data lifecycle from ingestion to transformation and deployment.

Key Responsibilities:

• Design develop and manage robust CI/CD pipelines to streamline the product development lifecycle and ensure seamless deployment of data solutions.
• Perform data mapping and integration between diverse source and target systems to support analytics and business intelligence needs.
• Build and optimize Data & AI solutions including data engineering pipelines data modeling and performance tuning with 3-5 years of relevant experience.
• Develop high-quality production-level code primarily in Python PySpark and SQL ensuring maintainability and scalability.
• Implement enterprise-scale production deployments applying DevOps best practices with hands-on experience in Git version control.
• Collaborate effectively within Agile teams contributing to sprint planning reviews and continuous improvement.
• Apply strong knowledge of Data Governance and Data Quality standards to maintain data integrity and compliance.

Technical Skills & Experience:

• Extensive experience in building data ingestion frameworks and scalable data pipelines.
• Proven expertise in AWS cloud services including S3 EC2 Glue Lambda and Secrets Manager for secure and efficient data processing.
• Hands-on experience with Databricks for unified analytics and collaborative data engineering.
• Proficient in DBT (Data Build Tool) for data transformation and modeling within modern data stacks.
• Skilled in orchestrating workflows using Apache Airflow to automate complex data pipelines.
• Strong programming skills in Python PySpark Spark and Scala for big data processing.
• Experience with software version control and CI/CD tools such as Git Jenkins and Apache Subversion.
• AWS / Databricks certifications or equivalent professional technical certifications are highly desirable.
• Familiarity with cloud and enterprise integration technologies.
• Demonstrated ability to write efficient Spark jobs and optimize performance in large-scale environments.
• Excellent analytical skills with deep knowledge of SQL and data querying.
• Minimum 3 years of experience working in very large data warehousing environments.
• Strong communication skills both written and verbal to effectively collaborate with cross-functional teams.
• At least 3 years of experience with data warehouse architectures ETL/ELT processes and reporting/analytics tools.
• Experience in Python and/or Java development in data engineering contexts.
• Familiarity with Big Data ecosystems including EMR Hadoop Databricks Hive and Pyspark.