Senior AssociateManager Data Engineer Consultant (Data Bricks) Technology Consulting

At EY we develop you with future-focused skills and equip you with world-class experiences. We empower you in a flexible environment and fuel you and your extraordinary talents in a diverse and inclusive culture of globally connected teams.

We work together across our full spectrum of services and skills powered by technology and AI so that business people and the planet can thrive together.

Were all in are you

Join EY and shape your future with confidence.

The opportunity

EY AI & Data is the data and advanced analytics capability within EY Asia-Pacific with over 500 specialist employees working across multiple industry sectors. You will be part of a dynamic team that implements information-driven strategies data platforms and advanced data analytics solution systems that help grow optimize and protect client organizations. We are seeking an experienced Data Engineer Consultant to join our team as a senior individual contributor
responsible for designing implementing and optimising robust data infrastructure and pipelines. This role offers the opportunity to work hands-on with cutting-edge AWS and Databricks technologies whilst providing technical guidance to team members and collaborating with stakeholders across the organisation on complex data engineering challenges.

Your key responsibilities

1) Data Architecture & Engineering

• Design and implement enterprise-scale data architectures including data lakes warehouses and real-time streaming platforms.
• Develop and maintain ETL/ELT pipelines that efficiently process large volumes of structured and unstructured data from diverse sources.
• Ensure data quality governance and security standards are embedded throughout all data engineering processes.

2) Technical Implementation

• Hands-on development of Databricks notebooks using PySpark Python and SQL for ETL automation.
• Create and optimise PySpark scripts for efficient data extraction transformation and loading from large datasets.
• Implement custom data manipulation validation and error handling solutions to enhance ETL robustness.

3) Technical Guidance

• Provide technical mentorship to junior data engineers and analysts.
• Lead code reviews establish best practices and drive adoption of modern data engineering tools and methodologies.
• Collaborate with cross-functional teams including data scientists analysts and software engineers to deliver integrated solutions.

4) Performance & Optimisation

• Monitor and optimise data pipeline performance implementing solutions for scalability and cost-effectiveness.
• Conduct testing debugging and troubleshooting of data transformation processes.
• Verify data integrity throughout pipeline stages and resolve complex technical issues.

To qualify for the role you must have

• Bachelors Degree in Computer Science Engineering Data Science or related field ORMasters Degree in Data Engineering Computer Science or MBA (preferred)
• 5-7 years of data engineering experience with AWS-native implementations
• Proven experience working as part of a data engineering team on ETL migration projects
• Hands-on experience implementing Databricks notebooks using PySpark Python and SQL for ETL automation
• Demonstrated expertise developing PySpark scripts for efficient data extraction transformation and loading from large datasets
• Experience utilising Python for custom data manipulation validation and error handling in ETL processes
• Proficiency employing SQL for complex joins aggregations and database operations within Databricks environments
• Track record of testing debugging and optimising data transformation processes for accuracy and performance
• Experience verifying data integrity throughout pipeline stages and resolving troubleshooting issues
• Proven ability collaborating with cross-functional teams to align ETL migration tasks with project goals and deadlines
• Experience in government sector data projects and compliance requirements (preferred)
• Experience in HR domain including workforce analytics payroll systems and employee data management (preferred)
• Experience providing technical mentorship to junior team members (preferred)
• Certified in:
  • AWS Certified Data Engineer - Associate (preferred)
  • AWS Certified Solutions Architect or AWS Certified Big Data - Specialty (preferred)
  • Databricks Certified Data Engineer Associate or Professional
  • Amazon Redshift or Snowflake certifications
  • Agile/Scrum Master certification (preferred)

Essential Skills:

1) Technical Skills:

• AWS cloud platform expertise (Redshift S3 Glue EMR Kinesis Lambda)
• Databricks platform proficiency with PySpark Python and SQL implementation
• Data warehouse design and implementation (Amazon Redshift Snowflake on AWS)
• ETL/ELT migration project experience and pipeline development
• Real-time streaming technologies (Kinesis Data Streams Kafka)
• Data exchange platforms and API integration (AWS Data Exchange Partner APIs)

2) Communication Skills:

• Clear written and verbal communication of technical concepts to non-technical
  stakeholders
• Presentation and reporting abilities for executive audiences
• Active listening to understand business requirements
• Stakeholder management across technical and business teams
• Meeting facilitation for technical design sessions

3) Leadership Skills:

• Technical mentorship and guidance for junior engineers
• Conflict resolution in cross-functional project environments
• Decision-making under pressure for critical data system issues
• Knowledge transfer and coaching abilities
• Change management for data platform migrations

4) Organisational Skills:

• Time management across multiple concurrent data projects
• Resource allocation for optimal pipeline performance
• Multi-tasking coordination of data engineering deliverables
• Attention to detail in data quality and governance processes
• Documentation of technical specifications and system architecture

5) Problem-Solving Skills:

• Critical thinking for complex AWS data architecture challenges
• Creative solution development for scalability and performance issues
• Issue escalation and resolution for production data systems
• Adaptability to evolving business and technical requirements
• Strategic planning for AWS data infrastructure roadmaps

6) Interpersonal Skills:

• Relationship building with business stakeholders and technical teams
• Negotiation of technical requirements and project timelines
• Emotional intelligence in managing diverse project teams
• Cultural awareness in global data initiatives
• Customer service orientation for internal data consumers

What we look for

Highly motivated individuals with excellent problem-solving skills and the ability to prioritize shifting workloads in a rapidly changing industry. An effective communicator youll be a confident team player that collaborates with people from various teams while looking to develop your career in a dynamic organization.

What working at EY offers

EY offers a competitive remuneration package where youll be rewarded for your individual and team are committed to being an inclusive employer and are happy to consider flexible working we offer:

• Continuous learning:Youll develop the mindset and skills to navigate whatever comes next.
• Success as defined by you:Well provide the tools and flexibility so you can make a meaningful impact your way.
• Transformative leadership:Well give you the insights coaching and confidence to be the leader the world needs.
• Diverse and inclusive culture:Youll be embraced for who you are and empowered to use your voice to help others find theirs.

Company description

EY is building a better working world by creating new value for clients people society and the planet while building trust in capital markets.

Enabled by data AI and advanced technology EY teams help clients shape the future with confidence and develop answers for the most pressing issues of today and tomorrow.

EY teams work across a full spectrum of services in assurance consulting tax strategy and transactions. Fueled by sector insights a globally connected multi-disciplinary network and diverse ecosystem partners EY teams can provide services in more than 150 countries and territories.

All in to shape the future with confidence.