EY - GDS Consulting - AI and DATA - Pyspark
EY - GDS Consulting - AI and DATA - Pyspark
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
At EY youll have the chance to build a career as unique as you are with the global scale support inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better too. Join us and build an exceptional experience for yourself and a better working world for all.
EY GDSData and Analytics (D&A) PYSPARK GIG
As part of our EYGDS D&A (Data and Analytics) team we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking Insurance Manufacturing Healthcare Retail Manufacturing and Auto Supply Chain and Finance.
The opportunity
We are currently seeking a seasoned Azure Data Engineer with proven experience in Databricks PySpark to join our team of professionals. The successful candidate will play a key role in our Metadata Management team providing cuttingedge data management strategies and leveraging their strong analytical skills to solve complex problems.
Primary Skills / Must have:
- Should have strong programming skills in Python.
- Experience in creating large scale data processing pipelines using a Python and Spark based framework.
- Work with different aspects of the Spark ecosystem including Spark SQL DataFrames Datasets and Streaming
- Should possess strong SQL skills.
- Excellent understanding of Unix ecosystem and should have experience in creating the shell scripts.
- Excellent understanding of Hive/Hadoop ecosystem.
- Solid Understanding of data engineering concepts and best practices.
- Excellent understanding of Job Scheduling mechanisms like Autosys TWS.
- Excellent problem solving and analytical skills.
- Excellent verbal and written communication skills.
- Experience in optimizing large data loads.
Secondary Skills / Desired skills
- Exposure to an Agile Development environment would be a plus.
- Strong understanding of Data warehousing domain.
- Ability to architect an ETL solution and data conversion strategy.
- Good understanding of dimensional modelling.
Roles and Responsibilities
As an ETL/ Python developer the candidate is expected to
- Design and development of highly optimized and scalable ETL applications using Python and Spark.
- Undertaking endtoend project delivery (from inception to postimplementation support) including review and finalization of business requirements creation of functional specifications and/or system designs and ensuring that endsolution meets business needs and expectations.
- Development of new transformation processes to load data from source to target or performance tuning of existing ETL code (mappings sessions).
- Analysis of existing designs and interfaces and applying design modifications or enhancements
- Coding and documenting data processing scripts and stored procedures.
- Providing business insights and analysis findings for adhoc data requests
- Testing software components and complete solutions (including debugging and troubleshooting) and preparing migration documentation.
- Providing reportingline transparency through periodic updates on project or task status.
- Should have 57 years work exp.
EY Building a better working world
EY exists to build a better working world helping to create longterm value for clients people and society and build trust in the capital markets.
Enabled by data and technology diverse EY teams in over 150 countries provide trust through assurance and help clients grow transform and operate.
Working across assurance consulting law strategy tax and transactions EY teams ask better questions to find new answers for the complex issues facing our world today.
Employment Type
Full Time
