Azure Data Engineer- Senior Associate
Share
Job Location:
Monthly Salary:
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
Industry/Sector
Not ApplicableSpecialism
Platform Engineering & ArchitectureManagement Level
Senior AssociateJob Description & Summary
At PwC our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights enabling informed decision-making and driving business growth.In data engineering at PwC you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines data integration and data transformation solutions.
Job Summary:
As an Azure Data Engineer you will play a crucial role in designing building and maintaining our enterprise data platform on the Microsoft Azure cloud. Your primary responsibility will be developing scalable reliable and efficient data pipelines to ingest transform and serve data to business analysts data scientists and operational systems. This role demands a deep understanding of modern data architecture a passion for automation and hands-on expertise with the Azure data stack.
Meaningful work youll be a part of:
As a Senior Associate youll work as part of a team of problem solvers helping to solve complex business issues from strategy to execution:
Architect design and own the development of robust end-to-end data pipelines for our production environment utilizing Azure Data Factory Azure Databricks and Azure Synapse Analytics.
Lead the design and implementation of advanced data models within our Azure Data Lake. Champion the Medallion Architecture (Bronze/Silver/Gold) and build complex business logic into scalable efficient transformations using PySpark and Delta Lake.
Develop ETL/ELT processes to ingest data from a wide variety of sources including relational databases (SQL Oracle) APIs and streaming data feeds.
Implement data models transformations and cleansing routines to prepare data for analytics.
Automate data workflows and implement CI/CD practices for data pipelines using Azure DevOps or GitHub Actions to ensure quality and rapid deployment.
Monitor troubleshoot and optimize data solutions for performance scalability and cost-effectiveness.
Proactively identify and resolve performance bottlenecks in data pipelines. This includes hands-on Spark job tuning query optimization data partitioning strategies and implementing cost-control measures across our Azure data services.
Work closely with data analysts data scientists and business stakeholders to understand their data requirements and deliver high-quality data solutions.
Mentor junior data engineers through code reviews paired programming and technical guidance. Establish and document coding standards best practices and reusable frameworks to elevate the entire teams technical capabilities.
Partner directly with data scientists analysts and business leaders to translate complex business requirements into robust scalable and reliable technical solutions.
Ensure data quality integrity and governance by implementing best practices data validation checks and robust documentation.
Experiences and skills youll use to solve:
Microsoft Azure Data Engineer Associate (DP-203) certification is or the ability to obtain it within the first 2 months.
5-7 years of dedicated hands-on experience in a data engineering role with a proven track record of delivering complex projects to production.
Deep hands-on expertise and production experience with core Azure data services: Azure Data Factory (ADF) Azure Databricks and Azure Data Lake Storage (ADLS Gen2).
Expert-level proficiency in SQL and Python. You must have extensive production-level experience with PySpark for large-scale distributed data processing.
Proven experience designing and implementing end-to-end data solutions on Azure demonstrating a strong grasp of modern ELT patterns data warehousing (star schema) and data lake architectures (e.g. Medallion).
Demonstrated ability to troubleshoot and solve complex ambiguous data problems in a production environment including performance tuning and root cause analysis.
Bachelors degree in computer science Engineering Information Technology or related field.
Travel Requirements
Not SpecifiedJob Posting End Date
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
At PwC, our purpose is to build trust in society and solve important problems. We’re a network of firms in 155 countries with over 284,000 people who are committed to delivering quality in assurance, advisory and tax services. Find out more and tell us what matters to you by vis ... View more
