| Azure Data Engineer ADF and
PySpark and python need
|
Competencies: 10 years experience required
Digital : Microsoft Azure
Agile Way of Working
Digital : Python
Role Description:
Take ownership of stories and drive them to completion through all phases of the entire 84.51 SDLC.
This includes external facing and internal applications as well as process improvement activities such as
Develop distributed data processing data pipeline solutions
Orchestrate multi-step data transformation pipelines
Perform unit integration and regression testing on packaged code
Build transformation logic and code in an Object Oriented Programming style
Enhance CICD pipelines in the path to production
Create data quality checks for ingested and post processed data
Ensure data observability via alerting and monitoring of automated pipeline solutions
Maintain and enhance existing applications
Build cloud resources via infrastructure as code
Provide mentoring to junior team members
Participate in retrospective reviews
Participate in the estimation process for new work and releases
Bring new perspectives to problems
Be driven to improve yourself and the way things are done
Essential Skills:
Bachelors degree typically in Computer Science Management Information Systems Mathematics Business Analytics or another technically strong program.
2 years of proven professional data development experience
2 years developing with SQL
Proficient with software engineering best practices
Proficient with data pipeline orchestration
Proficient with automated testing (PyTest etc)
Proficient with VCS (Git GitHub)
Proficient using Python frameworks
Proficient with Object Oriented Programming principles
Proficient with Java and Sprint Framework
Experience with distributed data processing (PySpark and/or Snowpark)
Experience with Data Observability
Experience with Data Quality checks and processes
Experience with Cloud Technologies Services (Azure preferred GCP AWS)
Experience with CICD
Experience with dependency management (Conda venv etc)
Experience with debugging enterprise applications
Understanding of performance tuning enterprise processes
Understanding of Infrastructure as Code
Understanding SOLID principles
Understanding of Agile Principles (Scrum)
|
Azure Data Engineer ADF and PySpark and python need Competencies: 10 years experience required Digital : Microsoft Azure Agile Way of Working Digital : Python Role Description: Take ownership of stories and drive them to completion through all phases of the entire 84.51 SDLC. This includes e...
| Azure Data Engineer ADF and
PySpark and python need
|
Competencies: 10 years experience required
Digital : Microsoft Azure
Agile Way of Working
Digital : Python
Role Description:
Take ownership of stories and drive them to completion through all phases of the entire 84.51 SDLC.
This includes external facing and internal applications as well as process improvement activities such as
Develop distributed data processing data pipeline solutions
Orchestrate multi-step data transformation pipelines
Perform unit integration and regression testing on packaged code
Build transformation logic and code in an Object Oriented Programming style
Enhance CICD pipelines in the path to production
Create data quality checks for ingested and post processed data
Ensure data observability via alerting and monitoring of automated pipeline solutions
Maintain and enhance existing applications
Build cloud resources via infrastructure as code
Provide mentoring to junior team members
Participate in retrospective reviews
Participate in the estimation process for new work and releases
Bring new perspectives to problems
Be driven to improve yourself and the way things are done
Essential Skills:
Bachelors degree typically in Computer Science Management Information Systems Mathematics Business Analytics or another technically strong program.
2 years of proven professional data development experience
2 years developing with SQL
Proficient with software engineering best practices
Proficient with data pipeline orchestration
Proficient with automated testing (PyTest etc)
Proficient with VCS (Git GitHub)
Proficient using Python frameworks
Proficient with Object Oriented Programming principles
Proficient with Java and Sprint Framework
Experience with distributed data processing (PySpark and/or Snowpark)
Experience with Data Observability
Experience with Data Quality checks and processes
Experience with Cloud Technologies Services (Azure preferred GCP AWS)
Experience with CICD
Experience with dependency management (Conda venv etc)
Experience with debugging enterprise applications
Understanding of performance tuning enterprise processes
Understanding of Infrastructure as Code
Understanding SOLID principles
Understanding of Agile Principles (Scrum)
|
View more
View less
Job Location:
Monthly Salary:
Not Disclosed
Posted on:
8 hours ago
Vacancies:
1 Vacancy
