Test Manager Test Architect
Share
Job Location:
Eden Prairie, MN - USA
Monthly Salary:
Not Disclosed
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
Job Summary:
We are seeking a highly skilled and experienced Test Architect / Test Manager to drive data engineering and migration initiatives using Azure Databricks Python and PySpark. The ideal candidate will lead the team in testing of scalable data pipelines and oversee the migration of legacy systems to modern cloud-based architectures.
Key Responsibilities:
Manage Testing of end-to-end data pipelines using Azure Databricks Python and PySpark.
Collaborate with cross-functional teams including data scientists analysts and business stakeholders to understand data requirements.
Ensure data quality integrity and governance across all migration and transformation processes.
Mentor junior engineers and provide technical leadership within the team.
Develop reusable frameworks and automation tools to streamline migration and data processing tasks.
Stay updated with the latest trends and best practices in cloud data engineering and migration.
Required Skills & Qualifications:
Bachelors or Masters degree in Computer Science Engineering or related field.
12 years of experience in data engineering with expertise in Python and PySpark.
3 years of experience working with Azure Databricks and other Azure data services (e.g. Azure Data Lake Azure Synapse).
Proven experience in ETL and data migration projects including strategy execution and validation.
Strong understanding of distributed computing data modeling and ETL/ELT processes.
Experience with CI/CD pipelines and version control tools (e.g. Git Azure DevOps).
Excellent problem-solving and communication skills.
Ability to lead and manage multiple projects simultaneously.
Preferred Qualifications:
Azure certifications (e.g. Azure Data Engineer Associate).
Experience with other cloud platforms (AWS GCP) is a plus.
Familiarity with Delta Lake MLflow and other Databricks ecosystem tools.
Experience in Agile/Scrum environments.
We are seeking a highly skilled and experienced Test Architect / Test Manager to drive data engineering and migration initiatives using Azure Databricks Python and PySpark. The ideal candidate will lead the team in testing of scalable data pipelines and oversee the migration of legacy systems to modern cloud-based architectures.
Key Responsibilities:
Manage Testing of end-to-end data pipelines using Azure Databricks Python and PySpark.
Collaborate with cross-functional teams including data scientists analysts and business stakeholders to understand data requirements.
Ensure data quality integrity and governance across all migration and transformation processes.
Mentor junior engineers and provide technical leadership within the team.
Develop reusable frameworks and automation tools to streamline migration and data processing tasks.
Stay updated with the latest trends and best practices in cloud data engineering and migration.
Required Skills & Qualifications:
Bachelors or Masters degree in Computer Science Engineering or related field.
12 years of experience in data engineering with expertise in Python and PySpark.
3 years of experience working with Azure Databricks and other Azure data services (e.g. Azure Data Lake Azure Synapse).
Proven experience in ETL and data migration projects including strategy execution and validation.
Strong understanding of distributed computing data modeling and ETL/ELT processes.
Experience with CI/CD pipelines and version control tools (e.g. Git Azure DevOps).
Excellent problem-solving and communication skills.
Ability to lead and manage multiple projects simultaneously.
Preferred Qualifications:
Azure certifications (e.g. Azure Data Engineer Associate).
Experience with other cloud platforms (AWS GCP) is a plus.
Familiarity with Delta Lake MLflow and other Databricks ecosystem tools.
Experience in Agile/Scrum environments.
Key Skills
- Apache Commons
- Asset
- Art And Craft
- Concrete Pump
- Accident Investigation
