Data QA Lead
New York City, NY - USA
Job Summary
Job Title:Data QA Lead
Location: 100% remote
Snowflake Databricks ETL SQL Pytest/Python Data Lake 7 years Data QA experience
Key Responsibilities
- Lead Quality Assurance activities for data engineering and analytics projects.
- Design and execute test strategies plans and cases for data pipelines and platforms.
- Validate data transformations business rules and source-to-target mappings across Databricks Snowflake and Azure Data Factory (ADF).
- Perform end-to-end data testing including source-to-target reconciliation and validation across different layers of the data architecture.
- Conduct data quality testing including null validation duplicate checks row counts and referential integrity.
- Develop reconciliation frameworks and reusable automation frameworks for data testing.
- Create and execute complex SQL-based test cases to validate large datasets.
- Participate in Agile ceremonies and collaborate with Data Engineers Architects and business stakeholders.
- Mentor and provide guidance to approximately 2-3 junior QA resources.
Required Qualifications
Education:A Bachelors degree in Computer Science Information Technology Engineering or a related field is required.
Experience:A minimum of 7 years of experience in QA Data Warehouse Testing ETL Testing or Data Quality Validation is required. This must include 1-2 years of experience in a formal team leadership or mentorship capacity. Experience with data migration validation and reconciliation is necessary.
Technical Skills:Strong experience with Snowflake Databricks Azure Data Factory (ADF) Azure Data Lake and SQL is required. Candidates must possess expertise in Data Warehouse and ETL/ELT testing concepts including source-to-target mapping validation.
Preferred Qualifications
- Automation experience using Python and PyTest.
- Experience with Snowflake Data Quality Monitoring and Data Metric Functions (DMFs).
- Exposure to Power BI or other reporting validation.
- Knowledge of Data Governance and Data Quality frameworks.
- Familiarity with Azure DevOps and CI/CD pipelines.
- Experience with transferable automation tools such as Selenium.
Thanks
Required Skills:
Data/QASnowflakeDatabricksETLSQLPytestPythonData LakeData Warehouse TestingQA EngineeringETL TestingData Quality ValidationData MigrationValidationreconciliationAzure Data FactoryADFAzure Data LakeETL/ELTSnowflake Data QualityData Metric FunctionsPower BIData GovernanceData Quality frameworksAzure DevOpsCI/CD pipelinesSelenium