Data Quality Tester

JD for a skilled Quality Assurance (QA) Engineer / Lead (B3):

Data Quality with 7 years of hands-on experience in data quality testing within large-scale data analytics projects. The ideal candidate will have strong expertise in Azure Cloud Databricks PySpark and Informatica Data Marketplace to ensure accuracy reliability and compliance of enterprise-grade data platforms. The QA Engineer will play a critical role in defining data quality frameworks test strategies automation and governance standards to support data-driven decision-making.

Key Responsibilities:

Test Strategy & Planning:

Design and implement end-to-end data quality test strategies for data ingestion transformation and consumption layers in Azure-based data analytics projects.

Define data validation frameworks to ensure completeness consistency accuracy timeliness and integrity of data across the pipeline.

Collaborate with business analysts data engineers and solution architects to translate business rules into testable requirements.

Data Quality Testing:

Perform ETL / ELT testing for pipelines built using Databricks (PySpark Delta Lake) and Informatica Data Marketplace.

Develop test scripts in PySpark/Python to validate large-scale data sets against source and target systems.

Validate data transformations aggregations joins schema mappings and business rules in Databricks notebooks and pipelines.

Conduct regression testing backward compatibility testing and end-to-end workflow validation on Azure services.

Implement data quality automation frameworks using PySpark Python and Azure DevOps pipelines.

Develop reusable test scripts validation utilities and SQL-based data checks for faster execution and reporting.

Leverage Informatica Data Marketplace and Data Quality rules to enforce governance and metadata validation.

Validate data lineage metadata and security policies across Azure Data Lake Databricks and Informatica layers.

Monitor and report data quality KPIs (e.g. % data completeness error rate exception trends).

Collaboration & Governance:

Work closely with data engineers data architects business analysts and product owners to align QA processes with business requirements.

Document test cases test execution results defects and traceability matrices in line with project governance standards.

Support user acceptance testing (UAT) by providing test data validation reports and issue resolutions.

Required Skills & Qualifications:

5 years of experience in Quality Assurance / Data Quality Testing within data analytics or data engineering projects.

Strong expertise in Databricks PySpark SQL and Azure Data Lake/Synapse.

Hands-on experience with Informatica Data Marketplace / Informatica Data Quality (IDQ).

Proficiency in Python/PySpark scripting for automation of large-volume data validation.

Strong knowledge of ETL/ELT validation data profiling data reconciliation and lineage tracking.

Experience with Azure Cloud services (Data Factory Synapse Key Vault DevOps CI/CD).

Familiarity with data governance metadata management and master data management (MDM) principles.

Strong analytical and problem-solving skills with attention to detail.

Excellent communication skills to work in cross-functional teams.