Sr. QA Data Engineer

Job Title: Senior QA Data Engineer
Location: Remote
Budget: 18- 25 LPA
No of positions: 1
Shift: 11:00 AM to 8:00 PM IST
Experience: 5 Years

Position Overview :
We are looking for an experienced Sr. QA Data Engineer to join our Data Engineering team and take ownership of the quality accuracy consistency and reliability of our data pipelines and data-driven systems. The ideal candidate will have strong hands-on experience in Data Testing ETL/ELT Testing Database Testing SQL Python Apache Spark and Databricks. The candidate should be comfortable validating complex data transformations investigating data discrepancies and testing end-to-end data workflows across multiple sources processing layers and downstream systems. You will work closely with Data Engineers Software Engineers Database Architects Data Analysts Data Scientists and Product teams to ensure data is processed correctly and delivered reliably to downstream consumers. This role requires strong analytical and problem-solving skills attention to detail and the ability to
independently understand complex data flows develop comprehensive test strategies automate data validation investigate defects and drive quality across the data engineering ecosystem.

What You'll Be Doing
Design and execute functional integration regression and end-to-end testing for data pipelines and data platforms.
Validate ETL/ELT workflows data transformations business rules and source-to-target data accuracy.
Test Apache Spark/PySpark Databricks and Delta Lake data processing workflows.
Perform advanced SQL-based data validation reconciliation and data-quality testing.
Validate data for accuracy completeness consistency uniqueness integrity and schema compliance.
Test APIs and integrations across data sources databases pipelines and downstream systems.
Develop and maintain Python/SQL-based test automation for data validation and regression testing.
Design and execute positive negative edge-case failure retry and recovery scenarios across data workflows.
Investigate data discrepancies and work with Data Engineers Developers Analysts and Data Scientists to identify root causes and resolve issues.
Review requirements technical designs and data flows to identify QA risks dependencies and coverage gaps early in the development lifecycle.
Maintain comprehensive test cases automation suites defect documentation test evidence and release validation for assigned data products.
Drive continuous improvement of QA processes data validation frameworks automation coverage and overall data quality.

Qualifications
Bachelor's degree in Computer Science Engineering Information Technology or a related field.
5 years of relevant QA/testing experience preferably working with Data Engineering Big Data or data-intensive applications.
Strong hands-on experience in ETL/ELT testing database testing and data validation.
Advanced SQL skills including complex joins subqueries CTEs window functions aggregations and data reconciliation.
Practical experience testing Apache Spark/PySpark data processing workflows.
Hands-on experience with Databricks and Delta Lake.
Strong understanding of Big Data concepts distributed data processing data pipelines and data architecture.
Experience validating Parquet JSON CSV and other structured and semi-structured data formats.
Strong programming/scripting skills in Python for test automation and data validation.
Experience with API and integration testing.
Working knowledge of Linux/Unix environments and shell scripting.
Understanding of Data Lakes Lakehouse architecture Data Warehousing and ETL/ELT pipelines.
Experience with Git defect management tools and CI/CD-based testing workflows.
Experience working in an Agile/Scrum environment.
Strong analytical troubleshooting and root-cause analysis skills.
Ability to work independently take ownership and collaborate effectively with cross-functional teams.
Excellent written and verbal communication skills in English.

Required Skills:

QA Data Engineer Testing Advanced SQL Python Apache Spark Databricks Delta Lake ETL/ELT Database Testing API/integration testing Linux/Unix shell scripting Git CI/CD Agile Jira Test Automation