QA Testing Specialist Data Engineer

Role Description

SMBC is seeking QA Testing Specialist- Data Engineer is responsible for ensuring the accuracy reliability and integrity of enterprise data platforms built on Databricks. This role combines strong data engineering knowledge with deep quality assurance expertise focusing on validating complex ETL pipelines data transformations and largescale datasets.

The position plays a critical role in enabling datadriven decisionmaking by ensuring that analytical and financial data is complete accurate and productionready before release. The role partners closely with Data Engineers Business Analysts Product Owners and Project Managers across globally distributed teams.

This role will report to the Director QA Testing Manager

Role Objectives: Delivery

Data Quality & Validation:

• Perform endtoend data validation including sourcetotarget mapping reconciliation and completeness checks.
• Validate data transformations across Databricks Medallion Architecture (Bronze Silver Gold).
• Ensure data quality standards are met prior to production releases.

Test Strategy & Execution:

• Participate in requirements analysis and solution design reviews.
• Define test scope scenarios riskbased coverage and overall test strategies.
• Design and execute test cases for complex ETL pipelines and Perform regression testing and manage test data conditions.

Data Validation & Automation:

• Write and execute advanced SQL queries (CTEs window functions complex joins).
• Develop Python validation scripts using PySpark and Pandas to support and enhance data test automation efforts where applicable.
• Monitor Databricks jobs and workflow executions.
• Investigate pipeline failures data inconsistencies and quality issues.
• Analyze defects both functionally and technically prior to assignment.

Defect Management & Collaboration:

• Log track and manage defects using JIRA ensuring highquality defect documentation. Collaborate closely with Data Engineers Architects and Business teams to resolve issues.
• Provide clear and timely communication on quality risks test progress and release readiness and Escalate issues through defined governance and quality escalation channels.

Qualifications and Skills

• 5 years of experience in Quality Assurance testing or relevant Data Engineering development roles.
• Demonstrated expertise in Advanced SQL including CTEs window functions and complex joins for largescale data validation.
• 6 years of experience in test management strategy; experience in the securities or financial services industry
• 3 years of experience in designing solutions in process flow business logic and UI.
• Strong experience within the Azure data ecosystem with handson expertise in Databricks notebooks and cluster management.
• Experience building or supporting data test automation frameworks with exposure to CI/CD pipelines and Gitbased version control.
• Hands-on manual and automated test development and design experience leading medium to large scales or enterprise-wide projects and production implementations.
• Strong command of Python particularly PySpark and Pandas for comprehensive data validation and analysis.
• Thorough understanding of ETL processes data pipelines and data warehouse architecture with proven experience in ETL/Data Warehouse testing including end sourcetotarget validation.
• Prior experience working directly with business users and finance stakeholders including exposure to Oracle or similar enterprise financial platforms.
• Strong analytical criticalthinking and problemsolving skills.
• Excellent written and verbal communication skills with the ability to articulate technical concepts clearly.

Ability to work independently while collaborating effectively across distributed and global teams.

Additional Requirements