Manage and support a team of Quality Engineers responsible for QA strategy tooling and implementation across annotation workflows. Define and refine scalable processes and metrics to assess the quality consistency and relevance of labeled data. Partner with Engineering teams to build automated validation logic for detecting inconsistencies and data anomalies. Collaborate with Data Scientists and ML Engineers to analyze how data quality impacts model behavior and identify opportunities for data improvement. Lead multi-functional alignment on annotation QA standards and ensure feedback loops between quality guidelines and tooling. Own and evolve golden set evaluations consensus grading protocols and annotator quality tracking mechanisms. Conduct root cause analyses on quality issues and drive corrective actions in collaboration with upstream teams. Stay ahead of QA and data quality best practices and drive continuous improvement in tools and methodologies.
7 years of experience in data quality quality engineering or a related field including 2 years in a management or leadership role.
Extraordinary leadership communication and organizational skills.
Meticulous strong analytical and problem-solving skills
Knowledge of data quality concepts challenges and best practices
Proven track record of driving process improvements and managing multi-functional initiatives.
Experience with AI/ML/LLM. Familiarity with ML model development cycles and the role of human-labeled data in training and evaluation.
Experience with large-scale data operations or data-centric ML infrastructure.
Familiarity with human-in-the-loop evaluation data quality frameworks and annotation tooling platforms
Knowledge of data quality standards frameworks and governance best practices.
Experience designing and implementing automated QA checks and quality monitoring systems.
Strong data analysis skills; experience with scripting languages (e.g. Python) and data tools (e.g. SQL).
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.