AIML with testing Experience

Title: AI/ML Engineer with Testing Experience

Location: SFO CA

Hybrid Model 3 days onsite

Key Responsibilities

Strategy & Leadership

• Define and own the end-to-end quality strategy test approach and release readiness criteria across squads.
• Lead mentor and grow a team of QA engineers; establish career paths skill matrices and a culture of continuous improvement.
• Drive shift-left testing practices ensuring quality gates in PRs CI/CD and design reviews.

Automation & Frameworks

• Architect and maintain Python-based automation frameworks (e.g. PyTest Selenium Playwright Robot Framework) for UI API integration and end-to-end tests.
• Implement data-driven and behavior-driven testing (BDD) with tools like Behave/Cucumber where applicable.
• Standardize test design patterns (Page Object Screenplay fixtures test data services) and enforce code quality (linting type hints reviews).

AI-Enabled Quality

• Integrate AI-assisted testing (e.g. intelligent test case generation flaky test detection failure clustering anomaly detection in logs).
• Collaborate with Data Science/ML teams to validate ML models including dataset integrity bias checks model drift monitoring and functional/non-functional validation of inference services.
• Evaluate and where appropriate adopt AI-powered test platforms (e.g. Mabl Testim) or build in-house utilities using scikit-learn/PyTorch/TensorFlow for prioritization and defect prediction.

CI/CD & DevOps Quality

• Embed tests into CI/CD pipelines (GitHub Actions/Jenkins/Azure DevOps/GitLab CI) enabling parallelization shards and caching.
• Define and monitor quality gates (code coverage mutation testing static analysis performance thresholds).
• Orchestrate environment management using Docker/Kubernetes service mocks test data services and synthetic data generation.

Quality Operations

• Establish metrics and reporting (DRE escape rate MTTR flaky rate coverage defect aging) with dashboards (Grafana/PowerBI).
• Lead root cause analyses and drive corrective/preventive actions (CAPA).
• Partner with Product and Engineering on release planning risk assessment and sign-off.

Required Skills & Experience

• Python: Advanced proficiency; building robust test frameworks utilities parsers and CLI tools; strong OOP and familiarity with concurrency (asyncio) typing packaging.
• Automation: Hands-on with PyTest Selenium/Playwright Requests Robot Framework; API testing (REST/GraphQL) contract testing (Pact) and service virtualization/mocking.
• AI/ML Knowledge: Understanding of ML lifecycle (data prep model training/evaluation drift monitoring) and AI-assisted testing concepts (prioritization flaky test detection anomaly detection). Ability to use pandas NumPy scikit-learn for analytics.
• CI/CD & DevOps: Experience integrating tests into pipelines containerized testing environment orchestration and test parallelization.
• Performance & Reliability: Exposure to load/stress testing (JMeter/Locust/k6) and reliability checks (resilience chaos testing basics).
• Cloud & Tools: Familiarity with AWS/GCP/Azure Docker/K8s; version control (Git) issue tracking (Jira/Azure Boards).
• Leadership: Proven experience leading QA teams setting standards coaching and delivering across multiple releases.