QA Manager, Software Engineering

At Penguin Solutions (Nasdaq: PENG) The AI Factory Platform Company were building a team of innovators who thrive on collaboration creativity and the opportunity to help shape the future of AI. As part of the AI technology revolution our teams design build deploy and manage AI factories for enterprises sovereign AI initiatives and neocloud providers worldwide.

Headquartered in Silicon Valley California Penguin Solutions operates globally through a network of R&D manufacturing and sales locations. For nearly three decades we have operated at the intersection of memory and AI/HPC infrastructure. That engineering expertise positions us to power the next generation of AI workloads from training to inference and agentic AI at scale.

Penguin Solutions brings together differentiated infrastructure software advanced memory compute systems end-to-end services and industry-leading partner solutions in a full-stack AI factory platform designed to help customers deploy and scale AI workloads with speed and precision.

At Penguin Solutions we value ideas over hierarchy and believe in servant leadership where leaders enable teams to do their best work. We empower employees to take ownership drive innovation and grow through challenging work continuous learning and exposure to advanced AI tools and technologies. With flexibility where it matters and a strong focus on outcomes Penguin Solutions is a place to do your best work grow your career and make a meaningful impact.

Job Overview

We are seeking a Manager Quality Engineering to lead quality engineering for ClusterWareAI a critical component of the Penguin Solutions AI Factory Platform.

ClusterWareAI enables automated provisioning orchestration monitoring and lifecycle management of large-scale AI this role you will define and drive the quality strategy for a complex distributed platform spanning GPU infrastructure compute networking storage Kubernetes workload orchestration and AI-enabled operations.

This is a hands-on technical leadership role. You will lead and develop a Quality Engineering team while partnering closely with Software Engineering Product Management Platform Engineering DevOps Support and Systems Integration teams to build quality into every stage of the product lifecycle.

The ideal candidate brings a strong product-engineering mindset deep experience with test automation and distributed systems and a passion for delivering reliable cloud and infrastructure software at scale.

Responsibilities

• Own the end-to-end Quality Engineering strategy for ClusterWareAI across functional integration system regression performance scalability reliability security and resilience testing.
• Lead mentor and grow a high-performing QE team while establishing strong technical automation and execution practices.
• Drive a shift-left automation-first quality culture partnering with Engineering Product Platform DevOps Support and Systems Integration throughout the product lifecycle.
• Design scalable test automation frameworks across APIs UI CLI control-plane services infrastructure integrations and end-to-end workflows integrated with CI/CD quality gates.
• Define test strategies for GPU infrastructure bare metal Kubernetes networking storage workload orchestration and distributed systems at scale.
• Validate infrastructure lifecycle workflows including provisioning configuration upgrades rollback deprovisioning monitoring health management and automated remediation.
• Drive performance scalability HA fault-injection longevity and failure/recovery testing across large-scale distributed environments.
• Define validation for multi-tenancy tenant isolation security access controls resource governance and workload separation.
• Establish quality and release-readiness metrics covering automation escaped defects test reliability performance scalability and product risk.
• Partner with Support and Engineering on customer issues root-cause analysis and converting field failures into automated regression coverage.
• Evaluate and adopt AI-assisted quality engineering approaches to improve test generation failure analysis defect triage and engineering productivity.

AI / Agentic Quality Engineering

• Own the AI Quality Engineering strategy for ClusterWareAIs AI Platform capabilities.
• Build automated AI evaluation frameworks and benchmark suites covering accuracy groundedness consistency reliability latency cost and task success.
• Establish LLM and agent evaluation methodologies using golden datasets scenario-based testing synthetic data LLM-as-a-Judge rubric-based evaluation and human evaluation.
• Test RAG tool calling multi-step agents planning/execution agent collaboration memory and infrastructure-aware reasoning.
• Develop AI regression pipelines to detect degradation caused by changes to models prompts agents tools knowledge sources and orchestration logic.
• Drive AI safety and adversarial testing including hallucination prompt injection unauthorized tool use policy violations data leakage and unsafe infrastructure actions.
• Validate agent guardrails policy enforcement approvals authorization boundaries tenant isolation auditability and human-in-the-loop controls.
• Establish production AI quality metrics including task success rate hallucination rate tool-call accuracy agent failure rate escalation rate latency and cost efficiency.
• Develop scenario-based autonomous infrastructure testing to validate that agents correctly observe reason plan obtain approval execute authorized actions verify outcomes and recover safely from failures.

Qualifications

• Bachelors degree in Computer Science Engineering or equivalent practical experience.
• 12 years in Quality Engineering Software Engineering Test Automation or related engineering roles with 3 years leading QE teams.
• Proven experience owning quality for a commercial software cloud infrastructure or distributed systems platform.
• Strong programming and automation skills preferably Python with experience building scalable automation frameworks.
• Strong experience testing APIs distributed systems cloud platforms Kubernetes infrastructure software and complex enterprise applications.
• Experience with CI/CD performance scalability reliability HA security and failure/recovery testing.
• Strong Linux debugging observability log analysis and distributed-systems troubleshooting skills.
• Demonstrated ability to provide technical leadership coaching and team development.
• Hands-on experience with AI/LLM quality engineering AI evaluation or testing AI-powered applications.
• Experience testing LLM applications RAG AI agents multi-agent systems or agentic workflows.
• Experience with AI evaluation methodologies including golden datasets benchmark testing LLM-as-a-Judge RAG evaluation adversarial testing and red teaming.
• Experience with frameworks such as DeepEval Ragas Promptfoo LangSmith/LangChain evaluation NVIDIA NeMo Guardrails or equivalent.
• Understanding of AI failure modes including hallucination prompt injection non-determinism tool-use errors context failures model regression and unsafe agent behavior.
• Experience integrating AI evaluation and regression testing into CI/CD pipelines.

Preferred Qualifications

• Experience with AI infrastructure GPU cloud/neocloud platforms or large-scale cloud infrastructure.
• Experience with Kubernetes container orchestration and multi-tenant cloud platforms.
• Experience with GPU infrastructure including GPU lifecycle drivers firmware health monitoring and heterogeneous accelerator environments.
• Experience with high-performance networking technologies such as InfiniBand RDMA/RoCE high-speed Ethernet and distributed storage.
• Experience with bare-metal provisioning infrastructure lifecycle management and fleet-scale automation.
• Experience with workload orchestration and scheduling technologies such as Kubernetes and Slurm.
• Experience with infrastructure automation and Infrastructure as Code technologies such as Ansible.
• Experience with cloud platform security tenant isolation identity/access controls and resource governance.
• Experience with observability SRE practices service-level objectives high availability fault tolerance and automated remediation.
• Experience testing AI/LLM-powered capabilities and developing repeatable AI evaluation frameworks.
• Background in enterprise infrastructure cloud platforms systems management software or High Performance Computing environments.

Location

Hybrid Bangalore India

Travel

As needed