About the Role:

Grade Level (for internal use):

The Team:
Join the elite Quality Engineering arm of S&P Globals AI transformation. Our team is responsible for the Trust Layer of our Agentic Workflows. We dont just test code; we validate AI reasoning. We ensure that our autonomous agentspowered by GCP (Gemini) AWS (Bedrock) and Azure AI Foundryact reliably and securely within the Moveworks ecosystem.

The Impact:
As a Senior Lead QA Engineer you are the guardian of AI reliability. You will design the frameworks that test the unpredictable nature of LLMs ensuring that agentic thoughts translate into safe accurate business actions. Your work prevents hallucinations secures enterprise data and ensures a flawless experience for every global employee.

Responsibilities

• Agentic Logic Validation: Design and execute test strategies for autonomous AI agents. You will move beyond functional testing to validate reasoning pathways tool-use accuracy and multi-step planning using frameworks like DeepEval or G-Eval.
• Multicloud AI Benchmarking: Oversee the quality of LLM deployments across a multicloud stack. You will benchmark the accuracy and latency of Gemini (GCP) Claude (AWS) and GPT (Azure) to ensure the right model is chosen for each automation.
• Automation-of-Automation Testing: Build automated test suites for Moveworks SparkActions and hybrid low-code/pro-code connectors. Ensure that a change in a ServiceNow or ADO API doesnt break the AI agents ability to execute.
• Evaluation & Observability: Implement LLM evaluation frameworks (e.g. RAGAS Arize Phoenix or LangSmith) to monitor agentic hit rates and detect drift or regressions in AI performance over time.
• Adversarial & Safety Testing: Lead Red-Teaming efforts for AI agents. Test for prompt injections data leakage and boundary-crossing to ensure agents never perform unauthorized actions.
• Technical QA Leadership: Lead a pod of QA engineers in Islamabad. Define the standards for AI testing conduct architectural reviews for testability and mentor staff on the shift from deterministic to probabilistic testing.
• Stakeholder Trust: Translate technical quality metrics into business confidence. Explain to HR or Finance leaders how we have validated that the AI will handle their complex data with 100% reliability.

What Were Looking For: REQUIRED SKILLS/QUALIFICATIONS

Technical Expertise:

• 8 years of experience in Quality Engineering SDET or Systems Integration Testing. We value candidates who have led the QA for complex business processes on platforms like ServiceNow Workato or Salesforce.
• Automation Strategy: 5 years focused on automated testing of workflows. You should have a proven track record of testing end-to-end user journeys across disparate business systems.
• AI & LLM Testing: A strong interest or experience in Testing Generative AI. You understand the challenges of testing LLM-driven decision-making (e.g. handling non-deterministic outputs).
• The Pro-Code Core: Expert-level Python (mandatory) for building custom test frameworks simulating API responses and automating AI evaluation pipelines.
• Connector & API Testing: Deep experience testing enterprise APIs (REST/SOAP) and handling the authentication/logic bridges between agents and systems like ADO Workday or Salesforce.
• Platform Mastery: Experience with Moveworks or similar conversational AI platforms and familiarity with cloud environments (GCP AWS or Azure AI Foundry) for hosting test runners.

Leadership & Soft Skills:

• Test Leadership: Experience leading QA for high-stakes technical projects. You enjoy mentoring others and defining the Quality Bar for new technologies.
• Analytical Communication: Exceptional ability to take a complex AI failure and explain the root cause (whether its the prompt the model or the underlying API) to technical and non-technical stakeholders.

Nice to Have:

• Certifications: ISTQB Advanced Level Google/AWS/Azure Cloud Certifications.
• Agentic Frameworks: Exposure to testing LangChain or LangGraph-based applications.
• Data Science Lite: Basic understanding of vector databases and RAG (Retrieval-Augmented Generation) to better test knowledge-retrieval accuracy.
• Performance Engineering: Experience testing the latency of LLM tokens and API response times.

Why Join Us

• Test the Untestable: You will be a pioneer in the field of AI Quality Engineering building the rules for how autonomous agents are validated.
• Multicloud Exposure: Work across the worlds leading AI clouds (Google AWS Azure).
• High-Impact Growth: At S&P Global the QA team is a key partner in the development lifecycle not an afterthought.

Build the Trust Layer for agentic automation. Join S&P Global.

Whats In It For You

Our Mission:

Advancing Essential Intelligence.

Our People:

Were more than 35000 strong worldwideso were able to understand nuances while having a broad perspective. Our team is driven by curiosity and a shared belief that Essential Intelligence can help build a more prosperous future for us finding new ways to measure sustainability to analyzing energy transition across the supply chain to building workflow solutions that make it easy to tap into insight and apply it. We are changing the way people see things and empowering them to make an impact on the world we live in. Were committed to a more equitable future and to helping our customers find new sustainable ways of doing business. Join us and help create the critical insights that truly make a difference.

Our Values:
Integrity Discovery Partnership

Throughout our history the worlds leading organizations have relied on us for the Essential Intelligence they need to make confident decisions about the road ahead. We start with a foundation of integrity in all we do bring a spirit of discovery to our work and collaborate in close partnership with each other and our customers to achieve shared goals.

Benefits:

We take care of you so you cantake care of business. We care about our people. Thats why we provide everything youand your careerneed to thrive at S&P Global.

Our benefits include:

• Health & Wellness: Health care coverage designed for the mind and body.

• Flexible Downtime: Generous time off helps keep you energized for your time on.

• Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.

• Invest in Your Future: Secure your financial future through competitive pay retirement planning a continuing education program with a company-matched student loan contribution and financial wellness programs.

• Family Friendly Perks: Its not just about you. S&P Global has perks for your partners and little ones too with some best-in class benefits for families.

• Beyond the Basics: From retail discounts to referral incentive awardssmall perks can make a big difference.

For more information on benefits by country visit: Hiring and Opportunity at S&P Global:

At S&P Global we are committed to fostering a connected andengaged workplace where all individuals have access to opportunities based on their skills experience and contributions. Our hiring practices emphasize fairness transparency and merit ensuring that we attract and retain top talent. By valuing different perspectives and promoting a culture of respect and collaboration we drive innovation and power global markets.

Recruitment Fraud Alert:

If you receive an email from a domain or any other regionally based domains it is a scam and should be reported to. S&P Global never requires any candidate to pay money for job applications interviews offer letters pre-employment training or for equipment/delivery of equipment. Stay informed and protect yourself from recruitment fraud by reviewing our guidelines fraudulent domains and how to report suspicious activityhere.

Equal Opportunity Employer

S&P Global is an equal opportunity employer and all qualified candidates will receive consideration for employment without regard to race/ethnicity color religion sex sexual orientation gender identity national origin age disability marital status military veteran status unemployment status or any other status protected by law. Only electronic job submissions will be considered for employment.

If you need an accommodation during the application process due to a disability please send an email to:and your request will be forwarded to the appropriate person.

US Candidates Only:Know Your Rights: Workplace discrimination is illegal