AutomationSTAR

Test Automation Conference Europe

Track Talk 12

Mr. Bond: It Takes an Agent to Test an Agent

Adam Sandman

13:45-14:30 CET, Thursday 5th November

Just like MI5 uses agents to test and find other agents that have gone rogue, when it comes to AI, we need to use the same approach. In this talk I will discuss the limitations of how we are current testing AI agents, and why this means we are not adequately ensuring the safety of agentic AI systems.

With non-deterministic systems like Generative/Agentic AI, we need to simulate a large number of inputs (millions) and measure the outputs using judge agents to find the statistical success rate. This a process that is more similar to how we traditionally do load testing rather than the simple functional testing we’re using with AI right now.

I will explain how you can instead use tools like AgentCore to create orchestration agents that build other types of agent to make this new type of non-deterministic testing possible. If you want to be ready for the next wave of QA, this talk is for you.

BACK TO PROGRAMME

What you will Learn

  1. Current functional testing techniques are inadequate for testing agentic/generative AI systems.
  2. What does it mean to use LLM as Judge agents? What are input agents?
  3. How can you create an AI testing orchestration pipeline for testing AI agents.

Session Details

  • Advanced
  • 30 minutes
  • 15mins Q&A
  • Emerging Landscapes (Vibe Testing & Agentic AI)
Buy Conference Ticket

Check out this video from Adam Sandman to hear more about his talk on Mr. Bond: It Takes an Agent to Test an Agent. We look forward to welcoming you to AutomationSTAR 2026.

Session Speaker

Adam Sandman

CEO - Inflectra, USA

Adam Sandman, who founded Inflectra in 2006, has been a programmer since the age of 10. Today, Adam serves as the company’s CEO. He is responsible for product strategy, technology innovation, and business development. He lives in Washington, D.C., with his family. When he is not working, he can often be found giving talks at events such as EutoSTAR, TestCon, Testing United, STAR East and TestingMind. He is passionate about entrepreneurship and technology and is excited to discuss how technology can improve the world around us as well as forge connections between communities and individuals.

part of the