Old Problems of New Agents: Oracles and Communication

Agent-based software development is taking over everywhere with its promise of boosted efficiency and autonomy. But for test automation professionals—especially those who champion quality and manage risk—fully trusting agents isn’t so straightforward. In this session, we’ll confront the two core challenges that stand between us and truly autonomous software creation: the communication problem and the oracle problem.

if you care about the future of test automation, understanding how AI agents collaborate, communicate, and validate outcomes is critical. We’ll explore problems and solutions, including Large Language Model (LLM) “judges,” Retrieval-Augmented Generation (RAG) workflows, and the power of Human-in-the-Loop strategies.

You’ll discover how these techniques can enhance communication clarity among both human stakeholders and AI-driven agents, preventing costly misunderstandings. We’ll then tackle the oracle problem—determining “correct” AI outputs—by showcasing supercharged feedback loops that leverage heuristics, known datasets, and real-time human insights.

Join us to learn how to design a resilient, value-focused AI-based development and testing pipeline. Attendees will leave with practical insights on integrating advanced LLM capabilities into their quality assurance strategies. From bridging communication gaps to validating AI-generated outputs, this talk offers a forward-looking view of how test automation teams can harness autonomous agents—safely and effectively. If you’re ready to future-proof your test strategies in an era of agentic development, this session is for you.