Poison Kills Poison: AI Tests AI

How do you test something that gives different answers every time? Traditional testing relies on deterministic outcomes—same input, same output. But AI systems break this rule entirely. Welcome to the world where poison kills poison: we need AI to test AI.

In this hands-on 3-hour masterclass, you’ll build, break, and evaluate AI systems from scratch. No slides, no theory dumps—just practical coding on your laptop.

We start by creating a simple ChatGPT-style application using FastAPI and OpenAI. You’ll see how quickly AI can help us build AI. Then we’ll construct a multi-agent system where AI agents collaborate—a planner, a coder, and a reviewer working together using LangChain.

But here’s the real challenge: how do we know these systems actually work? We’ll dive deep into AI evaluation parameters that traditional testers never learned—groundedness, relevance, factual accuracy, and hallucination detection. You’ll implement these evaluations using LangSmith or similar tools and see your agents scored in real-time.

For multi-agent systems, we’ll evaluate whether agents call the right tools, pass correct context to each other, and complete tasks effectively. This is testing for a new era.

You’ll leave with working code, practical evaluation frameworks, and a mindset shift: in the AI age, our testing tools must evolve too. The testers who understand AI evaluation will become invaluable as every application becomes AI-infused.

Whether you’re new to AI or already experimenting with agents, this tutorial gives you hands-on skills you can apply immediately. Bring your laptop, your curiosity, and prepare to see testing through a completely new lens.