Guardian of Quality Realms: My Role on OWASP FinBot CTF
OWASP FinBot CTF calls itself test
Heading
oneOWASP FinBot CTF calls itself "the Juice Shop for Agentic AI." It's an intentionally vulnerable, multi-agent vendor-management platform. LLM agents handle onboarding, fraud detection, invoices, payments, and communications, and they have real MCP tool access. Players try to break those agents with prompt injection, policy bypass, tool poisoning, data exfiltration, and remote code execution. An event-driven pipeline detects successful exploits automatically, so there are no static flags to copy and paste.
I'm one of the project leads, credited as "Guardian of Quality Realms." In practice, I own quality across the workstream. That means test strategy, bug triage, CI health, and making sure every feature does what it claims to do.
Testing a platform that's supposed to be vulnerable
QA on a deliberately vulnerable system is an odd job. Some bugs are features. The prompt-injection paths are the product. Other bugs are just bugs, and they quietly break the game. Most of my work is drawing that line clearly and then proving it with tests.
I've driven that through the CD001 test-coverage effort, a long series of pull requests that add unit tests across the platform:
CTF detectors: the code that decides whether a player actually landed an exploit
Agent tools and payments: the actions agents take on a player's behalf
MCP servers: FinDrive, FinStripe, FinMail, TaxCalc, and SystemUtils, plus the MCP factory and providers
OrchestratorAgent: the LLM coordinator that delegates work across six specialized agents
Chat assistants: the vendor chat and co-pilot streaming layer
Each suite follows the same format: a test ID, a title, a test question, steps, and expected results. Reviewers can read a test like a spec. The OrchestratorAgent suite alone has more than 60 cases across initialization, prompts, tool definitions, delegation limits, workflow-context propagation, event emission, and edge cases. It also includes bug-exposing tests that pin down real production defects, such as poisoned context from one agent being passed into a downstream agent that has dangerous tool access.
Keeping CI honest
Tests only help if CI can run them. One of my merged fixes (#495) set LLM_PROVIDER=mock in CI, because the OpenAI client was being instantiated at import time and breaking every contributor's pipeline. In the review thread, someone suggested running Ollama in CI instead. I argued for the mock: it's deterministic, it's instant, and it tests the code around the model rather than the model itself. Live-model checks belong in an on-demand integration suite.
The next posts cover the parts I'm proudest of: two CTF challenges I designed, a real authentication bug I fixed, and a tool that makes every pull request show its own test gaps.
Skills: AI security testing, Python/pytest, test architecture, CI/CD, OWASP Top 10 for LLM & Agentic Applications, open-source leadership



Comments