Raindrop AI Launches Workshop: Open Source Tool for Local AI Agent Debugging and Evaluation

Raindrop AI Unveils Open Source Workshop for AI Agent Debugging

Observability startup Raindrop AI today released Workshop, an open source, MIT-licensed tool that gives developers a local debugger and evaluation system specifically built for AI agents. The tool enables real-time monitoring of agent behavior without sending data to external servers.

Workshop acts as a local daemon that streams every token, tool call, and decision to a dashboard on the developer's machine—typically at localhost:5899. All traces are stored in a single lightweight SQLite database file (.db).

Real-Time, Private Debugging

"Developers have been struggling to see what their AI agents are doing in real time without relying on cloud-based telemetry," said Ben Hylak, co-founder and CTO of Raindrop (a former Apple and SpaceX engineer), in a direct message. "Workshop eliminates that latency and keeps data local, which is critical for enterprise users with strict privacy requirements."

The tool is available for macOS, Linux, and Windows via a one-line shell installation, or from source on GitHub using the Bun runtime.

Self-Healing Eval Loop

Workshop’s standout feature is the "self-healing eval loop." It allows coding agents like Claude Code to read traces, write evaluations against the codebase, and autonomously fix broken code. For example, if a veterinary assistant agent fails to ask necessary follow-up questions, Workshop captures the full trajectory. Claude Code then reads the trace, writes a specific eval, identifies the logic error, and re-runs the agent until all assertions pass.

Background

The agentic AI era, which kicked off in earnest last year, has exposed a critical gap in developer tooling. Existing debugging tools were designed for static code, not for autonomous agents that make decisions, call tools, and interact with environments dynamically. Workshop fills that gap by providing a dedicated local environment for inspection and iterative improvement.

"Our team built Workshop because we needed a sane way to debug agents locally," Hylak noted on X. "It changed how we build autonomous systems, and we wanted to share that with the community."

What This Means

For developers, Workshop means no more blind faith in black-box agents. They can now trace every decision, pinpoint errors, and fix them in real time. For enterprises, local storage ensures data sovereignty, addressing a growing concern about sending sensitive traces to external servers.

The MIT license opens the door for community contributions and enterprise adoption without licensing fees. Raindrop hopes the tool evolves into a standard component in the AI development stack.

To celebrate the launch, Raindrop is offering limited-edition physical merchandise.