Anatomy of an AI Agent

An AI agent isn't just a model with tools. What actually happens between your prompt and a completed task, and what keeps the system from falling apart when something goes wrong?

In Episode 30 of System Prompt, Peter and Val take apart the Hermes Agent harness to examine the software engineering behind an AI agent. Building on their episodes about AI hardware and inference runtimes, they follow the agent loop from context assembly and model calls through tool execution, persistence, security, recovery, and observability. Hermes is the example, not the endorsement: the point is to show where probabilistic model decisions end and deterministic software takes over.

They explore why switching model providers involves more than changing an API endpoint, why a tool call is a request rather than an action, and how seemingly ordinary components like SQLite, file-based memory, and approval rule