Agents

An agent is a loop: the model proposes a tool call, the application executes it, the result feeds back into the model, and the loop runs until the model emits a final answer or hits a budget. The loop is the primitive; everything else (planners, multi-agent choreography, “frameworks”) is sugar on top.

The basic loop

        flowchart TD
  U[user prompt] --> R["model.respond(messages, tools)"]
  R --> Q{tool_calls?}
  Q -->|yes| T[run tools]
  T --> R
  Q -->|no| A[return final answer]
    

Concretely, in a vendor SDK.

tools = [{"name": "search_logs", "input_schema": {...},
          "description": "Search SIEM logs by query."}]
messages = [{"role": "user", "content": user_query}]
for _ in range(MAX_TURNS):
    resp = client.messages.create(
        model=MODEL, tools=tools, messages=messages, ...)
    if resp.stop_reason == "end_turn":
        return resp.content[-1].text
    for block in resp.content:
        if block.type == "tool_use":
            result = run_tool(block.name, block.input)
            messages.append({"role": "assistant", "content": resp.content})
            messages.append({"role": "user", "content": [
                {"type": "tool_result", "tool_use_id": block.id,
                 "content": result}]})

Inside a model call

The model.respond arrow in the loop above is not atomic. Each call is the model running its decoder over the entire prompt, token by token, until it emits a stop token or hits the configured max_tokens budget. Five stages run in order on every call, with the transformer’s KV cache and the sampler forming a tight inner loop.

        flowchart LR
  P[prompt + tools + history] --> T[tokenize]
  T --> E[embed]
  E --> L[transformer layers]
  L --> S[sampler]
  S --> D[detokenize]
  D --> O[text + tool_use blocks]
  L -. KV cache .-> L
  S -. next token .-> L
    

Tokenizer

Bytes to integer IDs using the model’s tokeniser. Frontier models use byte-pair encoding variants (Anthropic’s tokeniser, tiktoken for OpenAI, SentencePiece for Gemini and Llama). The system prompt, tool schemas, and tool results count against the token budget exactly the same way user text does.

Concept	Detail
Vocabulary size	100k to 200k tokens on current frontier models. Larger vocabularies cover Unicode and code better, at the cost of a larger embedding matrix.
Special tokens	Role markers, end-of-turn markers, tool-call delimiters. Vendor-specific and usually hidden by the SDK.
Byte fallback	Unknown bytes split into raw byte tokens. Pathological for binary blobs (one token per byte). Strip or base64-encode binary before sending.
Cost asymmetry	Output tokens are 4x to 5x the price of input tokens at most vendors. Prompts can be long; completions should not be.

Embedding

Each token ID maps to a dense vector via the embedding matrix. Positional information is folded in via rotary position embeddings (RoPE) applied inside attention on modern decoders, rather than added to the embedding itself. The embedding weights are typically tied to the output projection, so the same matrix is read at the top of the stack and written at the bottom.

Transformer layers

A stack of decoder blocks, each containing self-attention followed by a feed-forward sublayer, with residual connections and layer normalization around both. The stack is causal (each token attends only to earlier tokens), which is why streaming output is possible.

Component	Detail
Self-attention	Multi-head attention. Frontier models often use grouped-query attention (GQA) so several query heads share one key/value head, cutting KV cache size without losing much quality.
KV cache	Past keys and values are stored per layer per head. On each new output token only the new row is computed and appended; everything earlier is read. The KV cache is the structural reason vendor-side prompt caching exists.
Feed-forward	Two linear layers with a non-linearity (SwiGLU or GeGLU on frontier models). In mixture-of-experts (MoE) models a router picks a subset of expert FFNs per token, so active parameters per forward pass are a fraction of the total.
Depth	Typically 60 to 120 layers on frontier-scale models. Latency per output token scales roughly linearly with depth and with the active parameter count.

Sampler

The transformer emits logits, a probability distribution over the vocabulary for the next token. The sampler turns logits into a single token, which is fed back into the inner loop for the next forward pass.

Knob	Effect
temperature	Scales logits before softmax. 0 is greedy (deterministic up to ties); higher values flatten the distribution and raise variability.
top_p	Nucleus sampling. Sample only from the smallest set of tokens whose probabilities sum to p.
top_k	Sample only from the top k tokens by probability.
Structured output / tool use	The sampler is constrained to a grammar or JSON schema; logits for tokens that would break the schema are masked out. This is how tool-call JSON stays parseable even at non-zero temperature.
Stop conditions	End-of-turn token, end-of-text, configured stop_sequences, or max_tokens. The first to fire terminates generation.

Detokenizer

Output IDs reassembled into text. Structured content blocks (text, tool_use, thinking) ride the same token stream with special delimiters; the SDK splits the stream into typed blocks before returning. Streaming SDKs emit partial blocks as the delimiters land.

Two practical consequences fall out of the pipeline.

• Output is sequential. Doubling max_tokens roughly doubles latency. Input length affects time-to-first-token, not subsequent tokens.

• Prompt caching exploits the prefix. Stable prefixes (system prompt, tool definitions, large attached corpora) can be marked so the vendor keeps the precomputed KV cache warm for a short TTL. Cache hits drop both latency and cost dramatically.

The Claude Code harness

Everything outside the model call is the harness. The CLI you are running owns the message list, the tool registry, the permission gate, hooks, compaction, and the on-disk trajectory. The model is treated as a stateless function.

        flowchart TD
  subgraph harness[Claude Code harness]
    direction TB
    SP[system prompt builder]
    REG[tool registry]
    CTX[context assembler]
    PARSE[response parser]
    PERM[permission gate]
    DISP[tool dispatcher]
    HOOK[hook runner]
    COMP[compaction]
    LOG[trajectory store]
  end
  CMD["CLAUDE.md, settings.json"] --> SP
  ENV["env, cwd, git, OS"] --> SP
  MCP["MCP servers, skills"] --> REG
  USR[operator input] --> CTX
  SP --> CTX
  REG --> CTX
  CTX --> API[(Anthropic API)]
  API --> PARSE
  PARSE -->|text| TERM[terminal]
  PARSE -->|tool_use| PERM
  PERM --> DISP
  DISP --> HOOK
  HOOK --> CTX
  CTX --> COMP
  CTX --> LOG
    

System prompt builder

The static prompt is assembled once at session start and patched on certain events. It sits at the head of every request and is the biggest single contributor to the cacheable prefix.

Layer	Source
Identity and policy	Compiled into the CLI binary. Defines what Claude Code is, the refusal policy, and the output-style rules.
Environment block	Working directory, OS version, shell, git status, today’s date, model ID, platform.
Tool list	Names and JSON schemas of every tool currently resident. Deferred tools appear by name only; their schemas are pulled in on demand via ToolSearch.
Skills list	User-invocable skills and their trigger descriptions. Skills are activated by name with the Skill tool.
Project rules	CLAUDE.md walking up from the working directory, plus any user-scope CLAUDE.md at ~/.claude/.
Memory index	The contents of MEMORY.md from the project’s memory directory, if present. Pointers, not the memories themselves; individual memory files are read on demand.

Tool registry

The registry holds the schemas the model sees. Each entry is a JSON description with name, description, and input_schema.

Mechanism	Detail
Built-in tools	Bash, Read, Edit, Write, Agent, TaskCreate, WebFetch, WebSearch, and more. Schemas live in the CLI.
MCP server tools	Tools exposed by Model Context Protocol servers configured in settings.json. Names are namespaced mcp__<server>__<tool> to avoid collisions.
Deferred tools	Tools whose schemas are too large to keep resident. Only the name appears in the prompt; the model loads the schema via ToolSearch before calling.
Schema validation	Every tool call is validated against the input schema before execution. Invalid calls return an InputValidationError as a tool_result so the model can self-correct without breaking the loop.
Parallel tool calls	The model may emit several tool_use blocks in one assistant message. The dispatcher runs the independent ones concurrently.

Context assembler

Maintains the running messages list submitted on every call. A message is a role (user or assistant) plus an ordered list of content blocks.

Content block	Role
text	Plain prose from user or assistant.
image	Inline image bytes or a file reference. Used for screenshots, diagrams, scanned documents.
tool_use	Assistant-only. The model’s request to invoke a tool, with id, name, input.
tool_result	User-only. The harness’s response to a prior tool_use, keyed by tool_use_id.
thinking	Assistant-only, extended-thinking mode. Hidden reasoning preserved across turns so the cache survives.

The harness enforces three ordering rules. The first user message is the operator’s prompt. Each assistant message that emits tool_use blocks must be followed by a user message containing one tool_result per call, in the same order. thinking blocks are preserved verbatim across turns; reordering them invalidates the cache.

Response parser

Splits the model’s streamed response into typed blocks. Text blocks go to the terminal as they stream; tool_use blocks are buffered until the message is complete, then routed to the permission gate. stop_reason on the final response tells the harness whether to loop or stop.

stop_reason	Meaning
end_turn	The model is done. Return control to the operator.
tool_use	One or more tool_use blocks need execution. Loop.
max_tokens	The output budget ran out. Continue with a follow-up call or surface a truncation warning.
stop_sequence	A configured stop string was hit. Rare in agent loops.
pause_turn	Streaming paused mid-response (long-running tool, server-side think). Resume by issuing the next request.

Permission gate

Every tool call passes through the gate before execution. The gate combines the current permission mode with per-tool allow / deny rules from settings.json.

Permission mode	Behavior
default	Tools listed in permissions.allow run silently; anything else prompts the operator.
acceptEdits	Edits to files in the working directory auto-approve; shell and external tools still prompt.
bypassPermissions	All tools run without prompting. High-trust contexts only, operator-opted.
plan	Read-only mode. Mutating tools refuse; the model produces a plan and exits via ExitPlanMode.

Allow / deny rules are written as Tool(pattern) strings, for example Bash(npm test), Read(/etc/**), or mcp__github__*. Deny rules win over allow rules, and the PreToolUse hook can override either.

Tool dispatcher

Executes approved tool calls and packages the result into a tool_result content block.

Concern	Behavior
Parallel execution	Independent tool_use blocks in the same assistant message dispatch concurrently. The dispatcher serialises only where one call’s output feeds another.
Timeouts	Each tool has a default timeout. Bash accepts a per-call timeout up to ten minutes. Hitting the timeout kills the process and returns an error result.
Background jobs	Bash with run_in_background: true returns a background task ID immediately; output is read via BashOutput and the lifetime is decoupled from the turn.
Output size	Stdout and stderr are truncated past a configured byte count; the truncation marker rides along in the tool_result.
Errors	Non-zero exit codes, raised exceptions, and validation failures all surface as tool_result blocks with is_error: true. The model treats them as content and decides whether to retry, switch tool, or give up.

Hook runner

Hooks are operator-configured shell commands invoked on harness events. They receive a JSON payload on stdin and may print JSON on stdout to influence behavior. Hooks run on the operator’s machine, not on the model.

Event	Fires
UserPromptSubmit	Just before the operator’s prompt is added to the message list. Can inject text or block the prompt.
PreToolUse	After the permission gate, before execution. Can deny the call, modify input, or pass it through.
PostToolUse	After the tool returns, before the result is appended. Can append text to the result the model sees.
Stop	Model emitted end_turn. Can request the model continue working.
SubagentStop	A subagent completed. Mirror of Stop for child contexts.
Notification	Idle or waiting on input. Useful for desktop notifications and pager wakeups.

A hook that exits non-zero with {"decision": "block"} cancels the event. Hook output text is fed back to the model as if from the operator, which is why UserPromptSubmit hooks can inject system reminders the model treats as authoritative.

Compaction

When the running token count crosses the configured threshold, the harness invokes the model to rewrite older turns as a compact summary, freeing window space for the next call.

        flowchart LR
  T1[turn 1] --> T2[turn 2] --> T3[turn 3] --> TN[... turn N]
  TN --> CHK{near limit?}
  CHK -->|no| NEXT[next call]
  CHK -->|yes| SUM[summarize turns 1..k]
  SUM --> KEEP["system + summary + turns k+1..N"]
  KEEP --> NEXT
    

• The system prompt, tool schemas, and CLAUDE.md stay verbatim; they are short and live inside the cache.

• Recent turns stay verbatim so the model retains detail on what is currently in flight.

• Older turns are replaced by a narrative summary the model writes itself. File-edit history, key decisions, and open threads are preserved; verbose tool output is dropped.

• /clear is the operator’s manual compaction. It resets the message list to the system prompt only.

Trajectory store

Every turn, tool call, and tool result is appended to a JSONL transcript under ~/.claude/projects/<encoded-cwd>/. The store is the audit trail, the input to /resume, and the most useful artifact when debugging a misfire.

Path	Contents
~/.claude/projects/<cwd>/<session>.jsonl	The full message stream, one JSON object per turn.
~/.claude/projects/<cwd>/memory/	Persistent memory files written by the model.
~/.claude/projects/<cwd>/memory/MEMORY.md	Index pointing at memory files; loaded into context on session start.
~/.claude/settings.json	User-scope permissions, hooks, environment variables.
<cwd>/.claude/settings.json	Project-scope overrides, checked into the repo.
<cwd>/.claude/settings.local.json	Project-scope, not checked in.

Tool call lifecycle

A single tool invocation is the smallest unit of an agent’s external effect. The harness mediates every step.

        sequenceDiagram
  participant O as Operator
  participant H as Harness
  participant M as Model
  participant T as Tool
  O->>H: prompt
  H->>M: messages + tool schemas
  M-->>H: tool_use(name, input)
  H->>H: validate against schema
  H->>H: permission check
  alt pre-authorized
    H->>T: execute(input)
  else gated
    H->>O: approve?
    O-->>H: yes
    H->>T: execute(input)
  end
  T-->>H: stdout / data / error
  H->>M: tool_result(id, content)
  M-->>H: more tool_use OR end_turn
  H-->>O: render text
    

Tool calls are content, not control flow. The model emits a tool_use block inside its assistant message; the harness threads the result back as a tool_result block on the next user turn. The model never calls anything directly.

Context window assembly

The model is stateless. Every turn the harness rebuilds the full prompt and submits it. Layer order matters because prompt caching keys on the prefix.

        flowchart TB
  subgraph window[context window]
    direction TB
    A[system prompt + harness rules]
    B[tool schemas]
    C["CLAUDE.md + project rules"]
    D[memory index entries]
    E["older turns (compacted summary)"]
    F["recent turns (verbatim)"]
    G[current user message]
    A --> B --> C --> D --> E --> F --> G
  end
  CACHE[("prompt cache, 5-min TTL")]
  CACHE -. covers .-> A
  CACHE -. covers .-> B
  CACHE -. covers .-> C
  CACHE -. covers .-> D
    

The first four layers are stable for most of a session and live inside the cache breakpoint. The lower layers change every turn and miss the cache by design. Short, frequent calls keep the cache warm; long sleeps (over five minutes for Anthropic) drop the warm state and the next call pays full prefix cost.

Subagents

A subagent is a child harness with its own context window. The parent spawns it via the Agent tool with a task description and sees only the final result.

        flowchart TD
  P[parent harness]
  SA["subagent A (own window)"]
  SB["subagent B (own window)"]
  P -->|"spawn(task A)"| SA
  P -->|"spawn(task B)"| SB
  SA -->|final result| P
  SB -->|final result| P
    

• Context protection. Searching a large repo produces tens of thousands of tokens of file content. A subagent reads, distils, and returns the answer; the parent’s window is untouched.

• Parallelism. Independent subtasks (run linter, run tests, fetch URLs) progress concurrently. Each subagent’s trajectory is independent.

• Isolation modes. isolation: "worktree" spawns the subagent inside a fresh git worktree so its edits do not collide with the parent’s working tree.

• Subagent types. Explore (read-only search), Plan (architect), general-purpose, plus user-defined agent types in .claude/agents/.

Other agent forms

The loop primitive admits more than one form. The differences are in how the loop is structured, not in what it is.

ReAct

Older pattern from before native tool use. The model alternates explicit “Thought” and “Action” tokens; the harness parses the substrings out of plain text. The modern equivalent is a loop with tool calls, but ReAct still applies when the model has no tool-use API or when running over a chat-completions endpoint.

        flowchart LR
  THK[Thought] --> ACT[Action]
  ACT --> OBS[Observation]
  OBS --> THK
  THK --> ANS[Final Answer]
    

OpenAI Assistants / Responses API

The vendor hosts the message list, the tool definitions, and the file context. The client creates a thread, attaches files, kicks off a run, and polls until the run completes or surfaces a tool call. Trades local control for a managed trajectory and built-in code interpreter, file search, and computer-use tools.

LangGraph and state-machine harnesses

Explicit state machine over the loop. Nodes are functions, edges are conditional transitions, state is a typed dict. Useful when the workflow is not a plain loop, for example branching plans, parallel fan-out / fan-in, retries with back-off, or human-in-the-loop checkpoints with persistence between sessions.

        flowchart LR
  START([start]) --> PLAN[plan]
  PLAN --> RUN[execute step]
  RUN --> CHECK{done?}
  CHECK -->|no| RUN
  CHECK -->|yes| REVIEW[review]
  REVIEW -->|approve| DONE([end])
  REVIEW -->|revise| PLAN
    

Designing tools

Tools are the agent’s surface to the world. Design them like any other API.

• One job per tool. search_logs(query, time_range) beats query_siem(action, params); the model picks the right tool more reliably.

• Strict schemas. JSON schema with required fields, enums where possible, no free-form params: object.

• Errors are content. Return {"error": "...", "hint": "..."} as the tool result. The model reads it and adjusts.

• Idempotent where possible. The agent will retry. Make retries safe.

• Authorize outside the prompt. Tool authorization lives in the application, not in the system prompt. The model proposes; the application disposes.

Operational concerns

• Budget the loop. Cap the turn count, the wall-clock time, the total tokens. Agents that are not budgeted will eventually run forever.

• Persist the trajectory. Every tool call, result, and intermediate response. The trajectory is the agent’s audit trail and the only useful debugging artifact.

• Compaction. Long trajectories blow past the context window. Summarize older turns; keep the system prompt and the most recent N turns verbatim.

• Human-in-the-loop on irreversible actions. Email, payment, rm -rf, branch deletes, prod deploys: gate behind a confirmation step. The model proposes; the operator confirms.

• Subagents for parallel or context-protected work. Each subagent gets its own context window; the parent only sees the result.

Multi-agent vs. single-agent

Multi-agent systems (planner / executor / critic) are easy to prototype and hard to operate: more state, more failure modes, more prompt drift. Default to a single agent with good tools. Add a second agent only when the workload genuinely splits along context or trust boundaries (e.g. a reviewing agent that audits the acting agent’s tool calls).

        flowchart TD
  P[Planner] -->|plan| E[Executor]
  E -->|result| C[Critic]
  C -->|approved| OUT[final answer]
  C -->|revise| P
    

Frameworks

• Vendor harnesses (Anthropic Agent SDK, OpenAI Agents) are the thinnest practical wrappers; they do the loop and the trajectory for you.

• LangGraph / LlamaIndex Agents add explicit state machines. Useful when the workflow is not a plain loop.

• Plain code beats both for anything you understand. The loop is twenty lines.

References

• SDKs for the underlying tool-use API.

• Evals for evaluating agent trajectories, not just final answers.

• LLM-Assisted Analysis for analyst-facing agentic patterns.

• Anthropic Agent SDK

• Anthropic prompt caching

• Model Context Protocol

• ReAct: Synergizing Reasoning and Acting in Language Models

• LangGraph documentation

• OpenAI Assistants API