Architecture

Atomic Agent is a local-first AI agent that does real work on your machine — browsing, editing files, running shell commands, remembering things across sessions. This page explains what happens between the moment you send a message and the moment the agent replies.

The short version: one model inference produces one step. A step is a batch of tool calls. The agent runs them, compresses the results, updates its state, and asks the model again. That repeats until the agent decides it’s done.

Developer Preview

Atomic Agent is a Developer Preview (this page describes v0.6.5). Internals, commands, and config keys described here can still change. Pin a release if you need stability.

The core idea

Most of the cost of running an agent is the model. Atomic Agent is built around keeping each model call small and cheap so that long, multi-step sessions stay affordable on a quantized local model.

It does this with a few deliberate choices:

• One inference per step. The model is never asked to “think and act in a loop” inside a single call. Each call emits a JSON array of tool calls and stops. The runtime executes them and calls the model again.
• State lives outside the prompt. Conversation history, memory, browser snapshots, and tasks all live in SQLite, not in an ever-growing context window. The prompt only ever carries a compact slice.
• A stable prompt prefix. The top of the prompt (persona, rules, tool catalog, skills) is byte-stable across a session, so llama-server can reuse its KV-cache and skip recomputing it (on Anthropic models, direct or via OpenRouter, the same prefix is marked for provider-side prompt caching). Only the prefix is held byte-stable — the variable tail below it is expected to change every step.
• Compressed results. Verbose tool output (test logs, grep dumps, stack traces) is shrunk to a summary before it ever reaches the model.

The result is a tight, observable loop you can trace, replay, and reason about.

The agent loop at a glance

flowchart TD
    User["User message<br/>or scheduled task"] --> Prompt["Build prompt<br/>(stable prefix + bounded tail)"]
    Prompt --> Infer["One LLM inference<br/>(GBNF grammar locally,<br/>native tool calls on cloud)"]
    Infer --> Calls["JSON array of<br/>1..N tool calls"]
    Calls --> Exec["Execute batch<br/>(parallel reads, approval gates)"]
    Exec --> Compress["Compress results"]
    Compress --> State["Update durable state<br/>(session, memory, world)"]
    State --> Terminal{"reply or finish?"}
    Terminal -->|no| Prompt
    Terminal -->|reply| Reply["Turn ends,<br/>session pending"]
    Terminal -->|finish| Done["Session completed"]
    State -.->|async, never awaited| Reflect["Memory reflection<br/>(learns from the turn)"]

Two terms recur:

• A turn is one user message (or task trigger) processed to completion. It can span many steps.
• A step is one model inference plus the tool batch it produced.

Walking through one turn

When you send a message, the runtime calls AgentLoop.runTurn(). Here is what it does, in order.

1. Set up the turn

• Your message is appended to the session transcript.
• Memory context is refreshed: relevant notes, profile facts, lessons, and procedures are recalled and staged into the prompt tail.
• The model profile and tool-calling grammar are synced with llama-server (in case the operator swapped models).
• A fresh ToolLoopTracker is created to watch for the agent getting stuck.

2. Run the step loop

The loop runs in legs of agent.maxSteps steps (default 25). At each leg boundary it checks that the leg produced something usable: if so it reports progress and keeps going, if not it stops. The task as a whole is capped by agent.task.maxSteps (default 1000) and agent.task.maxDurationMs (default 2 hours). When a ceiling is reached, or a whole leg made no progress, the agent pauses with a short note on why; say continue to pick up where it stopped. Set agent.task.autoContinue: false to stop after one leg.

Each step:

1. Re-probes llama-server’s /props if a model-profile manager is wired (this is how a mid-turn model swap is detected).
2. Calls executeStep, which builds the prompt, runs one inference, parses the tool calls, and executes them.
3. Inspects the outcome:
   • If the model emitted reply on its own, the turn ends (session left pending, so the user can continue). A reply batched with other calls is a progress note and does not end the turn (see Executing the batch).
   • If it emitted finish, the session is completed.
   • Otherwise, loop-detection signals are processed and memory context is refreshed for the next step.

3. Close out

When the loop exits, the session status is set, the turn count is incremented, and a lesson-lifecycle hook records whether the turn succeeded or failed.

A session is always in one of eight states:

Status	Meaning
pending	Idle and ready — the last turn replied, or none has run yet
running	A turn is executing right now
awaiting_approval	Blocked on you approving an approval-gated call
awaiting_llm	Blocked waiting on the model’s response
completed	The model emitted finish
failed	The turn ended in an unrecoverable error
cancelled	The turn was cancelled
stalled	The task hit a step or time ceiling, or a whole leg made no progress, without reply or finish

stalled does not break the session: your next message can drive a new turn. It is still counted as a failure for exit codes, so atomic-agent run exits 1. It exists so operators can tell that condition apart from a clean pending idle.

Finally, memory reflection fires — asynchronously, never awaited. It runs in the background on a separate llama-server slot so it never disturbs the main agent’s KV-cache or blocks your reply.

Inside a single step

executeStep is where the model meets the tools. The pipeline:

flowchart TD
    Build["buildPrompt()<br/>stable prefix + tail"] --> Slot["acquire KV-cache slot<br/>(prefix hash)"]
    Slot --> Complete["LLM completion<br/>(stream or unary)"]
    Complete --> Reason["resolveReasoning()<br/>(server channel or inline think)"]
    Reason --> Parse["parse tool calls<br/>(grammar or native_tools)"]
    Parse --> Validate["validate batch"]
    Validate -->|ok| Run["executeBatch()"]
    Validate -->|parse failed| Repair["one-shot repair completion"]
    Repair --> Run
    Run --> Effects["apply state effects<br/>(world snapshot, rare-tool autoload)"]
    Effects --> Record["append transcript turns"]

A few details worth knowing:

• One inference, an array of calls. The grammar constrains the model’s output to a JSON array. A single action is just an array of length one. This removes the model’s temptation to “choose” between solo and batched calls.
• Reasoning is captured two ways. resolveReasoning() prefers the server’s dedicated reasoning_content channel when it’s available (QwQ, DeepSeek-R1 with --reasoning-format deepseek), and otherwise falls back to extracting inline <think> blocks from the content body for models that stream chain-of-thought inline.
• Validation enforces the batching rules (covered below). If it fails for a recoverable reason, the runtime injects a notice and retries once with a repair completion rather than burning a whole step.

Executing the batch

Once the calls are parsed and validated, executeBatch runs them in three phases, grouped by resource class. There are nine classes:

Class	Covers
pure_read	Side-effect-free reads — the only class that fans out in parallel
fs_write	Declared, but no tool is assigned it today (see below)
browser	browser.* — one Playwright backend per process
memory_write	Mutations to the memory store
tasks_write	Mutations to the task queue
vision	vision.*
approval_gated	Anything that needs your approval, including os.shell.run and os.fs.write
terminal	reply and finish
unknown	Default for unlisted tools; the validator rejects these from any batch (fail-closed)

fs_write is declared but unused

fs_write exists in the type union, but no tool currently maps to it. Mutating filesystem verbs such as os.fs.write are classified approval_gated instead, so they go through the approval gate rather than a write group.

The three phases:

1. Sync loop gate. Every non-terminal call passes through the loop detector first, in batch order. Vetoed calls get a synthetic error result instead of running.
2. Grouped async execute. Groups in distinct resource classes execute concurrently with each other, so the total step wall time is roughly max(group durations). Within a group, pure_read tools fan out in parallel, while the other classes serialize in batch order to keep their writes deterministic.
3. Terminal barrier. If the batch ends with reply or finish, that call runs solo, last, after every other call settles — even if an earlier tool failed. This preserves the model’s intent: “do the work, then close the turn.”

A reply that arrives together with other tool calls is treated as a progress note: it is shown to you, the other calls run, and the turn continues. Only a reply on its own ends the turn. finish batched after work still runs last and ends the session.

Results are collected back in batch-index order, then each one is compressed.

Batching rules

Two constraints are enforced by the validator:

• finish is allowed only at the tail of a multi-call batch. (A batched reply becomes a progress note, as described above.)
• Approval-gated tools normally run one per step. If the model emits one in a batch, the runtime trims the batch to just that first approval-gated call, drops the rest, and injects a notice so the model retries them one at a time. This avoids a deadlock, because the approval gate is per-call. When nothing in the batch could ask (every call is auto-approved at the current level or covered by a session grant), the batch is not trimmed and the gated calls run one after another in the order the model emitted them. In practice file writes only qualify at level 5 (for example under --no-approval), because a write could land on the agent’s own config. fusion.delegate always runs alone.

Fusion mode

Besides the plain local and cloud run modes there is Fusion. Turn it on with llm.runMode.mode: "fusion" in config.json or /runmode fusion in the TUI (which writes the same key). In Fusion, one model runs the loop above as the orchestrator and hands parts of the job to workers through the fusion.delegate tool. By default the orchestrator is a cloud model and the workers run on your local llama-server; both legs can be changed under llm.runMode.fusion.

• One fusion.delegate call runs several worker turns at once (fan-out width llm.runMode.fusion.workers, default 2, or /runmode workers N), always as a solo call. It asks under fusion · fan-out below level 4, and that approval is never granted for the whole session.
• Each worker turn is short-lived and bounded: it cannot delegate further or call finish, it cannot write memory, and an approval prompt inside a worker is refused rather than shown. The orchestrator reads what the workers returned and decides what happens next.
• Fusion needs a cloud provider with a key and a downloaded local model. If a leg is missing, the TUI refuses to switch and says why, and a stored fusion setting with a missing leg falls back to plain local or cloud mode and reports itself as degraded (/runmode status).

Compressing results

Every tool result is passed through compressToolResult() before it re-enters the conversation. The compressor:

1. Keeps the last N non-blank lines (the tail). Each tool sets its own N: a shell command keeps up to 500 lines and 16 000 characters by default (ATOMIC_AGENT_SHELL_TOOL_RESULT_TAIL_LINES, ATOMIC_AGENT_SHELL_TOOL_RESULT_CHAR_CAP); the bare fallback is 12 lines.
2. For error results, scans for an error signature (error:, failed:, traceback, exception:) and surfaces the first match.
3. Enforces a maximum summary length (the bare fallback is 400 characters; read tools pass larger caps), appending [truncated] if it overflows.

All results in one batched step also share a 32 000-character cap (ATOMIC_AGENT_BATCH_TOOL_RESULT_CHAR_CAP). The model sees this bounded summary, not the raw output, which is what keeps each step under budget even when a shell command spews thousands of lines.

Detecting and breaking loops

Local models can get stuck repeating the same call. ToolLoopTracker watches for this within the turn (it has no memory of prior turns):

• Args-only repeats raise a warning once per bucket of repeats.
• Args + result no-progress streaks trip a critical veto.
• Wandering tools (os.web.search, os.web.fetch, os.http.request, browser.*) are watched for a spreading set of distinct args; crossing the escalation threshold jumps straight to the breaker.

The breaker is graceful, not fatal

When the loop breaker trips, it does not mark the session failed. It synthesizes a graceful reply (formatForcedLoopReply()), the transcript shows that reply, and the turn closes normally. The session is left pending, so you can redirect the agent and keep going.

Thresholds are set with environment variables (they are not config.json keys):

Knob	Env var	Default
Warning threshold	ATOMIC_AGENT_LOOP_WARNING_THRESHOLD	3
Critical threshold	ATOMIC_AGENT_LOOP_CRITICAL_THRESHOLD	5
Breaker veto streak	ATOMIC_AGENT_LOOP_BREAKER_VETO_STREAK	3
History window size	ATOMIC_AGENT_LOOP_HISTORY_SIZE	30
Wandering redirect threshold	ATOMIC_AGENT_LOOP_WANDERING_THRESHOLD	6
Wandering escalation	ATOMIC_AGENT_LOOP_WANDERING_ESCALATION	12
Max parallel tool calls	ATOMIC_AGENT_MAX_PARALLEL_TOOL_CALLS	8 (hard max 16)

How state stays small

The prompt is split into two zones:

• A stable prefix: persona, interaction rules, the tool catalog, the skill index, and environment capabilities. Byte-stable across the session, so it lives in llama-server’s KV-cache and is never recomputed. The runtime measures this scaffolding at roughly 5.2k tokens and budgets 6000 (AGENT_FIXED_PROMPT_TOKENS) as headroom for drift — that constant is only a startup sanity check against the model’s context window, and the real per-build figure comes from checkBudget.
• A variable tail: loaded skills, profile facts, recalled memory, the latest browser snapshot, and conversation history — ordered slowest-changing first so a hot section at the bottom never invalidates a slow section above it. It is sized by the remaining budget rather than a fixed allowance, and it is expected to change every step.

Everything durable lives outside the prompt entirely:

• Sessions (the full transcript) in sessions.sqlite.
• Memory (profile facts, notes, lessons, procedures, embeddings, link graph) in memory.sqlite.
• Tasks (deferred and recurring work) in tasks.sqlite.

The prompt only ever sees compact pointers into memory — never the whole store.

Where it all gets wired together

Every entry point — atomic-agent run, atomic-agent serve, atomic-agent tui, and the Tauri sidecar — constructs the runtime exactly once through createAgentRuntime(). That factory probes llama-server, builds the LLM provider registry and tool registry, opens the memory stores, wires the AgentLoop, and returns an AgentRuntime handle.

Turn serialization is the runtime’s job, via the TurnController:

• Per-session FIFO: turns submitted to the same session run sequentially, in order.
• Cross-session parallelism: different sessions never block each other.

graph TD
    Entry["run / serve / tui / sidecar"] --> Factory["createAgentRuntime()"]
    Factory --> Llama["probe llama-server"]
    Factory --> Tools["ToolRegistry"]
    Factory --> Memory["Memory stores (SQLite)"]
    Factory --> Loop["AgentLoop"]
    Factory --> Turn["TurnController"]
    Turn -->|per-session FIFO| Loop
    Loop -->|runTurn| Step["executeStep → executeBatch"]
    Step --> Tools
    Step --> Memory
    Step -.->|events| Trace["TraceRecorder (NDJSON)"]

Try it

The loop is easiest to watch with tracing on. Run a turn, then replay or inspect it:

# Run a session in the current directory
atomic-agent run --cwd .

# Optional: cap the whole task at 25 steps
atomic-agent run --cwd . --max-steps 25

# List recorded traces, then inspect one step-by-step
atomic-agent trace list
atomic-agent trace show <sessionId>

# Re-run the prompts against llama-server to check for drift
atomic-agent trace replay <sessionId>

Memory Profile facts, notes, lessons, and procedures, and how recall avoids prompt bloat.

Tools The tool namespaces, resource classes, and the approval gate.

Configuration The agent.* config block, task ceilings, and the ATOMIC_AGENT_* environment variables.

GBNF tool calling, explained From the blog: how grammar-constrained decoding keeps every tool call valid.