Durable Execution: Checkpoint and Resume Agent Runs

Long-running agent loops are vulnerable to server restarts, client disconnects, and unexpected errors. Anvil’s durable execution system saves a checkpoint after every completed iteration — including the accumulated message history, token usage, and the results of any side-effect tool calls. If the process crashes or is redeployed, the run resumes from the last checkpoint without restarting from scratch or re-executing tools that have already run.

How checkpointing works

The Checkpointer class persists AgentCheckpoint objects into any StateStore (in-memory for tests, SQLite for production). The agent runtime writes a checkpoint:

After each completed iteration — capturing the current message history and accumulated usage.
When the run suspends for HITL approval — storing the pending payload and the results of all tools that already ran.

On resume, resumeAgent loads the checkpoint, restores the message history and counters, and re-enters the loop at the point it left off. Tool calls recorded in the approvals map are replayed from their stored results rather than re-executed.

Checkpointer API

import { Checkpointer } from 'anvil-sdk/agent';
import { SqliteStateStore } from 'anvil-sdk/store';

const stateStore = await SqliteStateStore.open('.anvil/state.db');
const checkpointer = new Checkpointer(stateStore, 'run-abc-123');

StateStore

required

The key-value store where checkpoints are persisted. Use SqliteStateStore in production and MemoryStateStore in tests.

string

required

A stable identifier for this run. Use a value you can look up later — a request ID, a database primary key, or a UUID generated at request time.

Methods

Promise<void>

Persist the current run state. Called automatically by the Anvil runtime when you pass checkpoint to runAgent or streamAgent; call it manually only if you are driving the loop yourself.

Promise<AgentCheckpoint | undefined>

Retrieve the most recently saved checkpoint. Returns undefined if no checkpoint exists for this runId.

Promise<void>

Remove the checkpoint after a run completes successfully. Call this in your cleanup logic to avoid unbounded growth of the checkpoint store.

AgentCheckpoint fields

interface AgentCheckpoint {
  version: 1;
  runId: string;
  status: 'running' | 'suspended' | 'done';
  messages: ModelMessage[];
  iterations: number;
  totalUsage: { inputTokens: number; outputTokens: number };
  totalCostUsd: number;
  approvals?: Record<string, unknown>; // completed tool results keyed by call id
  pending?: { callId: string; payload: unknown }; // set when suspended for HITL
  finalText?: string;
  updatedAt: number; // Unix ms of the last save
}

'running' | 'suspended' | 'done'

Current lifecycle state. 'suspended' means the run is waiting for human approval; 'done' means it completed (successfully or by hitting the iteration cap).

Record<string, unknown>

Tool call results already persisted. When the run resumes, any tool call whose callId appears in this map is not re-executed — its stored output is used directly. This prevents side-effect tools from running twice.

{ callId: string; payload: unknown }

Set when status is 'suspended'. Contains the call ID and the payload the tool passed to ApprovalRequiredError, which the approver sees in the HITL queue.

Marking tools as side effects

Declare sideEffect: true on any tool that mutates external state — charges a card, sends an email, creates a database row. Anvil checkpoints after the tool completes and fences its result so a resume can never re-execute it:

const chargeCard: AgentTool = {
  name: 'chargeCard',
  description: 'Charge a customer payment card.',
  sideEffect: true,
  execute: async ({ amount, customerId }) => {
    const result = await paymentProvider.charge(customerId, amount);
    return result.transactionId;
  },
};

Running an agent with a checkpoint

Pass checkpoint to runAgent or streamAgent. Anvil instantiates a Checkpointer internally and writes checkpoints throughout the run:

import { runAgent } from 'anvil-sdk/agent';
import { SqliteStateStore } from 'anvil-sdk/store';

const stateStore = await SqliteStateStore.open('.anvil/state.db');
const runId = crypto.randomUUID();

const result = await runAgent({
  client,
  messages,
  tools: [lookupOrder, chargeCard],
  checkpoint: { store: stateStore, runId },
});

Resuming after a crash

If the process was interrupted, call resumeAgent with the same runId. It loads the checkpoint, restores message history and usage counters, and re-enters the loop from the last saved state:

import { resumeAgent } from 'anvil-sdk/agent';

const result = await resumeAgent({
  client,
  tools: [lookupOrder, chargeCard],
  checkpoint: { store: stateStore, runId: 'run-abc-123' },
  // Pass approval if resuming a HITL-suspended run:
  // approval: { approved: true, reviewer: 'alice@example.com' },
});

resumeAgent returns immediately if the checkpoint has status: 'done' — handy for idempotent retry logic.

Listing all persisted runs

Use listRuns to retrieve every run ID that has an active checkpoint. This is useful for building an admin queue of in-progress or suspended runs:

import { listRuns } from 'anvil-sdk/agent';

const runIds = await listRuns(stateStore);
// ['run-abc-123', 'run-def-456', ...]

listRuns returns only run IDs, not the full checkpoints. Load a specific checkpoint with new Checkpointer(store, runId).load() to inspect its state.

The state store

Durable execution uses StateStore — a simple key-value interface separate from TraceStore. The production default is SqliteStateStore:

import { SqliteStateStore, MemoryStateStore } from 'anvil-sdk/store';

// Production
const store = await SqliteStateStore.open('.anvil/state.db');

// Tests
const store = new MemoryStateStore();

You can share one SqliteStateStore across multiple agent routes, since each run uses a unique runId-namespaced key.

Pair durable execution with Human-in-the-Loop approvals. When a tool suspends the run, the suspended checkpoint holds the full state — the HITL queue in /_anvil shows it, and resumeAgent resumes it with the approval decision.

Get Started

Routing

MCP & Tools

Agent Routes

Observability

Safety & Durability

Memory & RAG

Background Tasks

Durable Execution: Checkpoint and Resume Agent Runs

How checkpointing works

Checkpointer API

Methods

AgentCheckpoint fields

Marking tools as side effects

Running an agent with a checkpoint

Resuming after a crash

Listing all persisted runs

The state store

​How checkpointing works

​Checkpointer API

​Methods

​AgentCheckpoint fields

​Marking tools as side effects

​Running an agent with a checkpoint

​Resuming after a crash

​Listing all persisted runs

​The state store

How checkpointing works

Checkpointer API

Methods

AgentCheckpoint fields

Marking tools as side effects

Running an agent with a checkpoint

Resuming after a crash

Listing all persisted runs

The state store