# defineAgent: Create Streaming Anvil Agent Handlers
Source: https://anvil.thatdevguy.in/agents/define-agent
Use defineAgent to turn any route file into a streaming AI agent. Configure the model client, system prompt, tools, and context pipeline in one place.
`defineAgent` is the single function you need to build an agent route. It returns a route handler that runs the model↔tool loop and streams the Vercel AI SDK data stream protocol back to the client. Export it as the default from an `agent.ts` file (or any `post.ts`) and Anvil mounts it automatically.
## Quick start
The example below creates a fully working agent route at `POST /chat` that uses Anthropic's Claude, answers questions, and can call a weather tool:
```ts theme={null}
// server/routes/chat/agent.ts
import { defineAgent } from 'anvil-sdk/agent';
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
import { z } from 'zod';
const client = new LlmClient({
drivers: [new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })],
defaultModel: 'claude-opus-4-8',
fallback: ['gpt-4o'], // requires an OpenAIDriver too
});
export default defineAgent({
client,
system: 'You are a helpful assistant.',
tools: [
{
name: 'get_weather',
description: 'Get current weather for a city',
zodSchema: z.object({ city: z.string() }),
execute: ({ city }) => fetchWeather(city),
},
],
});
```
## Configuration reference
`defineAgent(config: DefineAgentConfig)` accepts the following options:
The `LlmClient` instance to use for model calls. If omitted, the runtime falls back to `resolveClient` or the client attached by a `withLlm()` middleware.
A per-request factory for the `LlmClient`. Useful when the client depends on request state (e.g. a per-tenant API key stored in `ctx.state`). Takes precedence over a middleware-attached client but not over an inline `client`.
Override the default model for this specific agent route. The value is passed through to the `LlmClient` and must be supported by one of its drivers.
The system prompt. Pass a plain string for a static prompt, or an async function to build the prompt from request context — for example, to inject the authenticated user's name.
The tools available to the model during this run. Pass an array for a fixed set or a function to assemble tools dynamically from request context (e.g. scoped by user permissions).
Hard cap on the number of model↔tool turns. Defaults to `10`. When the cap is reached the stream ends with a `final` event — the model's last text (if any) is still returned. Use this to bound cost and latency on open-ended agent tasks.
A `Tracer` instance for trace instrumentation. Each request opens a named trace; model calls and tool executions are recorded as child spans. Traces appear in the `/_anvil` dashboard.
The name given to the trace opened for each request. Defaults to `"agent "`. Pass a string for a static name, or a function to derive it from the request context (e.g. to include the user ID).
Per-request cost cap. The runtime checks the accumulated cost before each model call and aborts the loop if the budget is exceeded.
An ordered array of guardrail policies applied to model output text and tool calls. Use the built-in helpers (`contentFilter`, `toolPolicy`, `injectionGuard`) or implement the `Guardrail` interface directly.
A context assembly pipeline that runs once before the loop starts. Steps can inject retrieved documents as system addenda (RAG), trim the message history to a token budget, or add static context fragments. Built-in steps: `retrievalContext`, `tokenBudget`, `systemContext`.
Custom message extractor. By default `defineAgent` reads `body.messages` — the standard AI SDK `useChat` request shape. Override this when your request body has a different structure.
## Attaching a client with middleware
Use `withLlm` when you want to share one `LlmClient` instance across many routes without importing it into every file:
```ts theme={null}
// server/routes/_middleware.ts
import { withLlm } from 'anvil-sdk/agent';
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
const client = new LlmClient({
drivers: [new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })],
defaultModel: 'claude-opus-4-8',
});
export default [withLlm(client)];
```
```ts theme={null}
// server/routes/chat/agent.ts
import { defineAgent } from 'anvil-sdk/agent';
// No `client` needed — withLlm() middleware attached it to ctx.state.llm
export default defineAgent({
system: 'You are a helpful assistant.',
});
```
`withLlm` is a regular Anvil middleware. Place it in a `_middleware.ts` file to scope it to a subtree of routes, or at the root to share the client across your entire application.
## Dynamic system prompt
Pass an async function to `system` to build the prompt from the incoming request:
```ts theme={null}
export default defineAgent({
client,
system: async (ctx) => {
const user = await getUser(ctx.state.userId);
return `You are a helpful assistant for ${user.name}. Their account tier is ${user.tier}.`;
},
tools: [...],
});
```
## Custom message extraction
Override `getMessages` when your frontend sends a request body that doesn't match the AI SDK `useChat` shape:
```ts theme={null}
export default defineAgent({
client,
system: 'You are a support bot.',
getMessages: async (ctx) => {
const body = await ctx.body<{ conversation: Array<{ from: string; text: string }> }>();
return body.conversation.map((m) => ({
role: m.from === 'agent' ? 'assistant' : 'user',
content: m.text,
}));
},
});
```
## Dynamic tools from context
Load tools based on the authenticated user's permissions:
```ts theme={null}
export default defineAgent({
client,
system: 'You are a helpful assistant.',
tools: async (ctx) => {
const scopes = ctx.state.scopes as string[];
const tools = [alwaysAvailableTool];
if (scopes.includes('admin')) tools.push(adminTool);
return tools;
},
});
```
# LlmClient: Provider-Agnostic Model Calls in Anvil JS
Source: https://anvil.thatdevguy.in/agents/llm-client
LlmClient wraps Anthropic, OpenAI, and Gemini behind one interface. Configure fallback chains, retries, structured output, and observability hooks.
`LlmClient` is the provider-agnostic model client at the heart of Anvil's agent system. It owns driver dispatch, transient retries, fallback chains, cost accumulation, and structured-output enforcement — so your agent handlers call `generate`, `generateObject`, or `stream` without worrying about provider-specific details.
Import it from `anvil/llm`:
```ts theme={null}
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
```
## Constructor
```ts theme={null}
const client = new LlmClient(options: LlmClientOptions);
```
One or more driver instances that handle model calls. Each driver advertises which models it supports via `driver.supports(model)`. At least one driver is required. Add multiple drivers to cover different model families.
The model used when a `generate`, `generateObject`, or `stream` call omits the `model` field. If omitted here too, the call throws unless the request specifies a model explicitly.
An ordered list of model IDs to try after the primary model fails with a transient error (`RetryableModelError`). The client walks the list left-to-right, retrying `maxRetries` times per model before advancing. Each fallback model must be supported by one of the registered drivers.
Attempts per model before moving to the next in the fallback chain. Defaults to `2`.
Base backoff delay in milliseconds between retry attempts. The actual wait before each attempt is `retryBaseMs * attemptNumber`. Defaults to `50`. Set to `0` in tests to eliminate wait time.
Observability hook called after every model call — successful or failed. The `TraceEvent` carries the model name, provider, attempt number, token usage, cost, and whether the call hit the fallback chain. Use this to feed your own metrics or logging pipeline.
## Drivers
Provider SDKs are optional peer dependencies loaded lazily at runtime — the Anvil kernel never pulls them in unless you instantiate the matching driver.
```ts theme={null}
import { AnthropicDriver } from 'anvil-sdk/llm';
new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })
```
Requires `@anthropic-ai/sdk` as a peer dependency. Supports all `claude-*` models (e.g. `claude-opus-4-8`, `claude-sonnet-4-5`, `claude-haiku-3-5`). Uses the top-level `system` parameter and adaptive thinking when `thinking: true` is set on the request.
```ts theme={null}
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
const client = new LlmClient({
drivers: [new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })],
defaultModel: 'claude-opus-4-8',
});
```
```ts theme={null}
import { OpenAIDriver } from 'anvil-sdk/llm';
new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY })
```
Requires `openai` as a peer dependency. Supports `gpt-*` models (`gpt-4o`, `gpt-4o-mini`) and `o`-series reasoning models (`o1`, `o3`, `o4-mini`). Uses the Chat Completions API with structured output via `response_format.json_schema`.
```ts theme={null}
import { LlmClient, OpenAIDriver } from 'anvil-sdk/llm';
const client = new LlmClient({
drivers: [new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY })],
defaultModel: 'gpt-4o',
});
```
```ts theme={null}
import { GeminiDriver } from 'anvil-sdk/llm';
new GeminiDriver({ apiKey: process.env.GEMINI_API_KEY })
```
Requires `@google/generative-ai` as a peer dependency. Supports all `gemini-*` models (e.g. `gemini-2.5-flash`, `gemini-2.5-pro`). Gemini function calls carry no IDs; the driver synthesizes stable IDs from the function name so the tool loop works correctly.
```ts theme={null}
import { LlmClient, GeminiDriver } from 'anvil-sdk/llm';
const client = new LlmClient({
drivers: [new GeminiDriver({ apiKey: process.env.GEMINI_API_KEY })],
defaultModel: 'gemini-2.5-flash',
});
```
## Methods
### `client.generate(req)`
Make a single-turn model call and return the complete result.
```ts theme={null}
const result = await client.generate({
model: 'claude-opus-4-8', // optional if defaultModel is set
system: 'You are a helpful assistant.',
messages: [{ role: 'user', content: 'What is the capital of France?' }],
});
console.log(result.text); // "Paris"
console.log(result.usage); // { inputTokens: 14, outputTokens: 6 }
console.log(result.costUsd); // 0.000042
```
The model's text response.
The model ID that produced the response (may differ from the request if the fallback chain was used).
The provider name, e.g. `"anthropic"`, `"openai"`, `"gemini"`.
Token counts for this call.
Estimated USD cost from the built-in pricing table. `undefined` if pricing data for the model isn't registered.
Tool invocations the model requested, when tools were provided and `stopReason` is `"tool_use"`.
***
### `client.generateObject(req, schema, opts?)`
Request structured output validated against a Zod schema. On validation failure the client automatically re-prompts the model with the validation error, up to `maxRepairs` times.
```ts theme={null}
import { z } from 'zod';
const SentimentSchema = z.object({
sentiment: z.enum(['positive', 'neutral', 'negative']),
confidence: z.number().min(0).max(1),
});
const result = await client.generateObject(
{
system: 'Classify the sentiment of the user message.',
messages: [{ role: 'user', content: 'This product is amazing!' }],
},
SentimentSchema,
{ maxRepairs: 2, name: 'sentiment_analysis' },
);
console.log(result.object.sentiment); // "positive"
console.log(result.object.confidence); // 0.97
```
The same request shape as `generate`. Do not set `responseFormat` manually — `generateObject` sets it automatically from the Zod schema.
The Zod schema to validate the response against.
Maximum number of repair attempts after a validation failure. Defaults to `2`. Each attempt re-prompts the model with the Zod error message.
Optional name for the JSON schema advertised to the model. Helps some providers choose a better response format.
***
### `client.stream(req)`
Stream tokens from the model as they arrive. Yields `StreamEvent` objects.
```ts theme={null}
for await (const event of client.stream({
messages: [{ role: 'user', content: 'Tell me a short story.' }],
})) {
if (event.type === 'text') process.stdout.write(event.text);
if (event.type === 'done') console.log('\nUsage:', event.result.usage);
}
```
A text fragment from the model. Concatenate these to build the full response.
The final event, carrying the complete `GenerateResult` with usage, cost, and stop reason.
Streaming does not retry mid-stream — bytes may already be on the wire. The client only falls back to the next model in the chain if the stream fails before yielding any events.
## Fallback chain example
Register both Anthropic and OpenAI drivers, set Claude as the primary, and fall back to GPT-4o on transient errors:
```ts theme={null}
import { LlmClient, AnthropicDriver, OpenAIDriver } from 'anvil-sdk/llm';
const client = new LlmClient({
drivers: [
new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY }),
new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY }),
],
defaultModel: 'claude-opus-4-8',
fallback: ['gpt-4o'],
maxRetries: 2,
onTrace: (event) => {
if (event.fallback) {
console.warn(`Fell back to ${event.model} after primary model failed`);
}
},
});
```
The client tries `claude-opus-4-8` up to two times. If both attempts throw a `RetryableModelError` (rate limit, 5xx, connection error) it moves to `gpt-4o` and tries that up to two times. Non-retryable errors (bad request, auth failure) stop the chain immediately.
Install only the provider SDKs you need. Adding `AnthropicDriver` to your drivers list does not pull in `@anthropic-ai/sdk` until the first request that routes through it.
# Multi-Agent Orchestration with Anvil AgentRegistry
Source: https://anvil.thatdevguy.in/agents/orchestration
Use Anvil AgentRegistry to register agents, wrap them as tools, and delegate work in-process — no HTTP round-trips, no separate orchestration framework.
Anvil's multi-agent system is built from the same primitives as everything else: route handlers, tools, and `LlmClient`. `AgentRegistry` lets you register named agents and invoke them from code or from inside another agent's tool loop. Sub-agents run entirely in-process — there are no HTTP round-trips, no additional servers, and no separate orchestration framework to learn.
## `AgentRegistry`
Import `AgentRegistry` from `anvil-sdk/agent`:
```ts theme={null}
import { AgentRegistry } from 'anvil-sdk/agent';
const registry = new AgentRegistry();
```
### `registry.register(name, config)`
Register a named agent with its `RunAgentOptions` (the same options `runAgent` accepts, minus `messages`):
```ts theme={null}
registry.register('summarizer', {
client,
system: 'You are an expert summarizer. Return a concise summary in 2–3 sentences.',
maxIterations: 2,
});
registry.register('researcher', {
client,
system: 'You are a research assistant. Use tools to find accurate information.',
tools: [searchTool, fetchPageTool],
maxIterations: 8,
});
```
`register` returns `this`, so you can chain calls.
### `registry.call(name, input, overrides?)`
Invoke a registered agent by name. Pass a string for a single user message, or a full `ModelMessage[]` conversation. Returns `AgentRunResult`.
```ts theme={null}
const result = await registry.call('summarizer', 'Summarize the French Revolution in 3 sentences.');
console.log(result.text);
console.log(result.iterations); // how many model turns it took
console.log(result.totalCostUsd); // accumulated cost
```
Pass `overrides` to adjust config for a specific call without modifying the registration:
```ts theme={null}
const result = await registry.call('researcher', userQuery, {
maxIterations: 3, // tighten the cap for this call
model: 'claude-haiku-3-5', // use a cheaper model
});
```
The final text response from the sub-agent.
The full conversation, including all tool calls and results.
The number of model turns that ran.
Accumulated token counts across all turns.
Accumulated USD cost.
`true` if the loop stopped because it hit `maxIterations` rather than finishing naturally.
### `registry.has(name)` and `registry.names()`
Check membership and list registered names:
```ts theme={null}
registry.has('summarizer'); // true
registry.names(); // ['summarizer', 'researcher']
```
## `agentAsTool`
Wrap a registered agent as an `AgentTool` so an orchestrator agent can delegate to it during its own tool loop. The sub-agent's final text becomes the tool result:
```ts theme={null}
import { agentAsTool } from 'anvil-sdk/agent';
const researchTool = agentAsTool(registry, 'researcher', {
toolName: 'research', // optional; defaults to `call_researcher`
description: 'Delegate a research task to a specialist agent. Provide a detailed question.',
});
```
The registry that contains the target agent.
The name of the registered agent to wrap.
The tool name advertised to the parent model. Defaults to `call_`.
The tool description. Defaults to a generic delegation prompt that includes the agent name.
Config overrides applied every time the parent model delegates to this sub-agent.
The returned tool uses a fixed `message` input schema — the parent model passes a plain string describing the subtask, which becomes the sub-agent's first user message.
## `withAgents` middleware and `callAgent` helper
Use `withAgents` to attach a registry to `ctx.state.agents`, then use `callAgent` anywhere in your handler tree:
```ts theme={null}
// server/routes/_middleware.ts
import { withAgents } from 'anvil-sdk/agent';
import { registry } from '../agents';
export default [withAgents(registry)];
```
```ts theme={null}
// server/routes/report/post.ts
import { callAgent } from 'anvil-sdk/agent';
import type { Context } from 'anvil-sdk';
export default async function handler(ctx: Context) {
const body = await ctx.body<{ topic: string }>();
const result = await callAgent(ctx, 'researcher', body.topic);
return { summary: result.text };
}
```
`callAgent(ctx, name, input)` is sugar for `getAgents(ctx).call(name, input)`. It throws a clear error if no registry is attached.
## Full orchestration example
This example registers a researcher and a summarizer, then wires them together in a parent orchestrator agent that the client talks to:
```ts theme={null}
// server/agents.ts
import { AgentRegistry } from 'anvil-sdk/agent';
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
import { searchTool } from './tools/search';
const client = new LlmClient({
drivers: [new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })],
defaultModel: 'claude-opus-4-8',
});
export const registry = new AgentRegistry()
.register('researcher', {
client,
system: 'You are a research assistant. Use the search tool to find accurate, up-to-date information.',
tools: [searchTool],
maxIterations: 6,
})
.register('summarizer', {
client,
system: 'You are an expert summarizer. Produce a concise 2–3 sentence summary from the provided text.',
maxIterations: 2,
});
```
```ts theme={null}
// server/routes/report/agent.ts
import { defineAgent, agentAsTool } from 'anvil-sdk/agent';
import { registry } from '../../agents';
import { client } from '../../llm';
export default defineAgent({
client,
system: `You are an orchestrator. For each user question:
1. Use the research tool to gather information.
2. Use the summarize tool to condense it.
3. Return the final summary to the user.`,
maxIterations: 6,
tools: [
agentAsTool(registry, 'researcher', {
toolName: 'research',
description: 'Research a question using live search. Provide a detailed question.',
}),
agentAsTool(registry, 'summarizer', {
toolName: 'summarize',
description: 'Summarize a block of text into 2–3 sentences. Provide the full text.',
}),
],
});
```
The `POST /report` route now orchestrates the two sub-agents. The orchestrator model decides when to call `research` and `summarize`; each sub-agent runs to completion in-process and returns its text as the tool result.
Sub-agents run synchronously in the parent's event loop via `runAgent` — no network calls, no spawned processes. The parent agent's `maxIterations` counts its own turns; sub-agent turns are independent.
## Sub-agent execution model
`agentAsTool` calls `registry.call(name, message)` inside `execute`, which calls `runAgent` directly. The entire sub-agent loop runs to completion before `execute` returns. This means:
* **No HTTP round-trips** — the sub-agent never makes an HTTP request to itself.
* **No shared streaming** — the parent stream does not include the sub-agent's intermediate events; only its final text appears as a tool result.
* **Nested cost tracking** — each sub-agent call uses the client's internal cost accumulator, so `client.totalCostUsd` reflects the total across all agents sharing that client.
Register sub-agents with their own `LlmClient` instances to use cheaper or faster models for specific tasks, while reserving your most capable model for the top-level orchestrator.
# Anvil JS Agent Routes: Build Streaming AI Agents
Source: https://anvil.thatdevguy.in/agents/overview
Build streaming AI agents in Anvil JS. Drop an agent.ts into any route folder and Anvil streams responses using the Vercel AI SDK data protocol.
Agent routes are the agentic core of Anvil JS. Place an `agent.ts` file inside any folder under `server/routes/` and Anvil automatically mounts it as a `POST` endpoint that streams the [Vercel AI SDK data stream protocol](https://sdk.vercel.ai/docs/ai-sdk-ui/stream-protocol). Because the wire format is compatible, you can point `useChat` directly at an agent route with no extra adapter code.
## How an agent route works
An `agent.ts` file exports a handler built with `defineAgent`. When a request arrives, the runtime:
1. Extracts the `messages` array from the JSON request body (AI SDK `useChat` shape by default).
2. Runs the **model↔tool loop** — calls the model, executes any tools the model requests, feeds results back, and repeats.
3. Streams each text token, tool call, and tool result back to the client as data stream parts.
4. Emits a final summary event with token usage and cost once the model produces a response with no tool calls (or the iteration cap is reached).
The runtime enforces several safety properties automatically:
* **Iteration cap** — `maxIterations` (default `10`) prevents the loop from running forever on a runaway tool-calling sequence.
* **AbortSignal propagation** — when the HTTP client disconnects, the cancellation signal flows into every model call and tool execution. Partial work is stopped immediately.
* **Zod schema validation** — each tool's `zodSchema` is checked before `execute` runs. Invalid inputs return an error result to the model rather than crashing the handler.
* **Token tracking** — input and output tokens are accumulated across every turn and reported in the `final` event.
## File conventions
```text theme={null}
server/routes/
chat/
agent.ts → POST /chat (agent route)
summarize/
post.ts → POST /summarize (also works with defineAgent)
```
Both `agent.ts` and `post.ts` work. Use `agent.ts` as the conventional name for routes whose primary purpose is running an agent loop.
## Connecting a frontend
Because Anvil streams the Vercel AI SDK data stream protocol, `useChat` works out of the box:
```tsx theme={null}
import { useChat } from 'ai/react';
export function Chat() {
const { messages, input, handleInputChange, handleSubmit } = useChat({
api: '/chat',
streamProtocol: 'data',
});
return (
);
}
```
`streamProtocol: 'data'` is required. Anvil uses the prefix-coded data stream format, not raw text streaming.
## What's in this section
The `defineAgent` API — system prompts, tools, context pipelines, budgets, and the `withLlm` middleware.
`LlmClient` and its drivers for Anthropic, OpenAI, and Gemini — plus fallback chains and structured output.
The `AgentTool` interface — Zod schemas, side-effect fencing, and the `maxIterations` safeguard.
`AgentRegistry`, `agentAsTool`, and `withAgents` — compose multi-agent systems without HTTP round-trips.
# Anvil Agent Tools: Zod Schemas and Execute Functions
Source: https://anvil.thatdevguy.in/agents/tools
Define Anvil agent tools with Zod schemas and execute functions. The runtime validates inputs and feeds results back to the model in the tool loop.
Tools are the primary way your agent interacts with the outside world — fetching data, querying databases, calling APIs, and performing computations. Each tool you pass to `defineAgent` is advertised to the model, and the runtime handles the full call-validate-execute cycle automatically.
## The `AgentTool` interface
Every tool you define implements this interface:
```ts theme={null}
interface AgentTool {
name: string;
description: string;
zodSchema?: z.ZodType;
inputSchema?: JsonSchema;
sideEffect?: boolean;
execute: (input: Input, meta: ToolExecMeta) => Promise | unknown;
}
```
A unique identifier for the tool. The model uses this name when requesting a call. Use `snake_case` — most model providers require it.
A plain-English description of what the tool does. The model reads this to decide when to call the tool, so write it from the model's perspective: what information does the tool provide, and when should it be used?
A Zod schema for the tool's input. **Preferred over `inputSchema`** — the runtime converts it to JSON Schema automatically (to advertise to the model) and validates the model's input against it before calling `execute`. If validation fails, the model receives a descriptive error result instead of a crash.
A raw JSON Schema for the tool's input. Use this if you cannot express the input as a Zod schema. If both `zodSchema` and `inputSchema` are present, `zodSchema` is used for validation and `inputSchema` is ignored.
Annotates the tool as one that mutates external state (sends an email, writes to a database, calls a payment API, etc.). This is a documentation signal — use it to communicate intent to other developers reading the agent definition.
The async function that runs when the model calls this tool. Return any serializable value — it becomes the tool result fed back to the model. Throw an error to return an error result.
## The `ToolExecMeta` argument
`execute` receives a second `meta` argument with runtime context:
The unique ID for this tool invocation, assigned by the model. Useful for correlating logs and traces.
The request's `AbortSignal`. Pass it to any `fetch` calls or async work inside the tool so that a client disconnect cancels in-flight I/O.
Throw this to suspend the run and request human approval before proceeding (human-in-the-loop). The run is checkpointed and the client receives a `suspended` event. Resume with `resumeAgent` after the operator approves.
## Complete tool example
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
import { z } from 'zod';
const client = new LlmClient({
drivers: [new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })],
defaultModel: 'claude-opus-4-8',
});
export default defineAgent({
client,
system: 'You are a helpful assistant. Use the provided tools to answer questions.',
maxIterations: 5,
tools: [
{
name: 'search_products',
description: 'Search the product catalog by keyword. Returns a list of matching products with name, price, and stock status.',
zodSchema: z.object({
query: z.string().describe('The search keyword or phrase'),
limit: z.number().int().min(1).max(20).default(5).describe('Maximum results to return'),
}),
execute: async ({ query, limit }, meta) => {
const results = await fetch(
`/internal/products?q=${encodeURIComponent(query)}&limit=${limit}`,
{ signal: meta.signal },
).then((r) => r.json());
return results;
},
},
{
name: 'place_order',
description: 'Place an order for a product. Only call this after confirming the product ID and quantity with the user.',
sideEffect: true, // annotates this tool as one that mutates external state
zodSchema: z.object({
productId: z.string(),
quantity: z.number().int().min(1),
}),
execute: async ({ productId, quantity }, meta) => {
// Request human approval before placing the order
if (quantity > 10) {
meta.requestApproval({ productId, quantity, reason: 'Large order requires approval' });
}
const order = await createOrder(productId, quantity);
return { orderId: order.id, status: 'placed' };
},
},
],
});
```
## Input validation
The runtime validates tool inputs against `zodSchema` before calling `execute`. You never need to validate inside the function body:
```ts theme={null}
// The runtime already ensures `city` is a non-empty string when execute runs.
{
name: 'get_weather',
description: 'Get the current weather for a city',
zodSchema: z.object({ city: z.string().min(1) }),
execute: ({ city }) => fetchWeather(city),
}
```
If the model sends invalid input (a missing required field, a number where a string is expected), the runtime returns a descriptive Zod error message to the model as the tool result. The model can then correct its call on the next iteration.
Always use `zodSchema` for user-facing tools. Without it, malformed model output reaches `execute` unvalidated. Raw `inputSchema` provides no runtime protection — it only shapes what the model is told.
## Preventing infinite loops with `maxIterations`
By default the agent runtime caps the model↔tool loop at **10 iterations**. Each call to the model counts as one iteration, regardless of how many tools it calls in that turn. Set `maxIterations` on `defineAgent` to tighten or loosen this limit:
```ts theme={null}
export default defineAgent({
client,
maxIterations: 3, // stop after at most 3 model calls
tools: [...],
});
```
When the cap is reached the runtime emits a `final` event with whatever text the model produced last. The `stoppedAtCap` flag in `AgentRunResult` tells you whether the run ended this way.
Set `maxIterations` to a small value (2–3) for tools that should complete in one or two steps, and only raise it for complex multi-step workflows. Smaller caps reduce latency and cost when a runaway tool loop occurs.
## Standalone tools and `anvil mcp`
Tools defined inline in `defineAgent` are only available to that agent route. If you also want a tool served via the MCP protocol (for use with Claude Desktop or other MCP clients), define it as a standalone tool in `server/tools/`:
```ts theme={null}
// server/tools/wordCount.ts
import { z } from 'zod';
export const description = 'Count the words in a piece of text';
export const inputSchema = z.object({ text: z.string() });
export default async function wordCount(args: { text: string }) {
const words = args.text.trim().split(/\s+/).filter(Boolean);
return { words: words.length, characters: args.text.length };
}
```
Running `anvil mcp` serves both `server/tools/*.ts` files and any MCP-exposed route-derived tools from the same endpoint. Route-level tools defined in `defineAgent` are not automatically MCP-exposed — only `server/tools/` files and routes with `meta.mcp.expose: true` appear there.
# A2A Protocol: Expose Agents via Agent-to-Agent Standard
Source: https://anvil.thatdevguy.in/background/a2a-protocol
Expose your Anvil agents over Google's Agent2Agent protocol alongside REST and MCP from a single route definition, using A2AServer and a2aHttpHandler.
Anvil supports Google's Agent2Agent (A2A) protocol, letting your agents communicate with other A2A-compatible systems using a standardized interface. The same agent definition that serves a REST endpoint and an MCP server can also serve an A2A endpoint — you choose which protocols to enable. A2A, MCP, and REST share the same routing, tool registry, and observability machinery, so there is no duplication of logic.
The A2A specification is still evolving. Anvil isolates all protocol logic behind an adapter layer (`A2AServer` and `a2aHttpHandler`), so spec updates require changes only to the adapter — not to your agent definitions.
## What is A2A?
Agent2Agent (A2A) is an open protocol proposed by Google for standardized communication between AI agents. It allows one agent to call another agent's capabilities using a JSON-RPC 2.0 transport. Agents advertise their skills via an Agent Card — a JSON document served at a well-known URL — and callers discover and invoke those skills via the `message/send` method.
Anvil's implementation covers:
* **Agent Card** — served at `/.well-known/agent-card.json` (and the alias `/agent.json`) so external agents can discover your skills
* **`message/send`** — synchronous task execution; calls the target skill and returns a completed Task artifact
* **`tasks/get`** — retrieve a previously created task by ID
## Setting up an A2A server
`A2AServer` dispatches incoming A2A requests to agents by skill ID. Each skill ID must match a registered agent name.
```ts theme={null}
import { AgentRegistry } from 'anvil-sdk/agent';
import { LlmClient } from 'anvil-sdk/llm';
const client = new LlmClient({ /* your driver config */ });
const registry = new AgentRegistry();
registry.register('support', {
client,
model: 'gpt-4o-mini',
system: 'You are a helpful customer support agent.',
});
registry.register('summarizer', {
client,
model: 'gpt-4o-mini',
system: 'Summarize the provided text concisely.',
});
```
```ts theme={null}
import { A2AServer } from 'anvil-sdk/a2a';
const a2a = new A2AServer({
registry,
name: 'Acme AI Gateway',
description: 'Customer-facing AI agents for Acme Corp.',
url: 'https://api.acme.com/a2a',
version: '1.0.0',
// Optional: explicitly declare skills. Defaults to all registered agents.
skills: [
{
id: 'support',
name: 'Customer Support',
description: 'Answers product and account questions.',
tags: ['support', 'faq'],
},
{
id: 'summarizer',
name: 'Text Summarizer',
description: 'Summarizes long text into key points.',
tags: ['summarization', 'nlp'],
},
],
defaultSkill: 'support', // used when a request names no skill
});
```
`a2aHttpHandler` returns a standard `fetch`-compatible handler you can mount directly in your Anvil app.
```ts theme={null}
import { a2aHttpHandler } from 'anvil-sdk/a2a';
import { createApp } from 'anvil-sdk';
const app = createApp();
const handleA2A = a2aHttpHandler(a2a, {
path: '/a2a', // JSON-RPC endpoint (default)
cardPath: '/.well-known/agent-card.json', // Agent Card URL (default)
});
// Mount the A2A handler alongside your existing routes
app.all('/a2a', handleA2A);
app.get('/.well-known/agent-card.json', handleA2A);
app.get('/agent.json', handleA2A); // alias, served automatically
export default app;
```
## A2AServer API
### Constructor options
| Option | Type | Required | Description |
| -------------- | --------------- | -------- | ------------------------------------------------------------------------------------------------------ |
| `registry` | `AgentRegistry` | ✅ | The agent registry that backs skill execution |
| `name` | `string` | ✅ | Human-readable name for this A2A server, advertised in the Agent Card |
| `description` | `string` | ✅ | Human-readable description advertised in the Agent Card |
| `url` | `string` | ✅ | The public URL of the A2A endpoint, included in the Agent Card |
| `version` | `string` | — | Server version string. Defaults to `'0.0.1'` |
| `skills` | `A2ASkill[]` | — | Skills to advertise. Each `id` must be a registered agent name. Defaults to all agents in the registry |
| `defaultSkill` | `string` | — | Skill to invoke when `message/send` includes no `skillId`. Defaults to the first skill |
### `server.agentCard()`
Returns the `AgentCard` object that describes this server's capabilities. The HTTP handler serves this automatically at the card path.
```ts theme={null}
const card = a2a.agentCard();
// {
// protocolVersion: '0.3.0',
// name: 'Acme AI Gateway',
// capabilities: { streaming: false },
// skills: [...],
// ...
// }
```
### `server.handle(message)`
Process a raw JSON-RPC 2.0 request object. Returns a JSON-RPC response or `null` for notifications (requests with no `id`). You can call this directly if you are integrating with a custom transport.
```ts theme={null}
const response = await a2a.handle({
jsonrpc: '2.0',
id: 1,
method: 'message/send',
params: {
message: {
role: 'user',
parts: [{ kind: 'text', text: 'What is your return policy?' }],
},
metadata: { skillId: 'support' },
},
});
```
## Supported JSON-RPC methods
| Method | Description |
| -------------- | ------------------------------------------------------------------------------- |
| `message/send` | Send a message to a skill and receive a completed `Task` artifact synchronously |
| `tasks/get` | Retrieve a previously created task by its `id` |
Any other method returns a `-32601 Unknown method` error.
## AgentCard and skill discovery
External agents discover your capabilities by fetching the Agent Card from `/.well-known/agent-card.json`. Anvil serves it automatically:
```json theme={null}
{
"protocolVersion": "0.3.0",
"name": "Acme AI Gateway",
"description": "Customer-facing AI agents for Acme Corp.",
"url": "https://api.acme.com/a2a",
"version": "1.0.0",
"capabilities": { "streaming": false },
"defaultInputModes": ["text/plain"],
"defaultOutputModes": ["text/plain"],
"skills": [
{
"id": "support",
"name": "Customer Support",
"description": "Answers product and account questions.",
"tags": ["support", "faq"]
}
]
}
```
## A2ASkill fields
| Field | Type | Required | Description |
| ------------- | ---------- | -------- | -------------------------------------------------------------- |
| `id` | `string` | ✅ | Must match a registered agent name in the `AgentRegistry` |
| `name` | `string` | ✅ | Human-readable skill name |
| `description` | `string` | ✅ | What the skill does — shown to calling agents during discovery |
| `tags` | `string[]` | — | Optional categorization tags |
## a2aHttpHandler options
| Option | Type | Default | Description |
| ---------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------- |
| `path` | `string` | `'/a2a'` | The path for the JSON-RPC endpoint |
| `cardPath` | `string` | `'/.well-known/agent-card.json'` | The path to serve the Agent Card. The alias `/.well-known/agent.json` is always served at this path too |
## One definition, three protocols
Because `A2AServer` wraps the same `AgentRegistry` your application already uses, enabling A2A requires only a few extra lines — your agent logic and tools are defined once and reused:
```ts theme={null}
import { createApp } from 'anvil-sdk';
import { a2aHttpHandler, A2AServer } from 'anvil-sdk/a2a';
import { registry } from './agents.js'; // shared AgentRegistry
const app = createApp();
// REST — agents are already mounted as HTTP route handlers
// A2A — same registry, A2A transport
const a2aServer = new A2AServer({ registry, name: 'My App', description: 'My Anvil app', url: 'https://example.com/a2a' });
app.all('/a2a', a2aHttpHandler(a2aServer));
app.get('/.well-known/agent-card.json', a2aHttpHandler(a2aServer));
export default app;
```
Enable only the transports your architecture requires. Each transport adapter adds a single route — you incur no overhead for protocols you do not mount.
# Scheduled Agents: Run Background Tasks on a Schedule
Source: https://anvil.thatdevguy.in/background/scheduled-agents
Use defineSchedule and Scheduler to run background agent tasks on a cron schedule with full tracing, cost-governor, and guardrail support.
Scheduled tasks run without an HTTP request, but under the same tracing, cost-governor, and guardrail machinery as request-driven routes — background work isn't a governance blind spot.
## Defining a scheduled task
Create a `schedule.ts` file anywhere under a background-tasks directory (conventionally `server/schedule/`):
```ts server/schedule/nightly-report/schedule.ts theme={null}
import { defineSchedule } from 'anvil-sdk/schedule';
export default defineSchedule({
name: 'nightly-report',
cron: '0 6 * * *', // 5-field cron: minute hour day-of-month month day-of-week
run: async (ctx) => {
// ctx.now — the Date this run fired
// ctx.trace — the trace span opened for this run, if a tracer is configured
await sendNightlyReport();
},
});
```
`defineSchedule` is an identity function — it exists purely for type inference and discovery, the same pattern as `defineAgent`.
Unique task name. Defaults to the folder path if omitted and the task is loaded via `loadBackgroundTasks`.
A 5-field cron expression (`minute hour day-of-month month day-of-week`). Supports `*`, comma lists (`1,15`), ranges (`9-17`), and step values (`*/15`).
The task body. Receives a `TaskContext` with `now`, `payload` (unused for scheduled tasks), and `trace`.
## Running the scheduler
```ts theme={null}
import { Scheduler, loadBackgroundTasks } from 'anvil-sdk/schedule';
import { Tracer, SqliteTraceStore } from 'anvil-sdk/trace';
const tracer = new Tracer(await SqliteTraceStore.open('.anvil/traces.db'));
const { schedules } = await loadBackgroundTasks('server/schedule');
const scheduler = new Scheduler({ tracer });
schedules.forEach((task) => scheduler.add(task));
scheduler.start(); // polls every minute; each due task runs once, wrapped in a trace
```
`loadBackgroundTasks(dir)` walks the directory tree, imports every `schedule.ts` file's default export, and derives each task's name from its folder path if `name` wasn't set explicitly.
### `Scheduler` API
Register a task. Chainable.
Run every task due at `now`. Idempotent per calendar minute — calling `tick` twice within the same minute only fires each task once. Returns the names of the tasks that ran.
Poll on an interval, default `60_000`ms. Returns a stop function that clears the interval.
Each task run is wrapped in a trace span named `schedule `, tagged with the task's cron expression. A task that throws is caught, the span closes with status `error`, and the failure is passed to `onError` (default: `console.error`) — one failing task never stops the scheduler or affects other tasks.
`Scheduler` runs in-process on a `setInterval` — it fires as long as your server process is alive. For multi-instance deployments, either run the scheduler on a single dedicated instance, or add your own leader-election/locking around `tick()` so the same task doesn't fire once per instance.
## Testing a schedule without waiting
Call `tick()` directly with a specific `Date` to simulate a firing time in tests, instead of waiting for the real clock:
```ts theme={null}
const scheduler = new Scheduler();
scheduler.add(nightlyReportTask);
const ran = await scheduler.tick(new Date('2026-01-01T06:00:00'));
// ['nightly-report']
```
## See also
* [Triggered agents](/background/triggered-agents) — the event-driven counterpart, fired by webhooks and queue messages instead of a clock
* [Observability](/observability/tracing) — every scheduled run appears in the same trace dashboard as request-driven agents
* [Durability & safety](/safety/durable-execution) — checkpoint long-running scheduled agent loops so a crash mid-run resumes cleanly
Scheduled tasks run without an HTTP request, but under the same tracing, cost-governor, and guardrail machinery as request-driven routes — background work isn't a governance blind spot.
## Defining a scheduled task
Create a `schedule.ts` file anywhere under a background-tasks directory (conventionally `server/schedule/`):
```ts server/schedule/nightly-report/schedule.ts theme={null}
import { defineSchedule } from 'anvil-sdk/schedule';
export default defineSchedule({
name: 'nightly-report',
cron: '0 6 * * *', // 5-field cron: minute hour day-of-month month day-of-week
run: async (ctx) => {
// ctx.now — the Date this run fired
// ctx.trace — the trace span opened for this run, if a tracer is configured
await sendNightlyReport();
},
});
```
`defineSchedule` is an identity function — it exists purely for type inference and discovery, the same pattern as `defineAgent`.
Unique task name. Defaults to the folder path if omitted and the task is loaded via `loadBackgroundTasks`.
A 5-field cron expression (`minute hour day-of-month month day-of-week`). Supports `*`, comma lists (`1,15`), ranges (`9-17`), and step values (`*/15`).
The task body. Receives a `TaskContext` with `now`, `payload` (unused for scheduled tasks), and `trace`.
## Running the scheduler
```ts theme={null}
import { Scheduler, loadBackgroundTasks } from 'anvil-sdk/schedule';
import { Tracer, SqliteTraceStore } from 'anvil-sdk/trace';
const tracer = new Tracer(await SqliteTraceStore.open('.anvil/traces.db'));
const { schedules } = await loadBackgroundTasks('server/schedule');
const scheduler = new Scheduler({ tracer });
schedules.forEach((task) => scheduler.add(task));
scheduler.start(); // polls every minute; each due task runs once, wrapped in a trace
```
`loadBackgroundTasks(dir)` walks the directory tree, imports every `schedule.ts` file's default export, and derives each task's name from its folder path if `name` wasn't set explicitly.
### `Scheduler` API
Register a task. Chainable.
Run every task due at `now`. Idempotent per calendar minute — calling `tick` twice within the same minute only fires each task once. Returns the names of the tasks that ran.
Poll on an interval, default `60_000`ms. Returns a stop function that clears the interval.
Each task run is wrapped in a trace span named `schedule `, tagged with the task's cron expression. A task that throws is caught, the span closes with status `error`, and the failure is passed to `onError` (default: `console.error`) — one failing task never stops the scheduler or affects other tasks.
`Scheduler` runs in-process on a `setInterval` — it fires as long as your server process is alive. For multi-instance deployments, either run the scheduler on a single dedicated instance, or add your own leader-election/locking around `tick()` so the same task doesn't fire once per instance.
## Testing a schedule without waiting
Call `tick()` directly with a specific `Date` to simulate a firing time in tests, instead of waiting for the real clock:
```ts theme={null}
const scheduler = new Scheduler();
scheduler.add(nightlyReportTask);
const ran = await scheduler.tick(new Date('2026-01-01T06:00:00'));
// ['nightly-report']
```
## See also
* [Triggered agents](/background/triggered-agents) — the event-driven counterpart, fired by webhooks and queue messages instead of a clock
* [Observability](/observability/tracing) — every scheduled run appears in the same trace dashboard as request-driven agents
* [Durability & safety](/safety/durable-execution) — checkpoint long-running scheduled agent loops so a crash mid-run resumes cleanly
# Triggered Agents: Fire Tasks on Webhooks and Events
Source: https://anvil.thatdevguy.in/background/triggered-agents
Use defineTrigger and TriggerRegistry to run background agent tasks in response to webhooks, queue messages, and other external events.
Triggered tasks are the event-driven counterpart to [scheduled agents](/background/scheduled-agents) — instead of firing on a cron clock, they fire when you call `fire(name, payload)`, typically from inside a webhook handler or queue consumer. Both run under the same tracing as request-driven agents.
## Defining a trigger
Create a `trigger.ts` file anywhere under a background-tasks directory (conventionally `server/triggers/`):
```ts server/triggers/order-created/trigger.ts theme={null}
import { defineTrigger } from 'anvil-sdk/schedule';
export default defineTrigger({
name: 'order.created',
run: async (ctx) => {
const order = ctx.payload as { id: string; total: number };
await notifyFulfillment(order);
},
});
```
Unique trigger name — this is the identifier you pass to `fire()`.
The task body. Receives a `TaskContext` with `now` (when `fire` was called), `payload` (whatever you passed to `fire`), and `trace`.
## Registering and firing triggers
```ts theme={null}
import { TriggerRegistry, loadBackgroundTasks } from 'anvil-sdk/schedule';
import { Tracer, SqliteTraceStore } from 'anvil-sdk/trace';
const tracer = new Tracer(await SqliteTraceStore.open('.anvil/traces.db'));
const { triggers } = await loadBackgroundTasks('server/triggers');
const registry = new TriggerRegistry({ tracer });
triggers.forEach((task) => registry.register(task));
```
Fire a trigger from any webhook handler or queue consumer:
```ts theme={null}
// server/routes/webhooks/orders/post.ts
import type { Context } from 'anvil-sdk';
import { registry } from '../../../triggers';
export default async function handler(ctx: Context) {
const payload = await ctx.body();
await registry.fire('order.created', payload);
return { received: true };
}
```
### `TriggerRegistry` API
Register a trigger task. Chainable.
Check whether a trigger with this name is registered.
Run the named trigger's `run` function, passing `payload` through `ctx.payload`. Returns whatever `run` returns. Throws if no trigger is registered under `name`.
Each `fire()` call opens a trace span named `trigger `. If `run` throws, the span closes with status `error` and the error propagates back to the caller — unlike scheduled tasks, a triggered task's failure is not swallowed, since the caller (your webhook handler) usually needs to know it failed to return the right HTTP status.
`fire()` runs the trigger synchronously, in the same request that called it. For triggers backed by slow or unreliable side effects, queue the actual work inside `run` (or hand off to a job queue) rather than blocking the webhook response on it.
## See also
* [Scheduled agents](/background/scheduled-agents) — the cron-driven counterpart
* [Multi-agent orchestration](/agents/orchestration) — call a registered `AgentRegistry` agent from inside a trigger's `run` function
* [Observability](/observability/tracing) — triggered runs appear in the same trace dashboard as request-driven agents
Triggered tasks are the event-driven counterpart to [scheduled agents](/background/scheduled-agents) — instead of firing on a cron clock, they fire when you call `fire(name, payload)`, typically from inside a webhook handler or queue consumer. Both run under the same tracing as request-driven agents.
## Defining a trigger
Create a `trigger.ts` file anywhere under a background-tasks directory (conventionally `server/triggers/`):
```ts server/triggers/order-created/trigger.ts theme={null}
import { defineTrigger } from 'anvil-sdk/schedule';
export default defineTrigger({
name: 'order.created',
run: async (ctx) => {
const order = ctx.payload as { id: string; total: number };
await notifyFulfillment(order);
},
});
```
Unique trigger name — this is the identifier you pass to `fire()`.
The task body. Receives a `TaskContext` with `now` (when `fire` was called), `payload` (whatever you passed to `fire`), and `trace`.
## Registering and firing triggers
```ts theme={null}
import { TriggerRegistry, loadBackgroundTasks } from 'anvil-sdk/schedule';
import { Tracer, SqliteTraceStore } from 'anvil-sdk/trace';
const tracer = new Tracer(await SqliteTraceStore.open('.anvil/traces.db'));
const { triggers } = await loadBackgroundTasks('server/triggers');
const registry = new TriggerRegistry({ tracer });
triggers.forEach((task) => registry.register(task));
```
Fire a trigger from any webhook handler or queue consumer:
```ts theme={null}
// server/routes/webhooks/orders/post.ts
import type { Context } from 'anvil-sdk';
import { registry } from '../../../triggers';
export default async function handler(ctx: Context) {
const payload = await ctx.body();
await registry.fire('order.created', payload);
return { received: true };
}
```
### `TriggerRegistry` API
Register a trigger task. Chainable.
Check whether a trigger with this name is registered.
Run the named trigger's `run` function, passing `payload` through `ctx.payload`. Returns whatever `run` returns. Throws if no trigger is registered under `name`.
Each `fire()` call opens a trace span named `trigger `. If `run` throws, the span closes with status `error` and the error propagates back to the caller — unlike scheduled tasks, a triggered task's failure is not swallowed, since the caller (your webhook handler) usually needs to know it failed to return the right HTTP status.
`fire()` runs the trigger synchronously, in the same request that called it. For triggers backed by slow or unreliable side effects, queue the actual work inside `run` (or hand off to a job queue) rather than blocking the webhook response on it.
## See also
* [Scheduled agents](/background/scheduled-agents) — the cron-driven counterpart
* [Multi-agent orchestration](/agents/orchestration) — call a registered `AgentRegistry` agent from inside a trigger's `run` function
* [Observability](/observability/tracing) — triggered runs appear in the same trace dashboard as request-driven agents
# anvil build: Generate the Manifest and Production Bundle
Source: https://anvil.thatdevguy.in/cli/build
Scan routes, validate schemas, emit .gen/routes.ts, and bundle a self-contained dist/server.mjs production artifact ready to deploy.
Run `anvil build` to compile your Anvil project for production. The command does two things in sequence: it runs the compiler to scan your routes and emit a typed manifest, then it bundles a self-contained Node.js server. The output is a single ESM file you can run with `node` or hand off to `anvil start`.
## Usage
```bash theme={null}
anvil build [options]
```
## Flags
Directory Anvil scans for route files. Must match the layout expected by the manifest loader.
Output directory for generated source files. Anvil writes `routes.ts` and an intermediate `server.ts` entry point here before bundling.
Path for the final output bundle. The `.mjs` extension signals that the file is an ES module.
Stop after emitting `.gen/routes.ts`. Skip generating `server.ts` and skip the bundling step entirely. Useful for CI validation and type-checking without producing a full build artifact.
## What the build produces
Anvil walks your routes directory, validates each module's param names and schemas, and writes a fully-typed `routes.ts` to the generated sources directory. The console prints how many routes were discovered:
```text theme={null}
[anvil] manifest: 6 route(s) → .gen/routes.ts
```
Anvil writes a minimal `.gen/server.ts` that imports your manifest, calls `createApp`, and starts the HTTP server. This file is the bundle entry point — you don't need to write or maintain it yourself:
```typescript theme={null}
// Generated by `anvil build`. Do not edit.
import { createApp, serve } from 'anvil-sdk';
import { manifest } from './routes.ts';
process.env.NODE_ENV ??= 'production';
const app = createApp(manifest);
const server = await serve(app, { port: Number(process.env.PORT) || 3000 });
console.log(`[anvil] listening on http://localhost:${server.port}`);
```
Anvil compiles and bundles `server.ts` targeting Node.js 20, ESM format, with source maps and all `node_modules` marked as external. The result lands at `dist/server.mjs`:
```text theme={null}
[anvil] bundle → dist/server.mjs
```
## Example: default build
```bash theme={null}
anvil build
```
```text theme={null}
[anvil] manifest: 4 route(s) → .gen/routes.ts
[anvil] bundle → dist/server.mjs
```
## Example: custom paths
```bash theme={null}
anvil build --routes src/api --gen-dir .generated --out build/app.mjs
```
## Example: manifest-only (CI validation)
Use `--manifest-only` to validate route structure and schema serializability in a CI pipeline without spending time on the full bundle:
```bash theme={null}
anvil build --manifest-only
```
```text theme={null}
[anvil] manifest: 4 route(s) → .gen/routes.ts
```
`--manifest-only` is ideal for a lightweight CI validation step. Run it alongside `anvil lint` to catch both structural errors and schema issues before deploying MCP tools.
The `.gen/` directory contains generated files. Add it to `.gitignore` and treat its contents as build artifacts — never edit them by hand.
# anvil dev: Start the Live-Reload Development Server
Source: https://anvil.thatdevguy.in/cli/dev
Run a live-reload dev server that watches your routes directory and hot-reloads TypeScript route files without a separate compilation step.
Run `anvil dev` to start a local development server that watches your routes directory and reloads route handlers the moment you save a file. Route modules are loaded on-the-fly so you write plain TypeScript — no compilation step required before your changes are live.
## Usage
```bash theme={null}
anvil dev [options]
```
## Flags
Path to the directory Anvil scans for route files. Passed to the compiler's manifest loader. Change this if your project uses a non-standard layout.
TCP port the HTTP server listens on. Useful when you need to run multiple Anvil apps side by side, or when port `3000` is already taken.
Hostname to bind. Omit to bind to all interfaces (`0.0.0.0`). Set to `127.0.0.1` to restrict the server to localhost only.
## How it works
When you run `anvil dev`, the following happens:
Anvil reads your routes directory, builds an in-memory manifest, and prints each discovered route with its HTTP method and URL pattern.
An HTTP server starts listening on the configured port. The server is backed by a live reference to the current app instance.
Anvil watches the routes directory recursively. Any `add`, `change`, or `unlink` event triggers a debounced reload (100 ms debounce to coalesce rapid saves).
On each reload, Anvil clears the module cache for edited files and rebuilds the manifest. The HTTP server is **not** restarted — it begins serving the new route table immediately on the next request.
## Example output
```text theme={null}
GET /api/health
POST /api/chat
GET /api/users/[id]
[anvil] dev server listening on http://localhost:3000
```
After you edit a route file:
```text theme={null}
[anvil] routes reloaded
```
If a reload fails (for example, a TypeScript syntax error), the error is printed and the server continues to serve the **last valid** route table:
```text theme={null}
[anvil] reload failed: Cannot find module './missing-dep.js'
```
## Example: custom port and routes directory
```bash theme={null}
anvil dev --routes src/routes --port 4000 --hostname 127.0.0.1
```
Each reload clears the module cache for the routes directory, ensuring edited files are always re-read from disk. Singletons shared across route files — such as a shared trace store — still resolve to one instance within a single build pass because the module cache is shared within that pass.
Add `"dev": "anvil dev"` to your `package.json` scripts so you can start the server with `npm run dev` without needing to prefix every invocation with `npx`.
# anvil eval: Run Deterministic and Judge-Based Eval Suites
Source: https://anvil.thatdevguy.in/cli/eval
Run a defineEvalSuite test file against an agent route with deterministic and LLM-as-judge assertions; exits non-zero on failure for CI use.
Run `anvil eval ` to execute a structured test suite against an agent route. Each suite is a TypeScript file that default-exports a suite object created with `defineEvalSuite` from `anvil/eval`. Anvil loads the file, runs every test case through your agent, evaluates all assertions, and prints a pass/fail report. Because `anvil eval` exits with code `1` when any case fails, you can plug it directly into CI pipelines.
## Usage
```bash theme={null}
anvil eval
```
`` is the path to a suite file that default-exports the result of `defineEvalSuite(...)`. Relative and absolute paths are both accepted.
## Assertions
`anvil/eval` ships two families of assertions.
### Deterministic assertions
These assertions run synchronously against the agent's recorded run and never make additional model calls:
| Function | What it checks |
| --------------------------- | ------------------------------------------------------------- |
| `outputContains(substring)` | Final text output includes the given substring |
| `outputMatches(re)` | Final text output matches the given `RegExp` |
| `outputJson(schema)` | Final text is valid JSON that satisfies a Zod schema |
| `toolCalled(name)` | At least one tool call with the given name appears in the run |
| `maxCost(usd)` | `result.totalCostUsd` does not exceed the given dollar amount |
| `maxIterations(n)` | The agent completed within `n` agentic iterations |
### LLM-as-judge assertion
`judge({ client, rubric, model? })` scores the agent's final output against a free-text rubric using a model. The judge prompt is strict: the model must respond with `{"pass": boolean, "reason": string}`. Judge calls run through the same `LlmClient` instance you provide — so they are **traced and cost-tracked** exactly like any other model call, and their spend is visible in the dashboard.
## `defineEvalSuite`
Import `defineEvalSuite` from `anvil/eval` and use it as the default export of your suite file. It is a typed identity function — its only job is to give you autocomplete and type safety:
```typescript theme={null}
import { defineEvalSuite } from 'anvil-sdk/eval';
export default defineEvalSuite({ /* EvalSuite */ });
```
A complete `EvalSuite` has three fields:
| Field | Type | Description |
| -------- | ------------ | ------------------------------------------------------------------- |
| `name` | `string` | Human-readable suite name printed in the report header |
| `runner` | `EvalRunner` | Async function that turns input messages into an `EvalRunContext` |
| `cases` | `EvalCase[]` | Array of test cases, each with a `name`, `input`, and `assert` list |
Use `agentRunner` from `anvil/eval` to build a runner from your agent configuration without wiring up the streaming loop yourself.
## Minimal eval suite example
```typescript theme={null}
// evals/chat.eval.ts
import { agentRunner, defineEvalSuite, judge, outputContains, toolCalled, maxCost } from 'anvil-sdk/eval';
import { LlmClient } from 'anvil-sdk/llm';
import { myTools } from '../server/tools/index.js';
const client = new LlmClient({ /* your driver config */ });
export default defineEvalSuite({
name: 'Chat agent — basic scenarios',
runner: agentRunner({
client,
tools: myTools,
model: 'gpt-4o-mini',
system: 'You are a helpful assistant.',
}),
cases: [
{
name: 'greets the user',
input: 'Hello!',
assert: [
outputContains('hello'),
maxCost(0.001),
],
},
{
name: 'calls the weather tool',
input: "What's the weather in Paris?",
assert: [
toolCalled('get_weather'),
outputContains('Paris'),
],
},
{
name: 'answer quality — judge',
input: 'Explain recursion in one sentence.',
assert: [
judge({
client,
rubric: 'The explanation must mention that a function calls itself and must be a single sentence.',
}),
],
},
],
});
```
Run it:
```bash theme={null}
anvil eval evals/chat.eval.ts
```
## Example report output
```text theme={null}
Chat agent — basic scenarios
PASS greets the user
PASS calls the weather tool
FAIL answer quality — judge
✗ judge: The explanation must mention... — The output was two sentences.
[anvil] eval: 2 passed, 1 failed
```
The process exits with code `1` because one case failed.
A fully passing run looks like:
```text theme={null}
Chat agent — basic scenarios
PASS greets the user
PASS calls the weather tool
PASS answer quality — judge
[anvil] eval: 3 passed, 0 failed
```
Exit code is `0`.
## CI integration
```yaml theme={null}
# .github/workflows/ci.yml
- name: Run agent evals
run: npx anvil eval evals/chat.eval.ts
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
```
Judge model calls are routed through the same `LlmClient` you provide to `judge(...)`. This means they are traced, and their token usage and cost appear in the Anvil dashboard alongside the agent calls they evaluate.
The suite file **must** default-export the return value of `defineEvalSuite(...)`. If the default export is missing or does not have a `cases` array, `anvil eval` prints an error and exits with code `1` without running any cases.
# anvil init: Scaffold a New Anvil JS Project
Source: https://anvil.thatdevguy.in/cli/init
Scaffold package.json, tsconfig.json, and starter routes for a new Anvil project — pick from a basic API, MCP server, or agent route template.
Run `anvil init` to scaffold a new Anvil project, or add Anvil to an existing one. It writes `package.json` (creating or merging into an existing file), `tsconfig.json`, `.gitignore`, and a starter route or two from a template you choose.
## Usage
```bash theme={null}
anvil init [dir] [options]
```
`dir` is the target directory. Defaults to the current directory. It's created if it doesn't exist.
## Flags
Template to scaffold: `basic`, `mcp`, or `agent`. Skips the interactive prompt.
Skip the interactive prompt without specifying `--template` — defaults to the `basic` template. Useful for scripting and CI.
Without `--template` or `--yes`, and when stdin is a TTY, `anvil init` prompts you to pick a template interactively.
## Templates
A minimal REST API: one static route and one dynamic `[id]` route.
The same routes, with the dynamic route exposed as an MCP tool, plus a standalone tool in `server/tools/`.
A streaming agent route at `/chat` using `MockDriver`, so it runs immediately with no API key.
## What it writes
If one doesn't exist, creates a minimal one with `"type": "module"`. If one exists, it's merged, not overwritten: `type` is forced to `"module"` (Anvil is ESM-only), the `dev`/`build`/`start`/`lint` scripts are added (your existing scripts are preserved and take precedence on name conflicts), and `anvil-sdk`/`zod` are added to `dependencies` — **without** clobbering a version you've already pinned for either.
Written only if missing. An existing file of either is left untouched.
Each file the chosen template defines is written only if it doesn't already exist — `anvil init` never overwrites a file you've edited. Re-running `anvil init` in the same directory is safe; it reports what it skipped.
## Example: non-interactive, for scripting
```bash theme={null}
mkdir my-api && cd my-api
npx anvil init --template mcp
npm install
npx anvil mcp
```
## Example: adding Anvil to an existing project
Run `anvil init` in a directory that already has a `package.json` — it merges in the Anvil scripts and dependency instead of creating a new project:
```bash theme={null}
cd my-existing-app
npx anvil init -y
```
`anvil init` only writes files — it does not run `npm install` for you, and it doesn't assume a package manager. Run `npm install` (or your equivalent) yourself afterward.
Prefer the interactive prompt when you're not sure which template fits — it shows a one-line description of each before you choose.
# anvil lint: Validate Routes, Params, and Tool Schemas
Source: https://anvil.thatdevguy.in/cli/lint
Validate routes for param mismatches and lossy MCP schemas, printing structured diagnostics and exiting non-zero on errors or strict-mode warnings.
Run `anvil lint` to validate your routes directory without starting a server or emitting any build artifacts. Lint catches two classes of problems: route parameter inconsistencies (where the folder name declares a param that the schema does not cover, or vice versa) and schema serializability issues (where a Zod schema used on an MCP-exposed route cannot be losslessly converted to JSON Schema). Catching these problems at lint time prevents silent failures at runtime.
## Usage
```bash theme={null}
anvil lint [options]
```
## Flags
Directory Anvil scans for route files. Must match the layout used by your application.
Treat warnings as errors. When set, `anvil lint` exits with code `1` if there are any warnings, even if there are no hard errors. Use this in CI pipelines where warnings must be resolved before merging.
## Rule categories
### `param-mismatch` — errors
Anvil derives expected URL parameters from your folder names (e.g. a folder named `[id]` declares an `id` param). Lint raises an error if:
* **`params-missing`** — `paramsSchema` exists but is missing one or more params declared by the folder path.
* **`params-extra`** — `paramsSchema` declares a key that has no corresponding folder segment.
These are always `error`-level and cause a non-zero exit regardless of `--strict`.
### `mcp-schema-not-serializable` — errors
For MCP-exposed routes, every schema export (`paramsSchema`, `querySchema`, `bodySchema`, `outputSchema`) must round-trip losslessly to JSON Schema. If a Zod type cannot be converted — for example, a `z.transform()` or a custom refinement — lint reports an `error`-level diagnostic with rule `mcp-schema-not-serializable`.
### `mcp-missing-description` — warnings
An MCP-exposed route with no `meta.mcp.description` string gets a `warning`-level diagnostic. The tool will still register, but agents typically rely on descriptions to decide when to call a tool. Pass `--strict` to promote this to an error in CI.
## Example output: errors present
```text theme={null}
error POST /api/orders/[id] paramsSchema is missing route parameter(s): "id". The folder declares [id].
server/routes/api/orders/[id]/POST.ts [params-missing]
warn GET /api/chat MCP-exposed route has no meta.mcp.description. Tools without descriptions are hard for agents to use.
server/routes/api/chat/GET.ts [mcp-missing-description]
[anvil] lint: 1 error(s), 1 warning(s)
```
The process exits with code `1` because there is at least one error.
## Example output: clean run
```text theme={null}
[anvil] lint: no issues found
```
The process exits with code `0`.
## Example output: warnings with `--strict`
```bash theme={null}
anvil lint --strict
```
```text theme={null}
warn GET /api/chat MCP-exposed route has no meta.mcp.description. Tools without descriptions are hard for agents to use.
server/routes/api/chat/GET.ts [mcp-missing-description]
[anvil] lint: 0 error(s), 1 warning(s)
```
Exits with code `1` because `--strict` is set and there is one warning.
## Recommended CI step
Add `anvil lint` as an early gate in your CI pipeline, especially before deploying MCP tools:
```yaml theme={null}
# .github/workflows/ci.yml
- name: Lint routes
run: npx anvil lint --strict
```
`anvil lint` does not start a server, write any files, or run your handler functions. It imports each route module purely to inspect its exported schemas and metadata, then exits.
Combine `anvil lint --strict` with `anvil build --manifest-only` as a fast, artifact-free CI validation pass. Both commands finish in seconds and together cover the full range of compile-time issues.
# anvil mcp: Serve Your Routes as an MCP Tool Server
Source: https://anvil.thatdevguy.in/cli/mcp
Serve MCP-annotated routes and server/tools/ as an MCP tool server over Streamable HTTP or stdio for AI agents and clients like Claude Desktop.
Run `anvil mcp` to serve your MCP-exposed routes and any standalone tool files in `server/tools/` as a Model Context Protocol (MCP) tool server. AI agents, orchestrators, and local clients such as Claude Desktop can then discover and call your tools without any additional glue code.
Anvil supports two transport modes: Streamable HTTP (the default) for networked agents and hosted deployments, and `--stdio` for local clients that communicate over standard input/output.
## Usage
```bash theme={null}
anvil mcp [options]
```
## Flags
Directory Anvil scans for route files. Only routes that carry `meta.mcp.expose = true` are registered as MCP tools.
Directory of standalone tool files to include alongside route-derived tools. If this directory does not exist, it is silently skipped — no error is thrown.
Serve the MCP server over stdio instead of HTTP. Use this mode when connecting a local client such as Claude Desktop directly to the process. In stdio mode, the protocol channel is `stdout`; all Anvil diagnostics are written to `stderr` so they do not corrupt the protocol stream.
HTTP port for Streamable HTTP mode. Ignored when `--stdio` is set.
URL path at which the Streamable HTTP endpoint is mounted. Ignored when `--stdio` is set.
## HTTP mode (default)
Start the MCP server on its default port:
```bash theme={null}
anvil mcp
```
```text theme={null}
[anvil] mcp (Streamable HTTP) on http://localhost:3100/mcp
[anvil] 3 tool(s): get_user, create_order, search_products
```
Connect any MCP-compatible HTTP client to `http://localhost:3100/mcp`.
### Custom port and endpoint
```bash theme={null}
anvil mcp --port 4200 --endpoint /tools/mcp
```
```text theme={null}
[anvil] mcp (Streamable HTTP) on http://localhost:4200/tools/mcp
[anvil] 2 tool(s): summarize, classify
```
## stdio mode
Start the MCP server over stdio for use with Claude Desktop or another local client:
```bash theme={null}
anvil mcp --stdio
```
```text theme={null}
[anvil] mcp stdio: 3 tool(s) — get_user, create_order, search_products
```
The process then communicates exclusively over stdin/stdout using the MCP wire protocol. Add it to your Claude Desktop config like this:
```json theme={null}
{
"mcpServers": {
"my-anvil-app": {
"command": "npx",
"args": ["anvil", "mcp", "--stdio"],
"cwd": "/path/to/your/project"
}
}
}
```
## Exposing a route as an MCP tool
To include a route as an MCP tool, add a `meta` export to the route file with `mcp.expose` set to `true`:
```typescript theme={null}
// server/routes/api/summarize/POST.ts
export const meta = {
mcp: {
expose: true,
description: "Summarize a block of text to a given maximum length.",
},
};
export const bodySchema = z.object({
text: z.string(),
maxWords: z.number().int().optional(),
});
export default async function handler(req) { /* ... */ }
```
If `server/tools/` does not exist, `anvil mcp` silently skips it. You do not need to create the directory or pass `--tools` with a different path unless you have standalone tool files to include.
In stdio mode, **do not write to stdout** from inside your route handlers or tools. Any non-protocol bytes on stdout will corrupt the MCP stream. Use `process.stderr.write(...)` or a logging library configured to write to stderr.
Run `anvil lint` before `anvil mcp` in CI. Lint catches `schema-lossy` errors — schemas that cannot round-trip to JSON Schema — which would cause tools to be unregistered or malformed at runtime.
# Anvil CLI Overview: Commands for Building Anvil Apps
Source: https://anvil.thatdevguy.in/cli/overview
The anvil CLI is the primary interface for developing, building, and running Anvil apps — covering dev, build, MCP serving, lint, eval, and replay.
The `anvil` CLI is the primary entry point for all Anvil development workflows. Once you install the package, the `anvil` command is available in your project's `node_modules/.bin` and, when using a package manager with `bin` hoisting, on your `PATH`. Every workflow — scaffolding, local development, production builds, MCP serving, linting, evaluations, and trace replay — runs through this single binary.
## Commands at a glance
| Command | What it does |
| -------------- | ----------------------------------------------------------------------------------- |
| `anvil init` | Scaffold package.json, tsconfig, and starter routes for a new (or existing) project |
| `anvil dev` | Start a live-reload development server with TypeScript route loading |
| `anvil build` | Scan routes, emit `.gen/routes.ts`, and bundle `dist/server.mjs` |
| `anvil start` | Run the production bundle produced by `anvil build` |
| `anvil mcp` | Serve MCP-exposed routes and `server/tools/` as an MCP tool server |
| `anvil lint` | Validate route params and schemas; exits non-zero on errors |
| `anvil eval` | Run a deterministic + LLM-judge eval suite against an agent route |
| `anvil replay` | Re-run a captured agent trace with mocked model responses |
## Explore each command
Scaffold a new project — pick a basic API, MCP server, or agent route template.
Start the live-reload development server with file watching and instant route hot-reloading.
Generate the route manifest and bundle a self-contained production server.
Run the production bundle created by `anvil build`, with optional port override.
Expose your routes as an MCP tool server over HTTP or stdio for AI agents and Claude Desktop.
Catch param mismatches and lossy MCP schemas at compile time, before a broken deploy.
Run deterministic and LLM-as-judge assertions against agent routes in a reproducible suite.
Replay a captured trace locally with no live model calls to debug production failures.
## Installation
The CLI is included in the main `anvil` package — no separate install is needed.
```bash theme={null}
npm install anvil-sdk
```
Run any command via your package manager's script runner:
```bash theme={null}
npx anvil dev
# or add to package.json scripts:
# "dev": "anvil dev"
```
Anvil requires **Node.js ≥ 20**. All CLI commands are ESM-native and use the Node.js `--input-type=module` environment automatically.
# anvil replay — Replay a Trace Without Live Calls
Source: https://anvil.thatdevguy.in/cli/replay
Re-run a captured agent trace with mocked model responses and recorded tool outputs — no live model spend, no re-fired side effects.
Run `anvil replay ` to re-execute a previously captured agent trace entirely from recorded data. Anvil reads the trace from a SQLite trace store, reconstructs the agent's original inputs, and drives the agent loop using a `ReplayDriver` that feeds back the recorded model responses in order. Tool functions are also replaced with stubs that return their recorded outputs, so no real side effects — database writes, external API calls, emails — are re-fired.
Use replay to reproduce a production failure on your laptop, step through an edge case, or verify that a code fix changes the agent's behaviour as expected.
## Usage
```bash theme={null}
anvil replay [options]
```
`` is the ID of the trace to replay. You can find trace IDs in the Anvil dashboard, your trace store, or in logs emitted by a traced agent route.
## Flags
**Required.** The identifier of the trace to replay. Anvil looks this up in the SQLite store specified by `--store`.
Path to the SQLite trace database written by `SqliteTraceStore`. If the file does not exist or cannot be opened, `anvil replay` prints an error and exits with code `1`.
## How replay works
Anvil opens the SQLite store and fetches the full trace by ID. The trace includes every span: the top-level agent span, each model call span (with the recorded response text and tool calls), and each tool span (with the recorded output).
The `ReplayDriver` (from `anvil/agent`) is initialised with the recorded model responses in chronological order. Each call to `generate()` or `stream()` pops the next response off the queue instead of making a live API request. Cost and token usage are reported as zero for replayed responses.
Each tool that appeared in the original trace is replaced with a stub that returns its recorded output in call order. The original tool implementation is never invoked, so no writes, notifications, or API calls happen.
Anvil runs the full agent loop — the same loop used in production — but entirely against the mocked driver and stub tools. Events are streamed and printed as they occur.
## Example
```bash theme={null}
anvil replay abc-123
```
```text theme={null}
[anvil] replaying chat-agent (abc-123) — mocked model, no live calls
— iteration 1
tool_call: get_weather({"city":"Paris"})
tool_result: get_weather → {"temp":18,"condition":"cloudy"}
assistant: The weather in Paris is 18 °C and cloudy.
[anvil] replay complete (1 iteration(s))
```
## Example: custom trace store
```bash theme={null}
anvil replay abc-123 --store data/production-traces.db
```
## `ReplayDriver` from `anvil/agent`
If you need to embed replay into your own scripts or tests, import `ReplayDriver` directly:
```typescript theme={null}
import { ReplayDriver, buildReplayFromTrace, replayToResult } from 'anvil-sdk/agent';
import { SqliteTraceStore } from 'anvil-sdk/trace';
const store = await SqliteTraceStore.open('.anvil/traces.db');
const trace = await store.getTrace('abc-123');
// High-level: replay to completion and get all events
const { events, result } = await replayToResult(trace);
// Low-level: build the replayable config and drive it yourself
const replayable = buildReplayFromTrace(trace);
// replayable.client uses a ReplayDriver under the hood
```
`buildReplayFromTrace` reconstructs the original agent inputs (messages, system prompt, model identifier) and wires up a `ReplayDriver` and stub tools. You can then pass the result to `streamAgent` for fine-grained event inspection.
## Error: trace not found
```text theme={null}
[anvil] no trace "abc-123" in .anvil/traces.db
```
This happens when the trace ID does not exist in the store, or when you are pointing at the wrong store file. Double-check the trace ID from your dashboard or logs, and confirm the `--store` path matches the file written by your application's tracer.
Side-effect tools are **never** re-invoked during replay. Stub implementations return the recorded output directly. If a tool's recorded output queue is exhausted (the tool was called more times than recorded), the stub returns a sentinel string rather than throwing, so the replay completes.
Replay is most effective when your application uses `SqliteTraceStore` in production (or staging). Attach it to your agent routes during development and the trace DB will accumulate cases you can replay at any time without re-running the real workload.
# anvil start: Run Your Built Production Server Bundle
Source: https://anvil.thatdevguy.in/cli/start
Launch the production server bundle built by anvil build, with an optional port override via flag or the PORT environment variable.
Run `anvil start` to execute the production bundle generated by `anvil build`. The command spawns the bundle as a child Node.js process, inheriting stdio so logs appear in your terminal or platform's log drain. This is the command you call in `Dockerfile` `CMD` entries, process manager configs, and deploy scripts.
## Usage
```bash theme={null}
anvil start [options]
```
## Flags
Path to the bundle produced by `anvil build`. If the file does not exist, `anvil start` prints an error and exits with code `1` — it will not attempt to build it automatically.
Override the HTTP port. When provided, this value is forwarded to the child process as the `PORT` environment variable, taking precedence over any `PORT` already set in the environment. When omitted, the bundle falls back to `PORT` from the environment, then to `3000`.
## Port resolution order
The port used by the running server follows this precedence, from highest to lowest:
1. `--port ` flag passed to `anvil start`
2. `PORT` environment variable already set in the shell
3. Default value of `3000` hardcoded in the generated bundle entry point
## Example: basic deploy script
A typical deploy pipeline builds the project then starts it:
```bash theme={null}
# Build the production artifact
anvil build
# Start the server
anvil start
```
## Example: custom port
```bash theme={null}
anvil start --port 8080
```
## Example: custom bundle path
```bash theme={null}
anvil start --entry build/app.mjs --port 8080
```
## Example: using PORT from the environment
```bash theme={null}
PORT=443 anvil start
```
## Example: Docker
```dockerfile theme={null}
FROM node:20-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
RUN npx anvil build
EXPOSE 3000
CMD ["npx", "anvil", "start"]
```
## Error: bundle not found
If you run `anvil start` before `anvil build`, you will see:
```text theme={null}
[anvil] /absolute/path/to/dist/server.mjs not found — run `anvil build` first
```
`anvil start` does **not** watch for file changes or restart on crashes. In production, wrap it with a process supervisor such as PM2, Docker restart policies, or your platform's native process management to get automatic restarts.
For zero-downtime deploys, build the new bundle alongside the running server, then send a signal to swap. Because `anvil start` spawns a standard Node.js child process, all normal graceful-shutdown patterns (`SIGTERM` handlers, drain logic) apply.
# Install Anvil JS: Setup, ESM Config, and Peer Deps
Source: https://anvil.thatdevguy.in/installation
Install Anvil JS via npm, yarn, or pnpm, configure your ESM project, add optional AI provider peer deps, and set up your package.json scripts.
Anvil is distributed as a single npm package, published as **`anvil-sdk`** — the `anvil` name on npm belongs to an unrelated package, so the install command uses `anvil-sdk` while every CLI command you run stays `anvil` (`anvil dev`, `anvil build`, and so on). This page covers everything needed to go from an empty directory to a correctly configured, running Anvil project.
## Requirements
* **Node.js ≥ 20** — Anvil relies on the native `Request`, `Response`, `Headers`, and `ReadableStream` APIs that ship with Node 20; there's no polyfill layer.
* **An ESM project** — Anvil is ESM-only. Your `package.json` must declare `"type": "module"`.
* **npm, yarn, or pnpm** — any of the three major package managers work; examples below cover all three.
## Install the package
```bash npm theme={null}
npm install anvil-sdk zod
```
```bash yarn theme={null}
yarn add anvil-sdk zod
```
```bash pnpm theme={null}
pnpm add anvil-sdk zod
```
`zod` is a direct dependency of your route and tool schemas — install it alongside Anvil. Anvil re-exports the pieces of Zod it needs internally, but you write your own `z.object(...)` calls, so it must be present in your own `dependencies`.
The package name is `anvil-sdk`, but every command you run is still `anvil` — `npx anvil dev`, `npx anvil build`, and so on. Only the string you pass to `npm install`/`yarn add`/`pnpm add` uses the `-js` suffix.
## Scaffold with `anvil init`
The fastest path to a working project is `anvil init` — it writes `package.json`, `tsconfig.json`, `.gitignore`, and a starter route or two, so you don't hand-assemble the project structure yourself:
```bash theme={null}
npx anvil init
```
Without any flags, and when stdin is a TTY, it prompts you to pick a starting point:
A minimal REST API — one static route (`GET /`) and one dynamic route (`GET /users/[id]`). The default template; good for plain HTTP APIs with no AI surface.
```bash theme={null}
npx anvil init --template basic
```
The same two routes, with the dynamic route also exposed as an MCP tool (`meta.mcp.expose`), plus a standalone tool file under `server/tools/` that has no backing HTTP route at all.
```bash theme={null}
npx anvil init --template mcp
```
A streaming agent route at `POST /chat`, wired to `MockDriver` so it runs immediately with no API key — swap in `AnthropicDriver`, `OpenAIDriver`, or `GeminiDriver` when you're ready for a real model.
```bash theme={null}
npx anvil init --template agent
```
Pass `-y` to accept the `basic` default without prompting — useful in CI or scripts:
```bash theme={null}
npx anvil init -y
```
`anvil init` never overwrites a file that already exists — running it in a directory that already has a `package.json` merges in the `dev`/`build`/`start`/`lint` scripts and the `anvil-sdk`/`zod` dependencies (without clobbering versions you've already pinned) instead of starting over. Re-running it is always safe; it reports what it wrote and what it skipped:
```text theme={null}
✎ update package.json
– skip server/routes/get.ts (already exists)
✔ create server/routes/users/[id]/get.ts
2 file(s) written, 1 skipped.
```
It only writes files — it doesn't run `npm install` for you and doesn't assume a package manager, so run that yourself afterward:
```bash theme={null}
npm install
npm run dev
```
Every flag, what gets merged versus created, and non-interactive usage for CI.
## Configure package.json by hand
If you'd rather not use `anvil init`, add `"type": "module"` yourself and wire up the scripts:
```json package.json theme={null}
{
"name": "my-api",
"version": "0.1.0",
"type": "module",
"scripts": {
"dev": "anvil dev",
"build": "anvil build",
"start": "anvil start",
"lint": "anvil lint"
},
"dependencies": {
"anvil-sdk": "latest",
"zod": "latest"
}
}
```
If you omit `"type": "module"`, Node treats your files as CommonJS and `import ... from 'anvil-sdk'` fails with a syntax error at the first `import`/`export` statement. ESM is required — there is no CommonJS build.
## Project structure
Route handlers live under `server/routes/`; everything else is up to you:
```text theme={null}
my-api/
├── server/
│ ├── routes/
│ │ ├── get.ts → GET /
│ │ └── users/
│ │ ├── get.ts → GET /users
│ │ └── [id]/
│ │ └── get.ts → GET /users/:id
│ └── tools/ → standalone MCP tools (no HTTP route)
├── package.json
├── tsconfig.json
└── .gitignore
```
A minimal `tsconfig.json` — the one `anvil init` writes:
```json tsconfig.json theme={null}
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"strict": true,
"skipLibCheck": true
}
}
```
`moduleResolution: "bundler"` matches how `anvil dev`/`anvil build` resolve subpath exports like `anvil-sdk/agent` and `anvil-sdk/llm` — using `"node16"` or `"nodenext"` instead still works, but `"bundler"` gives the cleanest editor experience for these deep imports.
## Optional peer dependencies
Core routing and middleware need nothing beyond `anvil-sdk` and `zod`. AI provider drivers and persistence adapters are **optional peer dependencies**, loaded lazily — install only what you actually use, and pure-REST projects stay lean.
Required to use `AnthropicDriver` from `anvil-sdk/llm` in an agent route or `LlmClient`.
```bash theme={null}
npm install @anthropic-ai/sdk
```
Required to use `OpenAIDriver` from `anvil-sdk/llm`.
```bash theme={null}
npm install openai
```
Required to use `GeminiDriver` from `anvil-sdk/llm`.
```bash theme={null}
npm install @google/generative-ai
```
Required for `SqliteStateStore`, `SqliteTraceStore`, or `SqliteVectorStore` — persists traces, checkpoints, memory, and vectors across restarts instead of losing them on process exit.
```bash theme={null}
npm install better-sqlite3
```
Using a driver or store whose peer dependency isn't installed throws a clear error at the call site naming the missing package and the in-memory alternative you can fall back to — it never fails silently.
## CLI commands reference
Once installed, the `anvil` binary is available in `node_modules/.bin/`. Run it through your package manager's script runner (`npm run dev`, or `npx anvil dev` directly):
| Command | What it does |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `anvil init` | Scaffolds `package.json`, `tsconfig.json`, `.gitignore`, and starter routes from a template (`basic`, `mcp`, or `agent`) — merges into an existing project instead of overwriting |
| `anvil dev` | Starts the dev server on port 3000 with watch mode; TypeScript routes load on the fly via `jiti`, no separate compile step |
| `anvil build` | Scans `server/routes/`, generates `.gen/routes.ts`, and bundles `dist/server.mjs` for production; fails hard on route conflicts |
| `anvil start` | Runs the production bundle. Requires `anvil build` first |
| `anvil lint` | Validates every route: `paramsSchema` keys against folder params, MCP-exposed schemas against lossless JSON Schema conversion. Add `--strict` to fail on warnings too |
| `anvil mcp` | Serves MCP-exposed routes and `server/tools/` over Streamable HTTP on port 3100. Pass `--stdio` for Claude Desktop and similar local clients |
| `anvil eval ` | Runs a `defineEvalSuite` file against an agent route — deterministic assertions plus LLM-as-judge — and exits non-zero on any failing case |
| `anvil replay ` | Re-runs a captured agent trace with mocked model responses and recorded tool outputs — no live model calls, useful for debugging a production failure locally |
Deeper reference for every command, including flags not covered above.
## Verify the installation
Create a minimal route and start the dev server to confirm everything is wired correctly:
```ts server/routes/get.ts theme={null}
export default function handler() {
return { status: 'ok' };
}
```
```bash theme={null}
npm run dev
```
```text theme={null}
[anvil] dev server listening on http://localhost:3000
```
```bash theme={null}
curl http://localhost:3000/
```
```json theme={null}
{ "status": "ok" }
```
If you see that JSON response, the installation is complete and the dev server is correctly loading your route files.
## Next steps
Build a validated dynamic route, expose it as an MCP tool, and add a streaming agent — one walkthrough.
Every template, flag, and file it writes — including how it merges into an existing project.
The full routing convention: verbs, groups, catch-alls, and the generated route manifest.
Expose routes and standalone tools as MCP tools over stdio or Streamable HTTP.
# Anvil JS: Node.js Backend Framework for AI Developers
Source: https://anvil.thatdevguy.in/introduction
Anvil is a Node.js backend framework with file-based routing, compile-time validation, and a native agentic layer: MCP, tools, tracing, and more.
Anvil is a Node.js backend framework designed for developers building AI-powered and agent-driven applications. It gives you the ergonomics of Express, the structural clarity of Next.js-style file-based routing, and a native agentic layer — MCP auto-exposure, a tool registry, agent orchestration, distributed tracing, durable checkpointing, and more — without requiring a second codebase, a separate config file, or a pile of duplicated schemas.
## The problem Anvil solves
Building a backend for an AI-first product today forces you to maintain parallel worlds: one codebase for your HTTP API, another set of schemas for your MCP tools, manual wiring for tracing and cost observability, and no standard way to resume an agent run after a crash. The result is duplicated logic, schema drift, and brittle glue code that breaks every time a model or SDK changes.
Anvil collapses all of that into a single framework:
* **No duplicate MCP schemas** — mark any route `mcp: { expose: true }` and it becomes an MCP tool automatically. The same handler, the same Zod schema, zero extra code.
* **No manual tracing wiring** — every request, model call, tool invocation, and token cost is captured in the built-in trace store and surfaced through the `/_anvil` dashboard.
* **No crash-resume glue** — durable checkpointing lets agent runs survive restarts; human-in-the-loop gates pause execution until approval arrives.
* **No framework-per-concern** — routing, middleware, CORS, static files, MCP, agent orchestration, evals, and RAG are all first-class citizens in the same package.
## Key features
Drop a `get.ts` in `server/routes/users/` and you have `GET /users`. No config, no decorators, no registration step.
Use `[param]` folders for path parameters and `[...param]` for catch-all segments. Validate them with Zod at compile time.
Place `_middleware.ts` next to your routes for folder-scoped middleware. Compose multiple middleware functions with the built-in `compose()` helper.
A single `Context` object replaces Express's `(req, res)` pair — typed params, cached body parsing, and first-class response helpers.
Any route annotated with `meta.mcp.expose` is instantly available as an MCP tool. Run `anvil mcp` to serve tools over Streamable HTTP or `--stdio` for Claude Desktop.
An `agent.ts` file is a streaming agent endpoint. It runs the full model↔tool loop, validates inputs, tracks cost, and aborts cleanly on client disconnect.
## Requirements
* **Node.js ≥ 20** (uses the native `Request`/`Response` Web API built into Node 20+)
* **ESM project** — your `package.json` must include `"type": "module"`
* npm, yarn, or pnpm
## CLI overview
Anvil ships with a CLI that handles the full development and deployment lifecycle:
| Command | Description |
| ------------------------ | ------------------------------------------------------------------------- |
| `anvil dev` | Dev server with watch mode — TypeScript routes are loaded on the fly |
| `anvil build` | Generates `.gen/routes.ts` and bundles `dist/server.mjs` |
| `anvil start` | Runs the production bundle |
| `anvil lint` | Validates routes: checks param schema keys, MCP JSON Schema compatibility |
| `anvil mcp` | Serves MCP-exposed routes and `server/tools/` over Streamable HTTP |
| `anvil eval ` | Runs an eval suite against an agent; exits non-zero on failure |
| `anvil replay ` | Re-runs a captured trace with mocked responses |
## Example apps
Three complete, runnable projects live in the [GitHub repo](https://github.com/subh05sus/anvil-js/tree/main/examples):
Express-parity REST API: dynamic routes, scoped middleware, static files, and a route exposed as an MCP tool.
A minimal project demonstrating the zero-extra-code MCP wedge — CRUD routes and a standalone tool, all served by `anvil mcp`.
An agent route with a refund tool gated behind human approval — suspend, approve, and resume, with durable checkpointing.
## Next steps
Ready to build? Start with the quickstart to have a working Anvil API in under five minutes.
Install Anvil, create your first route, and run the dev server.
Detailed setup: Node version, package managers, ESM config, and optional peer deps.
# Connect Anvil MCP Server to Claude Desktop — Guide
Source: https://anvil.thatdevguy.in/mcp/claude-desktop
Use anvil mcp --stdio to connect your Anvil tools to Claude Desktop with a single config snippet — no separate server process required.
Claude Desktop discovers MCP servers through a local config file and communicates with them over stdin/stdout. Anvil's `--stdio` flag switches the MCP server from Streamable HTTP to the newline-delimited JSON-RPC transport that Claude Desktop (and other local clients) expect.
## Transport modes
Anvil supports two MCP transport modes. Choose the one that fits your client:
The default mode starts an HTTP server that accepts `POST` requests at `/mcp`. Use this for remote agents, AI frameworks (Vercel AI SDK, LangChain, etc.), or any client that speaks HTTP.
```bash theme={null}
anvil mcp
# [anvil] mcp (Streamable HTTP) on http://localhost:3100/mcp
# [anvil] 2 tool(s): get_users_by_id, wordCount
```
Point your agent or framework at `http://localhost:3100/mcp` (or the host/port you configure with `--port` and `--endpoint`).
The `--stdio` flag switches to the stdio transport. Anvil reads JSON-RPC requests from stdin and writes responses to stdout, one message per line. Claude Desktop (and other local clients) spawn `anvil mcp --stdio` as a child process and communicate over its standard streams.
```bash theme={null}
anvil mcp --stdio
# Diagnostics go to stderr — stdout is reserved for the protocol.
```
In stdio mode, all diagnostic output (tool counts, startup messages, errors) goes to **stderr**. Never write to stdout in your tool handlers — stdout is the protocol channel and corrupting it will break the JSON-RPC framing.
## Configure Claude Desktop
To add your Anvil project as an MCP server in Claude Desktop, edit the `claude_desktop_config.json` file and add an entry under `mcpServers`:
```json theme={null}
{
"mcpServers": {
"anvil": {
"command": "npx",
"args": ["anvil", "mcp", "--stdio"],
"cwd": "/path/to/your/project"
}
}
}
```
Replace `/path/to/your/project` with the absolute path to the directory that contains your `server/` folder. Claude Desktop will spawn `npx anvil mcp --stdio` in that directory whenever it needs to call one of your tools.
**Where to find `claude_desktop_config.json`:**
* **macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
* **Windows:** `%APPDATA%\Claude\claude_desktop_config.json`
Create the file if it doesn't exist yet. After saving, restart Claude Desktop for the changes to take effect.
## Step-by-step setup
Make sure you have at least one MCP-exposed route (`meta.mcp.expose = true`) or a file in `server/tools/`. Without any tools, the server starts but Claude Desktop has nothing to call.
```ts theme={null}
// server/routes/users/[id]/get.ts
export const meta = { mcp: { expose: true, description: 'Fetch a user by ID' } };
export const paramsSchema = z.object({ id: z.string() });
export default (ctx) => findUser(ctx.params.id);
```
Confirm the server starts cleanly before touching the Claude Desktop config:
```bash theme={null}
npx anvil mcp --stdio
```
You should see a line on stderr listing your tools. If there are any schema errors, fix them before proceeding.
Open the config file for your OS (see the note above) and add the `anvil` server entry:
```json theme={null}
{
"mcpServers": {
"anvil": {
"command": "npx",
"args": ["anvil", "mcp", "--stdio"],
"cwd": "/absolute/path/to/your/project"
}
}
}
```
Quit and reopen Claude Desktop. Open a new conversation — your Anvil tools should now appear in the tool list. Ask Claude to call one of them to confirm the connection.
## Using a local binary instead of npx
If you have `anvil` installed globally or in your project's `.bin/`, reference it directly to avoid the `npx` startup overhead:
```json theme={null}
{
"mcpServers": {
"anvil": {
"command": "/absolute/path/to/your/project/node_modules/.bin/anvil",
"args": ["mcp", "--stdio"],
"cwd": "/absolute/path/to/your/project"
}
}
}
```
## Connecting other MCP-compatible clients
The Streamable HTTP mode (`anvil mcp` without `--stdio`) works with any client or framework that implements the MCP spec over HTTP:
| Client / Framework | Connect to |
| ------------------------ | -------------------------------- |
| Vercel AI SDK | `http://localhost:3100/mcp` |
| LangChain / LangGraph | `http://localhost:3100/mcp` |
| Custom JSON-RPC client | `POST http://localhost:3100/mcp` |
| Any MCP-compatible agent | `http://localhost:3100/mcp` |
Change the default port or endpoint path with `--port` and `--endpoint`:
```bash theme={null}
anvil mcp --port 4000 --endpoint /tools
# [anvil] mcp (Streamable HTTP) on http://localhost:4000/tools
```
Run `anvil lint` before connecting Claude Desktop to make sure every exposed route's schema converts cleanly. Lint catches issues at build time rather than surfacing them as confusing runtime errors inside Claude.
# Expose Anvil Routes as MCP Tools — Step-by-Step
Source: https://anvil.thatdevguy.in/mcp/exposing-routes
Add two exports to any route file and Anvil automatically converts it into a fully typed MCP tool — same handler, no extra code.
Exposing a route as an MCP tool requires only two additions to a file you've already written: a `meta` export that opts the route in, and a `paramsSchema` (or `bodySchema`) that describes its inputs. Anvil does the rest — it converts the Zod schema to JSON Schema, names the tool, and wires it into the MCP server.
## The `meta.mcp` annotation
Add a `meta` export with an `mcp` key to any route handler file:
```ts theme={null}
// server/routes/users/[id]/get.ts
import { HttpError, type Context } from 'anvil-sdk';
import { z } from 'zod';
import { USERS } from '../data';
export const meta = { mcp: { expose: true, description: 'Fetch a user by ID' } };
export const paramsSchema = z.object({ id: z.string() });
export default function handler(ctx: Context) {
const user = USERS.find((u) => u.id === ctx.params.id);
if (!user) throw new HttpError(404, `No user with id "${ctx.params.id}"`);
return user;
}
```
The three pieces work together:
Set to `true` to include this route in the MCP toolset. Routes without this field are ignored by `anvil mcp`.
The human-readable description shown to MCP clients and AI agents when they list available tools. Keep it concise and imperative: *"Fetch a user by ID"*, not *"This endpoint fetches a user"*.
An explicit tool name. Must match `[a-zA-Z0-9_-]{1,64}`. When omitted, Anvil derives a name from the route method and path segments — for example, `GET /users/:id` becomes `get_users_by_id`.
A Zod object schema that describes the route's URL parameters. Keys must match the dynamic segments in the folder path (`[id]` → `id`). Anvil merges `paramsSchema` and `bodySchema` into a single tool input schema.
A Zod object schema for the request body (non-GET routes). Merged with `paramsSchema` when both are present.
## Automatic schema conversion
When `anvil mcp` starts, it calls the built-in Zod-to-JSON-Schema converter on each exposed route's merged input schema. The result is advertised to MCP clients in the `tools/list` response so agents know exactly what arguments to pass.
At call time, Anvil re-validates the incoming arguments against the original Zod schema before invoking your handler — so any refinements or constraints you defined still enforce, even if the JSON Schema representation was slightly looser.
### Zod types that convert cleanly
The converter supports the full data-shape subset of Zod:
| Zod type | JSON Schema equivalent |
| ----------------------------------------------------------------------------------- | -------------------------------------------------------- |
| `z.string()` (with `.min()`, `.max()`, `.email()`, `.url()`, `.uuid()`, `.regex()`) | `{ type: "string", ... }` |
| `z.number()` / `z.number().int()` (with `.min()`, `.max()`) | `{ type: "number" }` / `{ type: "integer" }` |
| `z.boolean()` | `{ type: "boolean" }` |
| `z.literal(value)` | `{ const: value }` |
| `z.enum([...])` | `{ type: "string", enum: [...] }` |
| `z.object({...})` | `{ type: "object", properties: {...}, required: [...] }` |
| `z.array(...)` (with `.min()`, `.max()`) | `{ type: "array", items: {...} }` |
| `z.optional(...)` / `z.default(...)` | field omitted from `required` |
| `z.nullable(...)` | `{ type: ["string", "null"] }` |
| `z.union([...])` | `{ anyOf: [...] }` |
| `z.record(...)` | `{ type: "object", additionalProperties: {...} }` |
| `z.tuple([...])` | `{ type: "array", items: [...], minItems, maxItems }` |
The following Zod constructs cannot be losslessly represented as JSON Schema and will throw a `SchemaConversionError` during `anvil lint` or `anvil mcp` startup:
* **`.transform()` / `.refine()` / `.superRefine()`** — move transformation logic into the handler body; the schema must be a plain data shape.
* **`z.lazy()`** — recursive schemas cannot be expressed as a static JSON Schema tool. Flatten or bound the recursion.
* **`.pipe()`** — split into a plain input schema and a handler-side transform.
* **`z.date()`** — use `z.string().datetime()` instead; JSON has no native date type.
* **`z.bigint()`** — use `z.string()` instead; JSON has no bigint type.
* **`z.map()`** — use `z.record()` instead so it serializes as a JSON object.
* **`z.set()`** — use `z.array()` instead; uniqueness can be enforced in the handler.
* **`z.intersection()`** — use `.merge()` to combine object shapes instead.
* **`z.function()` / `z.promise()` / `z.symbol()` / `z.undefined()` / `z.never()`** — these types have no JSON-serializable representation.
These restrictions only apply to schemas on MCP-exposed routes. Non-exposed routes can use any Zod feature.
## Tool naming
Anvil derives the tool name from the HTTP method and path segments by default:
```text theme={null}
GET /users/:id → get_users_by_id
POST /orders → post_orders
GET / → get_root
```
Set `meta.mcp.name` to override this when you want a domain-friendly name:
```ts theme={null}
export const meta = {
mcp: { expose: true, name: 'fetch_user', description: 'Fetch a user by ID' },
};
```
Tool names must be globally unique across all routes and standalone tools. Collisions are a hard startup error.
## Validating with `anvil lint`
Run `anvil lint` before deploying to catch schema and naming issues at build time:
```bash theme={null}
anvil lint # warnings are reported, exits 0
anvil lint --strict # warnings become errors, exits non-zero
```
`anvil lint` checks two things for every MCP-exposed route:
1. **Param key match** — every key in `paramsSchema` corresponds to a `[dynamic]` segment in the folder path, and vice versa.
2. **Lossless conversion** — the schema converts to JSON Schema without throwing `SchemaConversionError`.
## Running the MCP server
Once your routes are annotated, start the server:
```bash theme={null}
anvil mcp
```
You'll see output like:
```text theme={null}
[anvil] mcp (Streamable HTTP) on http://localhost:3100/mcp
[anvil] 2 tool(s): get_users_by_id, get_widgets_by_id
```
Each tool listed corresponds to one `meta.mcp.expose = true` route that Anvil found under `server/routes/`. The server is now ready to accept `tools/list` and `tools/call` requests from any MCP-compatible client.
Add `anvil lint` to your CI pipeline so schema issues surface on every pull request — long before `anvil mcp` ever starts in production.
# Anvil JS MCP: Expose Any Route as an AI Agent Tool
Source: https://anvil.thatdevguy.in/mcp/overview
Anvil's MCP layer exposes your routes and standalone tools to any AI agent or MCP client — one codebase, one handler, no duplicated schemas.
The Model Context Protocol (MCP) is an open standard that lets AI agents and clients like Claude discover and call tools over a well-defined interface. Anvil builds MCP support directly into the framework: the same route handler you write for your REST API can simultaneously be an agent-callable tool, with zero schema duplication.
## Why Anvil's approach is different
Most frameworks require you to maintain two parallel surfaces — an HTTP API and a separate tool definition — each with its own schema and validation logic. Anvil eliminates that split. When you mark a route with `meta.mcp.expose`, Anvil reads its `paramsSchema` (a Zod object you already defined for HTTP validation), converts it to JSON Schema automatically, and advertises the tool to MCP clients. The same handler function runs in both contexts.
```
┌─────────────────────────────────────────────────┐
│ server/routes/users/[id]/get.ts │
│ │
│ meta.mcp.expose = true ──────────────────┐ │
│ paramsSchema = z.object({ id: z.string() })│ │
│ export default (ctx) => findUser(ctx.params.id) │
└─────────────────────────────────────────────────┘
│ │
▼ ▼
GET /users/:id MCP tool: get_users_by_id
(HTTP route) (agent-callable)
```
Standalone tools that don't map to an HTTP route live in `server/tools/` and are served by the same command.
## How it works
Add `meta.mcp.expose = true` and a `paramsSchema` to any route file, or drop a new file in `server/tools/` with a default function and an `inputSchema`.
Start the MCP server with `anvil mcp`. By default it listens on `:3100/mcp` over Streamable HTTP. Pass `--stdio` for local clients like Claude Desktop.
```bash theme={null}
anvil mcp # Streamable HTTP → http://localhost:3100/mcp
anvil mcp --stdio # stdio transport for Claude Desktop
```
Any MCP-compatible client — Claude Desktop, a custom agent built with the Vercel AI SDK, LangChain, or your own JSON-RPC client — can now discover and call your tools. Anvil handles the `tools/list` and `tools/call` protocol methods.
## Transport modes
| Mode | Command | Endpoint | Best for |
| --------------- | ------------------- | --------------------------- | -------------------------------- |
| Streamable HTTP | `anvil mcp` | `http://localhost:3100/mcp` | Remote agents, AI frameworks, CI |
| stdio | `anvil mcp --stdio` | stdin/stdout | Claude Desktop, local CLI tools |
The default port and endpoint (`--port` / `--endpoint` flags) can be changed at the command line. See the [CLI reference](/cli/mcp) for all flags.
## What gets served
`anvil mcp` collects tools from two places and merges them into a single MCP server:
1. **Route-derived tools** — every route file that exports `meta.mcp.expose = true`
2. **Standalone tools** — every file in `server/tools/` that default-exports a function
Tool names must be globally unique. Collisions are caught at startup with a clear error pointing to both files.
`anvil lint` validates MCP-exposed routes at build time: it checks that `paramsSchema` keys match the folder params in the path and that the schema converts losslessly to JSON Schema. Run `anvil lint --strict` to treat schema-conversion warnings as errors.
## Learn more
Mark a route with `meta.mcp.expose` and let Anvil generate the tool definition from your existing Zod schema.
Define tools that aren't tied to an HTTP route in `server/tools/` and serve them from the same `anvil mcp` command.
Connect your Anvil MCP server to Claude Desktop in minutes using stdio mode and a one-line config snippet.
# Standalone MCP Tools in Anvil JS via server/tools/
Source: https://anvil.thatdevguy.in/mcp/standalone-tools
Add tool files to server/tools/ to expose MCP-callable functions that aren't tied to any HTTP route — served by the same anvil mcp command.
Standalone tools let you expose logic to MCP clients that doesn't belong on any HTTP route. Drop a file into `server/tools/` and `anvil mcp` picks it up automatically — no registration, no config change needed.
## File shape
Every file in `server/tools/` must default-export a function. Alongside the default export, provide an `inputSchema` and a `description`:
```ts theme={null}
// server/tools/wordCount.ts
import { z } from 'zod';
export const description = 'Count the words in a piece of text';
export const inputSchema = z.object({ text: z.string() });
export default async function wordCount(args: { text: string }) {
const words = args.text.trim().split(/\s+/).filter(Boolean);
return { words: words.length, characters: args.text.length };
}
```
The tool implementation. Receives the validated arguments as its first argument and may return any serializable value. Async functions are supported.
Describes the arguments the tool accepts. Pass a Zod object and Anvil converts it to JSON Schema automatically. When omitted (or when the export is not a Zod schema), Anvil falls back to an empty schema — so always export a Zod object to get input validation and accurate schema advertisement.
A plain-English description shown to MCP clients and AI agents. You can also export `meta.description` if you prefer to group metadata together.
An explicit tool name. When omitted, Anvil uses the filename without its extension (e.g. `wordCount.ts` → `wordCount`). Must match `[a-zA-Z0-9_-]{1,64}`.
## Multi-parameter tools
Tools can accept any number of inputs. Define them all in the Zod object:
```ts theme={null}
// server/tools/add.ts
import { z } from 'zod';
export const description = 'Add two numbers';
export const inputSchema = z.object({ a: z.number(), b: z.number() });
export default async function add(args: { a: number; b: number }) {
return { sum: args.a + args.b };
}
```
When an MCP client calls this tool, Anvil validates the incoming arguments against the Zod schema before your function runs — any constraint violations are returned as a graceful tool error, not a protocol crash.
## Generating the input schema dynamically
If you need constructs that Zod can't express losslessly — or you're generating tool definitions programmatically — build the Zod schema at module load time rather than inlining it:
```ts theme={null}
// server/tools/search.ts
import { z } from 'zod';
export const description = 'Full-text search over the product catalog';
const filtersSchema = z.object({
category: z.string().optional(),
inStock: z.boolean().optional(),
});
export const inputSchema = z.object({
query: z.string().min(1),
limit: z.number().int().min(1).max(100).default(10),
filters: filtersSchema.optional(),
});
export default async function search(args: {
query: string;
limit?: number;
filters?: { category?: string; inStock?: boolean };
}) {
// ... implementation
return { results: [] };
}
```
Always export a Zod object as `inputSchema`. Anvil uses the Zod schema both to advertise the tool's shape to MCP clients (via JSON Schema conversion) and to validate arguments at call time. If `inputSchema` is absent or not a Zod schema, Anvil falls back to an empty schema and skips validation.
Files whose names start with `_` are ignored by the tool scanner. Use this convention for shared helpers you want to co-locate in `server/tools/` without exposing them as tools.
## How tools are served
When `anvil mcp` starts, it scans `server/tools/` for all `.ts`, `.mts`, `.js`, and `.mjs` files (recursive, sorted alphabetically, excluding `_`-prefixed files and `.d.ts` declaration files). Each valid file becomes one MCP tool.
Standalone tools are merged with route-derived tools into a single toolset. Tool names must be unique across both sources — a collision between a file in `server/tools/` and a route with the same derived name is a hard startup error.
```bash theme={null}
anvil mcp
# [anvil] mcp (Streamable HTTP) on http://localhost:3100/mcp
# [anvil] 3 tool(s): get_users_by_id, wordCount, add
```
Keep `server/tools/` files focused: one exported function per file, co-located with its schema. This makes each tool easy to test in isolation and keeps the MCP surface easy to audit.
# Prompt Registry: Version, Diff, and Manage Prompts
Source: https://anvil.thatdevguy.in/memory/prompt-registry
Store prompts as versioned, immutable artifacts with PromptRegistry. Pin routes to specific versions, diff changes, and roll back without a code deploy.
Prompts are code. When you hardcode a system prompt in a handler file, changing it means a full redeploy, you lose history, and rolling back means reverting a commit. `PromptRegistry` treats prompts as first-class, versioned artifacts backed by your existing `StateStore`. You can pin a route to a specific version, diff two versions line-by-line, and roll forward or backward entirely independently of code changes.
## Setup
Create a `PromptRegistry` backed by any `StateStore`:
```ts theme={null}
import { PromptRegistry } from 'anvil-sdk/prompt';
import { SqliteStateStore } from 'anvil-sdk/store';
const store = await SqliteStateStore.open(); // .anvil/state.db
const registry = new PromptRegistry(store);
```
For tests or one-off scripts, pass a `MemoryStateStore` instead — it requires no file I/O or native dependencies.
## Registering prompts
Call `registry.register(name, template, note?)` to append a new immutable version. Version numbers start at `1` and increment automatically.
```ts theme={null}
const v1 = await registry.register(
'support-system',
'You are a helpful customer support agent. Answer questions concisely and accurately.',
'Initial version',
);
// → { version: 1, template: '...', note: 'Initial version', createdAt: 1718000000000 }
// Registering again creates v2 — v1 is never modified
const v2 = await registry.register(
'support-system',
'You are a friendly customer support agent. Answer questions concisely, accurately, and with empathy. Always end with "Is there anything else I can help you with?"',
'Add empathy tone and closing question',
);
// → { version: 2, ... }
```
`register` returns a `PromptVersion` object:
| Field | Type | Description | |
| ----------- | -------- | ------------------------------------------------------------- | ------------------------------------------ |
| `version` | `number` | Monotonically increasing version number | |
| `template` | `string` | The prompt template (may contain `{{placeholder}}` variables) | |
| `note` | \`string | undefined\` | Optional human-readable change description |
| `createdAt` | `number` | Unix timestamp (ms) when this version was created | |
## Retrieving prompts
### `registry.get(name, version?)`
Returns the latest version when `version` is omitted, or the specific version if provided. Returns `undefined` if the prompt or version does not exist.
```ts theme={null}
// Get the latest version
const latest = await registry.get('support-system');
// Pin to a specific version
const pinned = await registry.get('support-system', 1);
```
### `registry.list(name)`
Returns all versions for a named prompt as a `PromptVersion[]` array, oldest first.
```ts theme={null}
const versions = await registry.list('support-system');
// → [{ version: 1, ... }, { version: 2, ... }]
```
### `registry.names()`
Returns an array of all prompt names registered in the store.
```ts theme={null}
const names = await registry.names();
// → ['support-system', 'onboarding-intro', 'safety-guardrail']
```
## Diffing versions
`registry.diff(name, from, to)` returns a line-level diff between two versions. Use this in a deploy pipeline or admin UI to review prompt changes before promoting them.
```ts theme={null}
const diff = await registry.diff('support-system', 1, 2);
// diff.from → 1
// diff.to → 2
// diff.added → lines present in v2 but not v1
// diff.removed → lines present in v1 but not v2
```
`PromptDiff` fields:
| Field | Type | Description |
| --------- | ---------- | ----------------------------------- |
| `from` | `number` | The base version number |
| `to` | `number` | The target version number |
| `added` | `string[]` | Lines added in the target version |
| `removed` | `string[]` | Lines removed from the base version |
## Rendering templates
`renderPrompt(template, vars)` fills `{{varName}}` placeholders in a template string with values from a plain object. Missing variables are replaced with empty strings.
```ts theme={null}
import { renderPrompt } from 'anvil-sdk/prompt';
const template = 'You are a support agent for {{company}}. Today is {{date}}.';
const rendered = renderPrompt(template, {
company: 'Acme Corp',
date: new Date().toDateString(),
});
// → 'You are a support agent for Acme Corp. Today is Mon Jun 10 2025.'
```
Placeholders support optional whitespace: `{{ varName }}` and `{{varName}}` are both valid.
## Using a registry in an agent
Fetch the prompt template at startup, render it with runtime variables, and pass it as the agent's `system` prompt.
```ts theme={null}
import { PromptRegistry } from 'anvil-sdk/prompt';
import { SqliteStateStore } from 'anvil-sdk/store';
const store = await SqliteStateStore.open();
const registry = new PromptRegistry(store);
await registry.register(
'support-system',
'You are a support agent for {{company}}. Be concise and empathetic. Today\'s date is {{date}}.',
'Initial version with date and company vars',
);
```
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { renderPrompt } from 'anvil-sdk/prompt';
// Fetch the latest version (or pin with a version number)
const promptVersion = await registry.get('support-system');
export default defineAgent({
name: 'support-agent',
model: 'gpt-4o-mini',
system: renderPrompt(promptVersion!.template, {
company: process.env.COMPANY_NAME ?? 'Acme',
date: new Date().toDateString(),
}),
async run(ctx) {
return ctx.complete();
},
});
```
Pin a different version number in `registry.get()` and restart the process — no code change required.
```ts theme={null}
// Roll back to v1 by changing the version argument
const promptVersion = await registry.get('support-system', 1);
```
## Reviewing changes programmatically
Run a diff in CI to surface prompt changes alongside code changes:
```ts theme={null}
const history = await registry.list('support-system');
const lastTwo = history.slice(-2);
if (lastTwo.length === 2) {
const diff = await registry.diff(
'support-system',
lastTwo[0].version,
lastTwo[1].version,
);
console.log('Added lines:');
diff.added.forEach((line) => console.log(' + ' + line));
console.log('Removed lines:');
diff.removed.forEach((line) => console.log(' - ' + line));
}
```
Store the `PromptRegistry` instance as a singleton module export so every route and agent shares the same registry and the same underlying `StateStore` connection.
# RAG Pipeline: Build Retrieval-Augmented Generation
Source: https://anvil.thatdevguy.in/memory/rag-retrieval
Index documents with Retriever, embed and store chunks in a VectorStore, then inject the most relevant context into your agent loop automatically.
Retrieval-Augmented Generation (RAG) lets your agents answer questions grounded in your own documents rather than relying solely on the model's training data. Anvil's `Retriever` handles the full pipeline: splitting documents into chunks, embedding them, storing vectors, and retrieving the most similar chunks for any query. The `retrievalContext` step wires this directly into the agent loop so relevant chunks appear in the system prompt automatically.
## Core concepts
Orchestrates chunking, embedding, and indexing. Call `index()` to add documents and `retrieve()` to query them.
Converts text to vectors. Use `HashEmbedder` for dev/tests, or plug in an OpenAI/Gemini provider embedder for semantic similarity.
Persists and queries `(id, text, embedding)` records. Choose `MemoryVectorStore` for tests or `SqliteVectorStore` for persistence.
A `ContextStep` that auto-retrieves chunks for the current query and appends them to the agent's system prompt before the model call.
## Setting up a Retriever
Import the pieces you need and assemble a `Retriever`:
```ts theme={null}
import { Retriever, HashEmbedder, SqliteVectorStore } from 'anvil-sdk/rag';
const embedder = new HashEmbedder(); // swap for OpenAI/Gemini in production
const store = await SqliteVectorStore.open(); // persists to .anvil/vectors.db
const retriever = new Retriever({ embedder, store });
```
### Constructor options
| Option | Type | Required | Description |
| ---------- | -------------- | -------- | ----------------------------------- |
| `embedder` | `Embedder` | ✅ | Converts text to vectors |
| `store` | `VectorStore` | ✅ | Persists and queries vector records |
| `chunk` | `ChunkOptions` | — | Chunking configuration (see below) |
## Indexing documents
Call `retriever.index(docs)` to chunk, embed, and store documents. It returns the number of chunks added.
```ts theme={null}
import type { IndexDoc } from 'anvil-sdk/rag';
const docs: IndexDoc[] = [
{
id: 'policy-001',
text: 'Our refund policy allows returns within 30 days of purchase...',
metadata: { source: 'handbook', section: 'refunds' },
},
{
id: 'policy-002',
text: 'Shipping times vary by region. Standard delivery takes 3–5 business days...',
metadata: { source: 'handbook', section: 'shipping' },
},
];
const chunksAdded = await retriever.index(docs);
console.log(`Indexed ${chunksAdded} chunks`);
```
### IndexDoc fields
| Field | Type | Required | Description |
| ---------- | ------------------------- | -------- | ---------------------------------------------------------------- |
| `id` | `string` | — | Stable document identifier. Auto-generated as `doc-N` if omitted |
| `text` | `string` | ✅ | Full document text to chunk and embed |
| `metadata` | `Record` | — | Arbitrary metadata returned with each result |
## Retrieving chunks
Call `retriever.retrieve(query, options?)` to get the top-K most similar chunks for a query string.
```ts theme={null}
const results = await retriever.retrieve('What is your refund policy?', { topK: 3 });
for (const result of results) {
console.log(`[${result.score.toFixed(3)}] ${result.text}`);
// [0.921] Our refund policy allows returns within 30 days...
}
```
Results are `VectorQueryResult[]` sorted by descending cosine similarity:
| Field | Type | Description |
| ---------- | ------------------------- | ------------------------------------- |
| `id` | `string` | Chunk identifier (`docId#chunkIndex`) |
| `text` | `string` | Chunk text |
| `score` | `number` | Cosine similarity (0–1) |
| `metadata` | `Record` | Metadata from the original `IndexDoc` |
### RetrieveOptions
| Option | Type | Default | Description |
| -------------- | ------------- | ------- | --------------------------------------------------------- |
| `topK` | `number` | `4` | Maximum number of chunks to return |
| `trace` | `TraceHandle` | — | Records a `retrieval` span visible in the Anvil dashboard |
| `parentSpanId` | `string` | — | Parent span for nested trace trees |
## Embedders
### HashEmbedder (dev/test)
`HashEmbedder` is a zero-dependency, deterministic embedder that uses a hashed bag-of-tokens approach. It requires no API key and no network calls, making it ideal for local development and CI. Because it is not semantic, replace it with a provider embedder before going to production.
```ts theme={null}
import { HashEmbedder } from 'anvil-sdk/rag';
const embedder = new HashEmbedder(); // defaults to 256 dimensions
const embedder512 = new HashEmbedder(512); // increase dimensions for larger corpora
```
### Provider embedders
Any object that implements the `Embedder` interface works as a drop-in replacement:
```ts theme={null}
export interface Embedder {
readonly dimensions: number;
embed(texts: string[]): Promise;
}
```
```ts theme={null}
import { OpenAIEmbedder } from 'anvil-sdk/providers/openai';
const embedder = new OpenAIEmbedder({
model: 'text-embedding-3-small',
apiKey: process.env.OPENAI_API_KEY,
});
```
```ts theme={null}
import { GeminiEmbedder } from 'anvil-sdk/providers/gemini';
const embedder = new GeminiEmbedder({
model: 'text-embedding-004',
apiKey: process.env.GEMINI_API_KEY,
});
```
## Vector stores
### MemoryVectorStore (tests)
In-process, zero-dependency brute-force cosine search. Data is lost when the process exits.
```ts theme={null}
import { MemoryVectorStore } from 'anvil-sdk/rag';
const store = new MemoryVectorStore();
```
### SqliteVectorStore (production)
Persistent SQLite-backed store. Requires the `better-sqlite3` optional peer dependency. Similarity search runs in JS over all stored records — appropriate for small-to-medium corpora embedded in an application.
```ts theme={null}
import { SqliteVectorStore } from 'anvil-sdk/rag';
const store = await SqliteVectorStore.open(); // .anvil/vectors.db
const custom = await SqliteVectorStore.open('./data/kb.db'); // custom path
```
Install `better-sqlite3` when you use `SqliteVectorStore`. The package loads lazily so it never breaks builds that use only `MemoryVectorStore`.
## Custom chunking
`chunkText(text, options?)` splits text into overlapping chunks, preferring natural paragraph and sentence boundaries. You can call it yourself before indexing if you need fine-grained control.
```ts theme={null}
import { chunkText } from 'anvil-sdk/rag';
const chunks = chunkText(longDocument, {
size: 600, // max characters per chunk (default 800)
overlap: 80, // overlap between consecutive chunks (default 100)
});
```
Pass the same options directly to the `Retriever` via the `chunk` constructor option:
```ts theme={null}
const retriever = new Retriever({
embedder,
store,
chunk: { size: 600, overlap: 80 },
});
```
## Using RAG in an agent
`retrievalContext` from `anvil/agent` is a `ContextStep` that retrieves chunks for the current user query and appends them to the agent's system prompt before each model call. This is the recommended way to integrate RAG into an agent loop.
### Complete example
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { retrievalContext } from 'anvil-sdk/agent';
import { Retriever, HashEmbedder, SqliteVectorStore } from 'anvil-sdk/rag';
// 1. Build the retriever
const embedder = new HashEmbedder();
const store = await SqliteVectorStore.open();
const retriever = new Retriever({ embedder, store });
// 2. Index your knowledge base once at startup (or on ingestion)
await retriever.index([
{ id: 'faq-1', text: 'Refunds are accepted within 30 days...', metadata: { section: 'refunds' } },
{ id: 'faq-2', text: 'Standard shipping takes 3–5 business days...', metadata: { section: 'shipping' } },
{ id: 'faq-3', text: 'Contact support at help@example.com...', metadata: { section: 'contact' } },
]);
// 3. Define an agent that retrieves context automatically
export default defineAgent({
name: 'support-agent',
model: 'gpt-4o-mini',
system: 'You are a helpful customer support agent. Use the provided context to answer questions accurately.',
context: [
retrievalContext(retriever, { topK: 3 }),
],
async run(ctx) {
return ctx.complete();
},
});
```
### Custom chunk rendering
Override how retrieved chunks appear in the system prompt with the `template` option:
```ts theme={null}
retrievalContext(retriever, {
topK: 5,
template: (chunks) =>
`## Knowledge Base\n${chunks.map((c, i) => `${i + 1}. ${c}`).join('\n')}`,
})
```
Pass the request's `trace` handle to `retriever.retrieve()` if you call it manually — this records a `retrieval` span in the Anvil dashboard so you can see exactly which chunks influenced each response.
# Semantic Cache: Skip Redundant LLM Calls by Similarity
Source: https://anvil.thatdevguy.in/memory/semantic-cache
Use SemanticCache to skip redundant LLM calls when a semantically equivalent prompt has already been answered, cutting latency and token spend.
LLMs are expensive. When users ask questions that are semantically equivalent — "What's your return policy?" and "How do I return an item?" — there is no reason to run two separate model calls. `SemanticCache` stores responses keyed on embedding similarity rather than exact string matching, so a new prompt that is close enough to a cached one returns the stored answer instantly without touching the model.
## How it works
When you call `cache.get(prompt)`, `SemanticCache` embeds the prompt and queries the backing vector store for the nearest stored entry. If the top result's cosine similarity exceeds the configured `threshold` (default `0.95`) and the entry has not expired, it's a cache hit and the stored response is returned. On a miss, your code calls the LLM as normal, then stores the response with `cache.set()` for future hits.
The `cache.wrap()` convenience method handles the full get-or-produce flow in one call.
## Setup
```ts theme={null}
import { SemanticCache, HashEmbedder, SqliteVectorStore } from 'anvil-sdk/rag';
const cache = new SemanticCache({
embedder: new HashEmbedder(), // swap for a semantic embedder in production
store: await SqliteVectorStore.open(), // persists hits across restarts
threshold: 0.92, // tune for your use case
ttlMs: 1000 * 60 * 60 * 24, // 24-hour TTL
});
```
### Constructor options
| Option | Type | Default | Description |
| ----------- | ------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `embedder` | `Embedder` | — (required) | Embeds prompts for similarity comparison |
| `threshold` | `number` | `0.95` | Minimum cosine similarity for a cache hit. Lower values cast a wider net; higher values require near-exact matches |
| `store` | `VectorStore` | `MemoryVectorStore` | Backing store for cached `(embedding, response)` pairs |
| `ttlMs` | `number` | none | Entry lifetime in milliseconds. Entries older than this are treated as misses |
The default `store` is an in-process `MemoryVectorStore`. Use `SqliteVectorStore` or another persistent store to keep the cache across process restarts.
## API
### `cache.get(prompt, opts?)`
Looks up a prompt in the cache. Returns a `CacheLookup` object.
```ts theme={null}
const result = await cache.get('What is your return policy?');
if (result.hit) {
console.log('Cache hit! Score:', result.score);
return result.value; // the stored response string
}
```
`CacheLookup` fields:
| Field | Type | Description | |
| ------- | --------- | ------------------------------------------------------------------- | ------------------------------------------ |
| `hit` | `boolean` | `true` if a sufficiently similar entry was found and is still fresh | |
| `value` | \`string | undefined\` | The cached response on a hit |
| `score` | \`number | undefined\` | The cosine similarity of the nearest entry |
The optional `opts` object accepts:
| Option | Type | Description |
| -------------- | ------------- | ------------------------------------------------ |
| `trace` | `TraceHandle` | Records a `cache` span in the Anvil dashboard |
| `parentSpanId` | `string` | Parent span for nested trace trees |
| `now` | `number` | Override the current timestamp (useful in tests) |
### `cache.set(prompt, value)`
Stores a response string paired with the prompt's embedding. Call this after a model response to prime future cache hits.
```ts theme={null}
const response = await callLLM(prompt);
await cache.set(prompt, response);
```
### `cache.wrap(prompt, produce, opts?)`
The recommended entry point. Checks the cache, calls `produce()` on a miss, stores the new response, and returns a result object.
```ts theme={null}
const { value, cached } = await cache.wrap(
userPrompt,
() => callYourLLM(userPrompt),
{ trace: ctx.trace },
);
console.log(cached ? 'Served from cache' : 'Fresh LLM response');
```
The return value has:
| Field | Type | Description |
| -------- | --------- | ------------------------------------------ |
| `value` | `string` | The response (cached or freshly generated) |
| `cached` | `boolean` | `true` if the response came from the cache |
## Integrating with an agent
Wrap the model call inside `defineAgent` with `cache.wrap()` to cache agent responses by their user query:
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { SemanticCache, HashEmbedder } from 'anvil-sdk/rag';
const cache = new SemanticCache({
embedder: new HashEmbedder(),
threshold: 0.93,
ttlMs: 60 * 60 * 1000, // 1 hour
});
export default defineAgent({
name: 'faq-agent',
model: 'gpt-4o-mini',
system: 'You answer frequently asked questions about our product.',
async run(ctx) {
const userMessage = ctx.messages.at(-1)?.content as string;
const { value, cached } = await cache.wrap(
userMessage,
() => ctx.complete().then((r) => r.text),
{ trace: ctx.trace },
);
ctx.setHeader('x-cache', cached ? 'HIT' : 'MISS');
return { text: value };
},
});
```
## Tuning the threshold
The right `threshold` depends on how similar "equivalent" prompts need to be for your use case.
| Threshold | Behavior |
| --------- | ------------------------------------------------------------------------------ |
| `0.99` | Near-exact matches only. Safe for factual Q\&A with precise phrasing. |
| `0.95` | Default. Catches minor wording variations like "What's" vs "What is". |
| `0.90` | Catches paraphrases and common synonyms. May produce occasional false hits. |
| `0.80` | Aggressive caching. Good for narrow-domain FAQs; avoid for open-ended queries. |
Using a `HashEmbedder` in production limits cache effectiveness because it tracks vocabulary overlap, not semantic meaning. Use a semantic provider embedder (OpenAI `text-embedding-3-small`, Gemini `text-embedding-004`) for accurate similarity scoring.
## Observability
When you pass a `trace` handle to `cache.get()` or `cache.wrap()`, Anvil records a `cache` span that includes:
* `hit` — whether the lookup was a cache hit
* `score` — the cosine similarity of the nearest entry
* `threshold` — the configured threshold
These spans appear in the Anvil dashboard timeline so you can track cache hit rates alongside latency and cost.
# Session Memory: Persist Agent Conversational State
Source: https://anvil.thatdevguy.in/memory/session-memory
Use MemoryStore and the withMemory middleware to give each conversation its own typed, namespaced key-value memory backed by a pluggable StateStore.
Session memory gives each conversation its own typed key-value scope over the same pluggable `StateStore` used for durability checkpoints — no hand-rolled per-project memory table, and no separate persistence layer to configure.
## Attaching memory to routes
`withMemory` is a middleware that attaches a `MemoryStore` to `ctx.state`, namespaced per request:
```ts theme={null}
// server/routes/_middleware.ts
import { withMemory } from 'anvil-sdk/memory';
import { SqliteStateStore } from 'anvil-sdk/store';
const store = await SqliteStateStore.open('.anvil/memory.db');
export default [withMemory(store)];
```
By default, the namespace is the `x-session-id` request header, falling back to `'default'` if the header is absent. Pass a custom function to derive the namespace differently — from a JWT claim, a cookie, or a route param:
```ts theme={null}
export default [withMemory(store, (ctx) => ctx.headers.get('x-user-id') ?? 'anonymous')];
```
## Reading and writing memory in a handler
```ts theme={null}
// server/routes/chat/agent.ts
import { getMemory } from 'anvil-sdk/memory';
import type { Context } from 'anvil-sdk';
export default async function handler(ctx: Context) {
const memory = getMemory(ctx);
if (!memory) throw new Error('withMemory middleware not attached');
await memory.append('history', { role: 'user', text: 'What did I ask before?' });
const history = await memory.get('history');
return { history };
}
```
`getMemory(ctx)` returns `undefined` if `withMemory` wasn't attached upstream — always check before use, or attach it at the root `_middleware.ts` so every route has it.
## `MemoryStore` API
Read a typed value. Returns `undefined` if the key was never set.
Write a value, overwriting anything already stored under `key`.
Append `item` to an array value, creating the array if it doesn't exist yet. Returns the new array length. Useful for conversation history without a separate read-modify-write round trip in handler code.
Remove a key.
List all keys currently set within this instance's namespace.
Every key is stored under a namespaced prefix (`anvil:memory::`) in the underlying `StateStore`, so different sessions never collide even when sharing one store instance.
## Choosing a backing store
`MemoryStore` works over any `StateStore` implementation:
```ts theme={null}
import { MemoryStateStore } from 'anvil-sdk/store';
// Zero-dependency, in-memory — good for tests and local dev. Lost on restart.
import { SqliteStateStore } from 'anvil-sdk/store';
// Persists to disk. Requires the optional better-sqlite3 peer dependency.
```
`SqliteStateStore` also backs [durable execution checkpoints](/safety/durable-execution) — a single store instance can serve both session memory and agent-run checkpointing, since they write to different key prefixes.
## See also
* [RAG & retrieval](/memory/rag-retrieval) — semantic search over documents, distinct from per-session conversational memory
* [Semantic cache](/memory/semantic-cache) — cache LLM responses across sessions by prompt similarity
* [Durable execution](/safety/durable-execution) — checkpoint and resume agent runs using the same `StateStore` interface
Session memory gives each conversation its own typed key-value scope over the same pluggable `StateStore` used for durability checkpoints — no hand-rolled per-project memory table, and no separate persistence layer to configure.
## Attaching memory to routes
`withMemory` is a middleware that attaches a `MemoryStore` to `ctx.state`, namespaced per request:
```ts theme={null}
// server/routes/_middleware.ts
import { withMemory } from 'anvil-sdk/memory';
import { SqliteStateStore } from 'anvil-sdk/store';
const store = await SqliteStateStore.open('.anvil/memory.db');
export default [withMemory(store)];
```
By default, the namespace is the `x-session-id` request header, falling back to `'default'` if the header is absent. Pass a custom function to derive the namespace differently — from a JWT claim, a cookie, or a route param:
```ts theme={null}
export default [withMemory(store, (ctx) => ctx.headers.get('x-user-id') ?? 'anonymous')];
```
## Reading and writing memory in a handler
```ts theme={null}
// server/routes/chat/agent.ts
import { getMemory } from 'anvil-sdk/memory';
import type { Context } from 'anvil-sdk';
export default async function handler(ctx: Context) {
const memory = getMemory(ctx);
if (!memory) throw new Error('withMemory middleware not attached');
await memory.append('history', { role: 'user', text: 'What did I ask before?' });
const history = await memory.get('history');
return { history };
}
```
`getMemory(ctx)` returns `undefined` if `withMemory` wasn't attached upstream — always check before use, or attach it at the root `_middleware.ts` so every route has it.
## `MemoryStore` API
Read a typed value. Returns `undefined` if the key was never set.
Write a value, overwriting anything already stored under `key`.
Append `item` to an array value, creating the array if it doesn't exist yet. Returns the new array length. Useful for conversation history without a separate read-modify-write round trip in handler code.
Remove a key.
List all keys currently set within this instance's namespace.
Every key is stored under a namespaced prefix (`anvil:memory::`) in the underlying `StateStore`, so different sessions never collide even when sharing one store instance.
## Choosing a backing store
`MemoryStore` works over any `StateStore` implementation:
```ts theme={null}
import { MemoryStateStore } from 'anvil-sdk/store';
// Zero-dependency, in-memory — good for tests and local dev. Lost on restart.
import { SqliteStateStore } from 'anvil-sdk/store';
// Persists to disk. Requires the optional better-sqlite3 peer dependency.
```
`SqliteStateStore` also backs [durable execution checkpoints](/safety/durable-execution) — a single store instance can serve both session memory and agent-run checkpointing, since they write to different key prefixes.
## See also
* [RAG & retrieval](/memory/rag-retrieval) — semantic search over documents, distinct from per-session conversational memory
* [Semantic cache](/memory/semantic-cache) — cache LLM responses across sessions by prompt similarity
* [Durable execution](/safety/durable-execution) — checkpoint and resume agent runs using the same `StateStore` interface
# Cost Governor: Cap Per-Request Token and USD Spend
Source: https://anvil.thatdevguy.in/observability/cost-governor
Cap per-request token usage and USD spend. Choose to block, degrade to a cheaper model, or route to human approval when a budget limit is hit.
Anvil's Cost Governor gates every model call in an agent run against a per-request spending budget. Set a `budget` on `defineAgent` and the runtime automatically tracks cumulative token usage and USD cost, checking the budget before each iteration of the agent loop. When a cap is breached you choose the response: throw an error, switch to a cheaper model, or route the run to human approval.
## Configuring a budget on defineAgent
Pass a `BudgetConfig` to the `budget` option on `defineAgent`. The governor is instantiated per request, so limits apply to a single agent run, not across all users.
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { openai } from 'anvil-sdk/llm';
export const post = defineAgent({
client: openai({ model: 'gpt-4o' }),
budget: {
maxTokens: 4000,
maxUsd: 0.10,
onBreach: 'block',
},
// ...
});
```
## BudgetConfig fields
Hard cap on the total number of tokens (input + output) consumed by this run. The governor checks cumulative usage before each model call; once the count equals or exceeds this value, the configured `onBreach` action fires.
Hard cap on the total USD cost for this run. Computed from per-call `costUsd` values reported by the LLM client. Like `maxTokens`, this is checked before each model call.
Action to take when a cap is hit:
* **`'block'`** — throws `BudgetExceededError` immediately, stopping the run and returning a 500 to the client.
* **`'degrade'`** — signals that the run should continue on a cheaper model. The `onBreach` callback on `CostGovernor` receives the breach info so your code can switch models.
* **`'approve'`** — suspends the run and routes it through the human-in-the-loop approval queue, just like a tool that throws `ApprovalRequiredError`.
## BudgetExceededError
When `onBreach` is `'block'`, the runtime throws a `BudgetExceededError`. Catch it in error-handling middleware to return a structured response:
```ts theme={null}
import { BudgetExceededError } from 'anvil-sdk/trace';
// In a global error handler or try/catch around the agent run:
try {
// ... agent logic
} catch (err) {
if (err instanceof BudgetExceededError) {
console.warn(
`Budget exceeded: $${err.spentUsd.toFixed(4)} spent, ` +
`${err.spentTokens} tokens used. ` +
`Limit: maxUsd=${err.limit.maxUsd}, maxTokens=${err.limit.maxTokens}`
);
return Response.json({ error: 'Budget exceeded' }, { status: 429 });
}
throw err;
}
```
`BudgetExceededError` exposes three properties:
Total USD cost accumulated by this run at the time of the breach.
Total tokens (input + output) consumed by this run at the time of the breach.
The `BudgetConfig` that was in effect — useful for building informative error messages.
## Using CostGovernor directly
`defineAgent` wires `CostGovernor` for you when you pass a `budget`. For custom agent loops built with `runAgent` or `streamAgent`, construct and pass the governor explicitly:
```ts theme={null}
import { runAgent } from 'anvil-sdk/agent';
import { CostGovernor } from 'anvil-sdk/trace';
const governor = new CostGovernor(
{ maxUsd: 0.05, maxTokens: 2000, onBreach: 'block' },
(info) => {
// Called for 'degrade' and 'approve' — not for 'block'.
console.log(`Budget breach (${info.action}): $${info.spentUsd.toFixed(4)}`);
},
);
const result = await runAgent({
client,
messages,
governor,
// ...
});
```
### CostGovernor methods
Accumulate usage from a completed model call. Called automatically by the Anvil agent runtime; call it manually only if you are driving the LLM client directly.
Check the current spend against configured caps. Returns `'ok'` if within budget, throws `BudgetExceededError` if `onBreach` is `'block'`, or calls the `onBreach` callback and returns the action for `'degrade'` and `'approve'`.
Returns `true` if the current accumulated spend has reached or exceeded any configured cap. Does not throw or trigger the breach action.
Read-only accessor for the total USD cost accumulated so far.
Read-only accessor for the total token count accumulated so far.
## Tracking cost across LLM calls
The LLM client reports `costUsd` per generate call based on the model's pricing. The governor accumulates these values across every iteration of the agent loop. Aborted streams (client disconnects mid-response) are counted too, so a disconnect never creates a cost blind spot.
Set both `maxTokens` and `maxUsd` together to guard against runaway loops independently of pricing changes. A `maxTokens` cap fires predictably regardless of the model's cost per token.
## Example: degrade to a cheaper model on breach
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { CostGovernor } from 'anvil-sdk/trace';
import { openai } from 'anvil-sdk/llm';
const premiumClient = openai({ model: 'gpt-4o' });
const fallbackClient = openai({ model: 'gpt-4o-mini' });
export const post = defineAgent({
// resolveClient picks the model dynamically based on request state
resolveClient: (ctx) => ctx.state.useFallback ? fallbackClient : premiumClient,
budget: {
maxUsd: 0.20,
onBreach: 'degrade',
},
// ...
});
```
When `onBreach` is `'degrade'`, the `CostGovernor` fires its `onBreach` callback with `{ action: 'degrade', spentUsd, spentTokens }`. Wire that callback to update request state or swap the active client.
# The /_anvil Dashboard: Inspect Traces and Agent Costs
Source: https://anvil.thatdevguy.in/observability/dashboard
Anvil's built-in observability UI shows trace trees, token usage, and per-run costs in your browser — no extra tooling required.
Anvil ships a self-contained observability dashboard that renders trace trees, token usage, and cost per run directly in your browser. In development, `anvil dev` serves it automatically at `/_anvil`. In production, mount it yourself as middleware so you control who can reach it.
The dashboard reads from the same `TraceStore` you pass to your `Tracer`, so there is no separate sink to configure — traces written during a request are visible the moment the run ends.
## What the dashboard shows
* **Trace list** — every agent run, sorted by most recent, with name, status badge, duration, token count, and cost.
* **Span tree** — drill into any trace to see the nested agent → model → tool → retrieval hierarchy, with per-span attributes (model, provider, token usage, tool inputs/outputs) and error details.
* **Token usage and cost** — rolled-up `totalInputTokens`, `totalOutputTokens`, and `totalCostUsd` displayed per trace and per span.
The page polls the trace list every three seconds, so it stays current during an active development session without any WebSocket setup.
## Enabling in production with dashboardMiddleware
For deployed environments, import `dashboardMiddleware` from `anvil/trace` and add it to your root `_middleware.ts`:
```ts theme={null}
import { dashboardMiddleware } from 'anvil-sdk/trace';
import { SqliteTraceStore } from 'anvil-sdk/trace';
const store = await SqliteTraceStore.open('.anvil/traces.db');
export default dashboardMiddleware(store);
```
Mount it alongside your auth middleware to protect it:
```ts theme={null}
import { dashboardMiddleware } from 'anvil-sdk/trace';
import { SqliteTraceStore } from 'anvil-sdk/trace';
import { requireInternalToken } from '../lib/auth.js';
const store = await SqliteTraceStore.open('.anvil/traces.db');
export default [
requireInternalToken,
dashboardMiddleware(store, { path: '/_anvil' }),
];
```
In production, always protect the dashboard with authentication middleware. The dashboard exposes full trace data including tool inputs, model outputs, and cost information that should not be publicly accessible.
## dashboardMiddleware arguments
`dashboardMiddleware` accepts a `TraceStore` as its first argument and an optional `DashboardOptions` object as its second.
The first positional argument is the trace store to read from. Wire this to the **same** `SqliteTraceStore` (or other `TraceStore`) instance you passed to your `Tracer`, so the dashboard and the tracer share one underlying database.
`DashboardOptions` has one field:
The URL prefix at which the dashboard is mounted. The middleware intercepts all requests whose path starts with this value. Change it if `/_anvil` conflicts with an existing route.
## Sharing the store with the Tracer
The dashboard and the tracer must point at the same `TraceStore` instance. The idiomatic pattern is to open the store once at application startup and pass it to both:
```ts theme={null}
import { Tracer, SqliteTraceStore, dashboardMiddleware } from 'anvil-sdk/trace';
import { defineAgent } from 'anvil-sdk/agent';
import { openai } from 'anvil-sdk/llm';
// Open once — shared by tracer and dashboard.
const store = await SqliteTraceStore.open('.anvil/traces.db');
export const tracer = new Tracer(store);
// In your root _middleware.ts
export const middleware = dashboardMiddleware(store);
// In your agent route
export const post = defineAgent({
client: openai({ model: 'gpt-4o' }),
tracer,
// ...
});
```
## Dashboard API routes
The middleware also exposes a small JSON API used by the built-in UI:
| Route | Description |
| ---------------------------- | ---------------------------------------------------------------------- |
| `GET /_anvil` | Serves the self-contained dashboard HTML (no external assets) |
| `GET /_anvil/api/traces` | Lists traces (spans omitted for payload efficiency). Accepts `?limit=` |
| `GET /_anvil/api/traces/:id` | Returns a single trace with the full span tree |
You can query these endpoints directly from your own tooling or scripts.
Pair the dashboard with the [Cost Governor](/observability/cost-governor) to see budget breaches in the trace tree, and with the [OpenTelemetry exporter](/observability/otel) to forward the same traces to Datadog, Grafana, or Langfuse alongside the local view.
# OpenTelemetry Export: Forward Agent Traces to Any Backend
Source: https://anvil.thatdevguy.in/observability/otel
Forward Anvil traces to any OTel-compatible backend — Datadog, Grafana, Langfuse, Braintrust — using the built-in OTLP/HTTP exporter.
Anvil traces map directly to OpenTelemetry spans following the GenAI semantic conventions, so you can forward them to any OTLP-compatible backend without a separate SDK or agent. Use `otlpHttpExporter` to create an `onExport` callback and pass it to `Tracer` — every finished trace is serialised to the OTLP JSON format and POSTed to your collector in the background.
## How export works
When a `Tracer` is constructed with an `onExport` callback, that callback fires once per trace, immediately after the agent run ends and `traceHandle.end()` is called. The `otlpHttpExporter` factory returns exactly such a callback: it converts the `Trace` object into an OTLP `resourceSpans` payload and sends it via `fetch`. Export is fire-and-forget — failures are logged to `console.error` but never propagate to the request, so a collector outage cannot break your API.
## Setting up the exporter
```ts theme={null}
import { Tracer, SqliteTraceStore, otlpHttpExporter, traceToOtelSpans } from 'anvil-sdk/trace';
const store = await SqliteTraceStore.open('.anvil/traces.db');
const exporter = otlpHttpExporter({
url: 'https://your-collector/v1/traces',
headers: {
'Authorization': 'Bearer YOUR_API_KEY',
},
serviceName: 'my-anvil-app',
});
const tracer = new Tracer(store, { onExport: exporter });
```
Pass `tracer` to `defineAgent` as usual. Every completed trace is now forwarded to your collector alongside being stored locally.
## OtlpExporterOptions
The OTLP/HTTP endpoint to POST traces to. This is the `/v1/traces` path on your collector. Common values:
| Backend | Endpoint |
| --------------- | ------------------------------------------------------ |
| Datadog | `https://http-intake.logs.datadoghq.com/api/v2/otlp` |
| Grafana Tempo | `https://.grafana.net/otlp/v1/traces` |
| Langfuse | `https://cloud.langfuse.com/api/public/otel/v1/traces` |
| Braintrust | `https://api.braintrustdata.com/otel/v1/traces` |
| Local collector | `http://localhost:4318/v1/traces` |
HTTP headers added to every export request. Use this to pass API keys, tokens, or dataset identifiers required by your backend.
The `service.name` resource attribute included in every exported payload. Set this to identify your application in the backend's UI.
## Converting traces manually with traceToOtelSpans
If you need to control the HTTP transport yourself, use `traceToOtelSpans` directly to obtain the OTLP payload and send it with your own client:
```ts theme={null}
import { traceToOtelSpans } from 'anvil-sdk/trace';
import type { Trace } from 'anvil-sdk/trace';
function myExporter(trace: Trace): void {
const resourceSpans = traceToOtelSpans(trace, 'my-service');
fetch('https://collector.example.com/v1/traces', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ resourceSpans: [resourceSpans] }),
}).catch(console.error);
}
const tracer = new Tracer(store, { onExport: myExporter });
```
`traceToOtelSpans(trace, serviceName?)` returns an `OtlpResourceSpans` object ready to be nested under `{ resourceSpans: [...] }` in the OTLP JSON body.
## GenAI semantic convention attributes
Anvil maps span fields to the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). The following attributes are emitted on `model` spans:
| OTel attribute | Source |
| ---------------------------- | ------------------------------------ |
| `gen_ai.system` | `span.attributes.provider` |
| `gen_ai.request.model` | `span.attributes.model` |
| `gen_ai.response.model` | `span.attributes.model` |
| `gen_ai.usage.input_tokens` | `span.attributes.usage.inputTokens` |
| `gen_ai.usage.output_tokens` | `span.attributes.usage.outputTokens` |
| `gen_ai.usage.cost` | `span.attributes.costUsd` |
| `gen_ai.operation.name` | `span.kind` |
Tool spans emit `gen_ai.tool.name` from `span.attributes.name`.
## Sending to multiple backends
Compose multiple exporters inside a single `onExport` callback:
```ts theme={null}
import { otlpHttpExporter } from 'anvil-sdk/trace';
const datadogExporter = otlpHttpExporter({
url: 'https://http-intake.logs.datadoghq.com/api/v2/otlp',
headers: { 'DD-API-KEY': process.env.DD_API_KEY! },
serviceName: 'anvil-prod',
});
const langfuseExporter = otlpHttpExporter({
url: 'https://cloud.langfuse.com/api/public/otel/v1/traces',
headers: {
'Authorization': `Basic ${Buffer.from(`${process.env.LANGFUSE_PUBLIC_KEY}:${process.env.LANGFUSE_SECRET_KEY}`).toString('base64')}`,
},
});
const tracer = new Tracer(store, {
onExport: (trace) => {
datadogExporter(trace);
langfuseExporter(trace);
},
});
```
Both exporters run concurrently and independently. A failure in one does not affect the other, and neither can throw back into the request path because export errors are caught and logged internally.
## OtlpResourceSpans type reference
For advanced use cases, the full OTLP type shapes are exported from `anvil/trace`:
```ts theme={null}
interface OtlpResourceSpans {
resource: { attributes: OtlpAttribute[] };
scopeSpans: Array<{
scope: { name: string; version: string };
spans: OtlpSpan[];
}>;
}
interface OtlpSpan {
traceId: string; // 32-char lowercase hex
spanId: string; // 16-char lowercase hex
parentSpanId?: string;
name: string;
kind: number; // 1 = SPAN_KIND_INTERNAL
startTimeUnixNano: string;
endTimeUnixNano: string;
attributes: OtlpAttribute[];
status: { code: number; message?: string };
}
```
# Agent Tracing: Capture and Inspect Every Run in Anvil JS
Source: https://anvil.thatdevguy.in/observability/tracing
Record every model call, tool execution, and retrieval step as a structured span tree. Query traces in the dashboard or export them to any OTel backend.
Anvil's tracing system captures every agent run as a tree of spans — one span per model call, tool call, retrieval, cache hit, or judge evaluation — and rolls up token usage and cost into the parent trace. Pass a `Tracer` instance to `defineAgent` and every request handled by that route is automatically instrumented, with no changes required inside individual handlers or tools.
## Stores: where traces live
Anvil ships two `TraceStore` implementations. Choose one based on your environment.
Zero-dependency in-memory store. Traces survive only for the lifetime of the process — ideal for unit tests and local experimentation where you don't need a database.
SQLite-backed store powered by `better-sqlite3` (optional peer dependency). Persists traces across restarts. The recommended default for production. Default path: `.anvil/traces.db`.
### MemoryTraceStore
```ts theme={null}
import { MemoryTraceStore } from 'anvil-sdk/trace';
const store = new MemoryTraceStore();
```
Call `store.clear()` between test cases to reset state.
### SqliteTraceStore
```ts theme={null}
import { SqliteTraceStore } from 'anvil-sdk/trace';
const store = await SqliteTraceStore.open('.anvil/traces.db');
```
`SqliteTraceStore.open` creates the file and schema on first call. Pass `':memory:'` for a process-scoped SQLite database that behaves like `MemoryTraceStore` but uses the same SQL interface.
`better-sqlite3` is an optional peer dependency. Install it before using `SqliteTraceStore`:
```bash theme={null}
npm install better-sqlite3
```
## Creating a Tracer
`Tracer` wraps a `TraceStore` and optionally accepts an `onExport` hook that fires when each trace ends — this is the seam used by the [OpenTelemetry exporter](/observability/otel).
```ts theme={null}
import { Tracer, SqliteTraceStore } from 'anvil-sdk/trace';
const store = await SqliteTraceStore.open('.anvil/traces.db');
const tracer = new Tracer(store, {
// Optional: called with the finished Trace object after each run ends.
onExport: (trace) => console.log('trace finished', trace.id),
});
```
`TracerOptions` accepts one field:
Callback invoked with the completed `Trace` after `traceHandle.end()` is called. Use this to pipe traces to an external system without coupling your store to the export logic.
## Wiring the Tracer into defineAgent
Pass the `tracer` as an option to `defineAgent`. Anvil opens one `TraceHandle` per incoming request and closes it when the response stream ends.
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { Tracer, SqliteTraceStore } from 'anvil-sdk/trace';
import { openai } from 'anvil-sdk/llm';
const store = await SqliteTraceStore.open('.anvil/traces.db');
const tracer = new Tracer(store);
export const post = defineAgent({
client: openai({ model: 'gpt-4o' }),
tracer,
tools: [/* ... */],
});
```
### Naming traces
By default, Anvil names each trace `agent `. Override this with `traceName`:
```ts theme={null}
defineAgent({
tracer,
traceName: 'customer-support-agent',
// ...
});
```
```ts theme={null}
defineAgent({
tracer,
traceName: (ctx) => `support:${ctx.query.userId ?? 'anonymous'}`,
// ...
});
```
The function receives the Anvil `Context` object for the current request, so you can embed user IDs, session tokens, or any other request-scoped data in the trace name.
## The Trace and Span types
Every finished trace is a `Trace` object containing a flat list of `Span` records that form a tree via `parentId` references.
### Trace
```ts theme={null}
interface Trace {
id: string;
name: string;
startedAt: number; // Unix ms
endedAt?: number; // Unix ms; set when the run ends
status: SpanStatus; // 'running' | 'ok' | 'error' | 'aborted'
spans: Span[]; // full span tree
totalCostUsd: number; // rolled-up cost across all model spans
totalInputTokens: number;
totalOutputTokens: number;
attributes: Record;
}
```
### Span
```ts theme={null}
interface Span {
id: string;
traceId: string;
parentId?: string; // links into the tree; absent on root spans
name: string;
kind: SpanKind; // 'agent' | 'model' | 'tool' | 'retrieval' | 'cache' | 'judge' | 'http'
startedAt: number;
endedAt?: number;
status: SpanStatus;
attributes: Record; // model, provider, usage, cost, tool args/result, etc.
error?: string;
}
```
`SpanKind` identifies what produced the span:
| Kind | Description |
| ----------- | ---------------------------------------------------- |
| `agent` | The top-level agent loop span |
| `model` | A single LLM generate call, including usage and cost |
| `tool` | One tool execution |
| `retrieval` | A `ctx.retrieve()` call |
| `cache` | A semantic cache hit or miss |
| `judge` | An LLM-as-judge evaluation |
| `http` | An outbound HTTP call |
Retrieval spans from `ctx.retrieve()` are recorded in the **same** trace tree as the agent run. They appear as `kind: 'retrieval'` children of the enclosing `agent` span, so you can see the full RAG context alongside model calls in one view.
## TraceStore interface
If you need a custom backend (Redis, Postgres, etc.) implement `TraceStore` directly:
```ts theme={null}
interface TraceStore {
saveTrace(trace: Trace): void | Promise;
saveSpan(span: Span): void | Promise;
getTrace(id: string): (Trace | undefined) | Promise;
listTraces(opts?: ListTracesOptions): Trace[] | Promise;
}
```
`ListTracesOptions` accepts `limit` and `offset` for pagination.
## Full example
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { Tracer, SqliteTraceStore } from 'anvil-sdk/trace';
import { anthropic } from 'anvil-sdk/llm';
const store = await SqliteTraceStore.open('.anvil/traces.db');
const tracer = new Tracer(store);
export const post = defineAgent({
client: anthropic({ model: 'claude-3-5-sonnet-latest' }),
tracer,
traceName: (ctx) => `billing-agent:${ctx.query.customerId}`,
tools: [lookupOrder, issueRefund],
guardrails: [/* ... */],
});
```
Every request to this route produces a trace persisted to SQLite. Open `/_anvil` in the browser to inspect the span tree, token usage, and cost in real time.
# Quickstart: Build Your First Anvil JS API in Minutes
Source: https://anvil.thatdevguy.in/quickstart
Install Anvil, create your first file-based route, run the dev server, and expose a route as an MCP tool — all in under five minutes.
Anvil gets you from an empty directory to a running TypeScript API in minutes. This guide installs the package, scaffolds a project with `anvil init`, walks through a static route and a validated dynamic route, runs the dev server, exposes that route as an MCP tool with zero duplicated code, and turns a second route into a streaming AI agent — all with the same file-based routing convention.
Coming from an empty directory? Every command below works as-is. Adding Anvil to an existing project? Skip to [Installation](/installation) — `anvil init` merges into an existing `package.json` instead of overwriting it.
Create a project directory and install Anvil plus Zod (used for route and tool schemas):
```bash npm theme={null}
mkdir my-api && cd my-api
npm init -y
npm install anvil-sdk zod
```
```bash yarn theme={null}
mkdir my-api && cd my-api
yarn init -y
yarn add anvil-sdk zod
```
```bash pnpm theme={null}
mkdir my-api && cd my-api
pnpm init
pnpm add anvil-sdk zod
```
Then scaffold the project structure:
```bash theme={null}
npx anvil init
```
```text theme={null}
_____________
________/ \
_____/_______________________\___
\_______________________________/
\ /
\_______________/
|||||
|||||
_____|||||_____
/_______________\
\_______________/
ANVIL — Express for humans. Anvil for agents.
Which template do you want to start from?
1) Basic API — A minimal REST API — one static route, one dynamic route.
2) MCP server — REST routes exposed as MCP tools, plus a standalone tool.
3) Agent route — A streaming agent endpoint using the AI SDK data-stream protocol.
Pick 1-3 [1]:
```
Pick **1) Basic API** for this walkthrough — press Enter to accept the default. `anvil init` writes:
* `package.json` — adds `"type": "module"`, the `dev`/`build`/`start`/`lint` scripts, and `anvil-sdk`/`zod` to `dependencies` (merged in, not overwritten, if the file already exists)
* `tsconfig.json` — a strict, ESM-ready config
* `.gitignore` — excludes `node_modules/`, `dist/`, `.gen/`, `.anvil/`
* `server/routes/get.ts` — a starter route:
```ts server/routes/get.ts theme={null}
export default function handler() {
return { message: 'Hello from Anvil!' };
}
```
Plain objects returned from a handler are automatically serialized to JSON — no `res.json(...)` boilerplate to write. `anvil init` never overwrites a file that already exists, so re-running it is always safe.
Skip the interactive prompt with `npx anvil init --template mcp` (or `agent`), or `npx anvil init -y` to accept the `basic` default — useful when scripting or in CI. Full flag reference: [`anvil init`](/cli/init).
```bash theme={null}
npm install
npm run dev
```
```text theme={null}
[anvil] dev server listening on http://localhost:3000
GET /
```
Route files are loaded on the fly through `jiti` — no separate compile step, and no restart needed when you edit a handler. Confirm it's working:
```bash theme={null}
curl http://localhost:3000/
```
```json theme={null}
{ "message": "Hello from Anvil!" }
```
File-based routing maps folders to URL segments. A `[param]` folder captures a path parameter, typed from the folder name — add one under `server/routes/`:
```ts server/routes/users/[id]/get.ts theme={null}
import { HttpError, type Context } from 'anvil-sdk';
const USERS = [
{ id: '1', name: 'Ada Lovelace' },
{ id: '2', name: 'Alan Turing' },
];
export default function handler(ctx: Context) {
const user = USERS.find((u) => u.id === ctx.params.id);
if (!user) throw new HttpError(404, `No user "${ctx.params.id}"`);
return user;
}
```
`ctx.params.id` is typed as a `string` because the compiler reads the `[id]` folder name at build time — no manual param typing, and no way for the folder structure and your handler to drift apart. Try it:
```bash theme={null}
curl http://localhost:3000/users/1
```
```json theme={null}
{ "id": "1", "name": "Ada Lovelace" }
```
```bash theme={null}
curl http://localhost:3000/users/99
```
```json theme={null}
{ "error": "No user \"99\"" }
```
`HttpError` sets the response status (`404` here) and serializes a clean JSON error body — no manual `ctx.res.status(...)` calls.
See [Dynamic Routes](/routing/dynamic-routes) for catch-all segments (`[...path]`) and validating params against a Zod schema so malformed input 400s before your handler runs.
Add a `meta` export and a `paramsSchema`, and the exact same handler becomes callable by any MCP client — no second implementation, no schema drift:
```ts server/routes/users/[id]/get.ts theme={null}
import { HttpError, type Context } from 'anvil-sdk';
import { z } from 'zod';
export const meta = { mcp: { expose: true, description: 'Fetch a user by ID' } };
export const paramsSchema = z.object({ id: z.string() });
const USERS = [
{ id: '1', name: 'Ada Lovelace' },
{ id: '2', name: 'Alan Turing' },
];
export default function handler(ctx: Context) {
const user = USERS.find((u) => u.id === ctx.params.id);
if (!user) throw new HttpError(404, `No user "${ctx.params.id}"`);
return user;
}
```
Serve it over MCP:
```bash theme={null}
npx anvil mcp
```
```text theme={null}
[anvil] mcp (Streamable HTTP) on http://localhost:3100/mcp
[anvil] 1 tool(s): get_users_by_id
```
Any MCP client — Claude Desktop, an agent framework, or a raw `curl` JSON-RPC call — can now discover and call `get_users_by_id`. The `paramsSchema` becomes the tool's input schema automatically; there is no separate tool definition to keep in sync with the route. See [MCP Overview](/mcp/overview) for stdio transport and standalone tools that don't back an HTTP route.
Agent routes use the same file convention — an `agent.ts` file instead of `get.ts`/`post.ts` — and stream responses using the Vercel AI SDK data-stream protocol, so `useChat` works against them with no adapter code:
```ts server/routes/chat/agent.ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
const client = new LlmClient({
drivers: [new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })],
defaultModel: 'claude-opus-4-8',
});
export default defineAgent({
client,
system: 'You are a helpful assistant.',
});
```
```bash theme={null}
npm install @anthropic-ai/sdk
curl -N -X POST http://localhost:3000/chat \
-H 'content-type: application/json' \
-d '{"messages":[{"role":"user","content":"hi"}]}'
```
No API key handy? Scaffold the **agent** template instead (`npx anvil init --template agent`) — it wires up `MockDriver`, which returns canned responses with no network calls or API key required, so you can see the streaming protocol working immediately.
See [Agent Routes](/agents/define-agent) for tool calling, [LLM Client](/agents/llm-client) for the OpenAI and Gemini drivers, and [Multi-Agent Orchestration](/agents/orchestration) for composing agents together.
```bash theme={null}
npx anvil lint
```
```text theme={null}
[anvil] lint: 3 route(s) checked, 0 error(s), 0 warning(s)
```
`anvil lint` checks that every `paramsSchema` key matches the route's dynamic segments, and that every MCP-exposed schema (`paramsSchema`, `bodySchema`, `querySchema`, `outputSchema`) converts losslessly to JSON Schema — a `.transform()` or `.refine()` on an exposed schema is a **build-time error**, not a runtime surprise discovered by a caller. Run it in CI alongside `anvil build`, which also fails hard on structural route conflicts.
## What you just built
A single route file (`server/routes/users/[id]/get.ts`) that is simultaneously a typed REST endpoint, a schema-validated MCP tool, and lint-checked at build time — with no duplicated logic between any of those surfaces. That's the core idea behind Anvil: one file-based route definition, multiple protocols.
## Next steps
The full routing convention: verbs, groups, catch-alls, and the generated route manifest.
`[param]` and `[...param]` segments, Zod validation, and route-conflict rules.
Expose routes and standalone tools as MCP tools — stdio and Streamable HTTP.
Tool calling, streaming, context assembly, and the `LlmClient` driver model.
Every template, flag, and file `anvil init` writes — including merging into an existing project.
Every agent run traced automatically, with a built-in dashboard at `/_anvil`.
# The Anvil Context Object: Request and Response API
Source: https://anvil.thatdevguy.in/routing/context
Reference for the Context class: typed request properties, cached body parsing, response helpers, ctx.state for middleware data, and HttpError.
Every Anvil handler and middleware receives a single `Context` object. It is an immutable view of the incoming request plus a set of response-building helpers — Anvil's replacement for Express's mutation-heavy `(req, res)` pair. Import the type from `anvil`:
```ts theme={null}
import type { Context } from 'anvil-sdk';
export default async function handler(ctx: Context) {
// ...
}
```
## Request properties
The underlying Web-standard `Request` object. Use this when you need access to APIs not exposed directly on `Context`, such as `ctx.req.arrayBuffer()` or `ctx.req.signal`.
```ts theme={null}
export default async function handler(ctx: Context) {
const rawBody = await ctx.req.arrayBuffer();
// ...
}
```
The HTTP method in uppercase, e.g. `"GET"`, `"POST"`, `"DELETE"`. Delegates to `ctx.req.method`.
```ts theme={null}
export default function handler(ctx: Context) {
console.log(ctx.method); // "GET"
}
```
The URL pathname without the query string, e.g. `"/users/42"`. Equivalent to `new URL(ctx.req.url).pathname`.
```ts theme={null}
export default function handler(ctx: Context) {
console.log(ctx.path); // "/users/42"
}
```
The request headers as a Web-standard `Headers` object. Access individual headers with `.get()`.
```ts theme={null}
export default function handler(ctx: Context) {
const contentType = ctx.headers.get('content-type');
const auth = ctx.headers.get('authorization');
}
```
Named path parameters captured from the route pattern. The keys correspond to the dynamic folder names in your `server/routes/` tree (without the brackets).
```ts theme={null}
// Route: server/routes/users/[id]/get.ts → GET /users/:id
export default function handler(ctx: Context) {
console.log(ctx.params.id); // "42" for GET /users/42
}
```
For catch-all routes (`[...param]`), `ctx.params.param` contains the full remaining path joined with `/`:
```ts theme={null}
// Route: server/routes/files/[...path]/get.ts → GET /files/*
export default function handler(ctx: Context) {
console.log(ctx.params.path); // "images/2024/photo.png" for GET /files/images/2024/photo.png
}
```
The URL query string parsed into a plain object. Repeated keys are collected into an array; single-occurrence keys are plain strings.
```ts theme={null}
// GET /search?q=anvil&tag=node&tag=typescript
export default function handler(ctx: Context) {
console.log(ctx.query.q); // "anvil"
console.log(ctx.query.tag); // ["node", "typescript"]
}
```
The result is computed lazily on first access and cached for the lifetime of the request.
A mutable scratch space for passing data from middleware to the handler (or to a later middleware). Anvil does not touch this object — it is entirely yours to use.
```ts theme={null}
// In middleware:
ctx.state.user = await verifyToken(token);
// In the handler:
const user = ctx.state.user as User;
```
See [Scoped Middleware](/routing/middleware) for a full auth-guard example.
## Body parsing
Parse the request body based on the `Content-Type` header:
* `application/json` → parsed with `JSON.parse()`, returns the resulting value
* `application/x-www-form-urlencoded` or `multipart/form-data` → returns a `FormData` object
* Anything else → returns the body as a plain `string`
The result is cached after the first call — calling `ctx.body()` from both a middleware and the handler is safe and reads the stream only once.
```ts theme={null}
export default async function handler(ctx: Context) {
const body = await ctx.body<{ name: string; email: string }>();
// body is typed as { name: string; email: string }
return { received: body };
}
```
If the body cannot be parsed (e.g., malformed JSON), Anvil throws an `HttpError(400, 'Malformed request body')` automatically.
## Response helpers
All response helpers accept an optional `init` parameter of type `ResponseInit` (the same second argument as the Web `Response` constructor) for setting status codes and custom headers.
Serialize `data` to JSON and return a `Response` with `Content-Type: application/json`. Delegates to `Response.json()`.
```ts theme={null}
return ctx.json({ id: '1', name: 'Alice' });
// With a custom status:
return ctx.json({ created: true }, { status: 201 });
```
Return a `Response` with `Content-Type: text/plain; charset=utf-8`.
```ts theme={null}
return ctx.text('Hello, world!');
return ctx.text('Not found', { status: 404 });
```
Return a `Response` with `Content-Type: text/html; charset=utf-8`.
```ts theme={null}
return ctx.html('
Hello
');
```
Return a redirect response. `status` defaults to `302`. Use `301` for a permanent redirect.
```ts theme={null}
return ctx.redirect('/users/profile');
return ctx.redirect('https://example.com', 301);
```
Return a streaming response. Pass a `ReadableStream` as the body — useful for server-sent events, large file transfers, and AI model output.
```ts theme={null}
export default function handler(ctx: Context) {
const stream = new ReadableStream({
start(controller) {
controller.enqueue(new TextEncoder().encode('data: hello\n\n'));
controller.close();
},
});
return ctx.stream(stream, {
headers: { 'content-type': 'text/event-stream' },
});
}
```
## Throwing HTTP errors
Use `HttpError` (imported from `anvil`) to throw an HTTP error from a handler or middleware. Anvil's kernel catches it and converts it to a structured JSON error response automatically.
```ts theme={null}
import { HttpError, type Context } from 'anvil-sdk';
export default async function handler(ctx: Context) {
const token = ctx.headers.get('authorization');
if (!token) throw new HttpError(401, 'Missing Authorization header');
const body = await ctx.body<{ name?: string }>();
if (!body || typeof body !== 'object' || !('name' in body)) {
throw new HttpError(400, 'Expected a JSON body with a "name" field');
}
return { ok: true };
}
```
The JSON response looks like:
```json theme={null}
{
"error": "Missing Authorization header",
"status": 401
}
```
The full constructor signature is:
```ts theme={null}
new HttpError(status: number, message?: string, opts?: {
details?: unknown; // extra data included in the response body (4xx only by default)
expose?: boolean; // override the default expose logic (true = send message/details to client)
cause?: unknown; // the underlying error, forwarded as Error.cause for stack traces
})
```
The HTTP status code. 4xx errors expose their message to the client by default; 5xx errors do not (the client sees the generic status message instead).
The error message. For 4xx errors, this is sent to the client. For 5xx errors, it is hidden in production and only visible in dev mode (`NODE_ENV !== 'production'`).
Optional additional data included in the `details` field of the JSON response. Only sent to the client when `expose` is true (the default for 4xx errors).
Override the default expose behaviour. Set to `true` to force the message and details onto a 5xx response in development, or `false` to hide them on a 4xx.
The underlying error that caused this `HttpError`. Forwarded as the standard `Error.cause` property — visible in stack traces but never sent to the client.
## Full handler example
Here is a handler that demonstrates multiple `Context` features together:
```ts server/routes/users/[id]/get.ts theme={null}
import { HttpError, type Context } from 'anvil-sdk';
import { z } from 'zod';
export const meta = { mcp: { expose: true, description: 'Fetch a user by ID' } };
export const paramsSchema = z.object({ id: z.string() });
const USERS = [
{ id: '1', name: 'Alice', role: 'admin' },
{ id: '2', name: 'Bob', role: 'user' },
];
export default function handler(ctx: Context) {
// Read a dynamic route param
const { id } = ctx.params;
// Read an optional query parameter
const format = ctx.query.format; // e.g. ?format=verbose
// Read a request header
const requestId = ctx.headers.get('x-request-id') ?? 'unknown';
// Read authenticated user from middleware-set state
const viewer = ctx.state.user as { role: string } | undefined;
const user = USERS.find((u) => u.id === id);
if (!user) throw new HttpError(404, `No user with id "${id}"`);
// Restrict sensitive fields based on the viewer's role
if (viewer?.role !== 'admin' && user.role === 'admin') {
throw new HttpError(403, 'Forbidden');
}
return ctx.json(
format === 'verbose' ? { user, requestId } : user,
);
}
```
# Dynamic Routes: URL Params and Catch-All Segments in Anvil
Source: https://anvil.thatdevguy.in/routing/dynamic-routes
Use [param] folders for named path parameters and [...param] for catch-all segments. Validate params at compile time with a Zod paramsSchema.
Dynamic routes let you capture variable segments of a URL path without defining a separate file for every possible value. Anvil uses a folder-naming convention borrowed from Next.js: square brackets for a named parameter, three dots inside brackets for a catch-all.
## Named parameters: `[param]`
Wrap a folder name in square brackets to make it a dynamic segment. The captured value is available on `ctx.params` under the same name as the folder.
```text theme={null}
server/routes/
users/
[id]/
get.ts → GET /users/:id
ctx.params.id = "42" for GET /users/42
```
Read the captured value in your handler:
```ts server/routes/users/[id]/get.ts theme={null}
import { HttpError, type Context } from 'anvil-sdk';
const USERS = [
{ id: '1', name: 'Alice' },
{ id: '2', name: 'Bob' },
];
export default function handler(ctx: Context) {
const user = USERS.find((u) => u.id === ctx.params.id);
if (!user) throw new HttpError(404, `No user with id "${ctx.params.id}"`);
return user;
}
```
You can nest multiple dynamic segments. Each one contributes its own key to `ctx.params`:
```text theme={null}
server/routes/
orgs/
[orgId]/
repos/
[repoId]/
get.ts → GET /orgs/:orgId/repos/:repoId
```
```ts server/routes/orgs/[orgId]/repos/[repoId]/get.ts theme={null}
import type { Context } from 'anvil-sdk';
export default function handler(ctx: Context) {
const { orgId, repoId } = ctx.params;
return { orgId, repoId };
}
```
## Catch-all segments: `[...param]`
Prefix with `...` to capture the rest of the path as a single string. Everything after the catch-all folder's parent is joined with `/` and stored in `ctx.params` under the param name.
```text theme={null}
server/routes/
files/
[...path]/
get.ts → GET /files/*
ctx.params.path = "a/b/c" for GET /files/a/b/c
```
```ts server/routes/files/[...path]/get.ts theme={null}
import type { Context } from 'anvil-sdk';
export default function handler(ctx: Context) {
// ctx.params.path is the full remaining path string
const filePath = ctx.params.path; // e.g. "images/2024/photo.png"
return { requestedPath: filePath };
}
```
A catch-all directory cannot contain subdirectories. The scanner enforces this at build time: placing a folder inside `[...path]/` throws a `RouteConflictError`.
## Router precedence
When multiple route patterns could match the same URL, Anvil resolves the conflict deterministically — insertion order never matters:
1. **Static segments** win over dynamic ones (`/users/me` beats `/users/[id]` for the path `/users/me`)
2. **Named parameters** win over catch-alls
3. **Backtracking** ensures `/a/[b]/c` still matches when `/a/x` exists as a static sibling
## Validating params with `paramsSchema`
Export a `paramsSchema` Zod object from any route file to enable compile-time validation and MCP tool generation. `anvil lint` checks that every key in the schema corresponds to a real folder param in the path.
```ts server/routes/users/[id]/get.ts theme={null}
import { HttpError, type Context } from 'anvil-sdk';
import { z } from 'zod';
// Tells `anvil lint` to verify `id` matches the [id] folder,
// and tells `anvil mcp` how to describe the tool's input.
export const meta = { mcp: { expose: true, description: 'Fetch a user by ID' } };
export const paramsSchema = z.object({ id: z.string() });
export default function handler(ctx: Context) {
// ctx.params.id is already validated — safe to use directly
const user = USERS.find((u) => u.id === ctx.params.id);
if (!user) throw new HttpError(404, `No user with id "${ctx.params.id}"`);
return user;
}
```
`paramsSchema` is a compile-time and MCP-exposure hint. At runtime, `ctx.params` values are always plain strings — they are not coerced or parsed by the schema. Use the schema keys as a contract that `anvil lint` enforces.
## Naming rules
Anvil enforces several rules to prevent subtle bugs and cross-platform inconsistencies:
The same param name cannot appear twice in one route path. `/[id]/items/[id]/get.ts` throws a `RouteConflictError` at scan time.
Two routes cannot use different param names at the same depth. `/users/[id]/get.ts` and `/users/[slug]/post.ts` conflict — use the same name (`[id]`) across all files in that folder.
`anvil lint` rejects `Users/` and `users/` in the same parent because Windows treats them as the same directory while Linux does not.
Param names must match `/^[A-Za-z_$][A-Za-z0-9_$]*$/`. `[123]` and `[my-param]` are rejected at scan time.
# File-Based Routing: Map Files to HTTP Endpoints in Anvil
Source: https://anvil.thatdevguy.in/routing/file-based-routing
How Anvil maps server/routes/ to HTTP endpoints: one file per HTTP verb, auto-serialization, route groups, and the build-time static manifest.
Anvil turns your directory structure into an HTTP API. Place a file named after an HTTP verb inside `server/routes/` and Anvil registers the corresponding route — no import lists, no decorator registration, no central router config. The filesystem is your router.
## The convention
Anvil recognizes these file names as HTTP verb handlers:
| File name | HTTP method |
| ------------ | --------------------------------- |
| `get.ts` | `GET` |
| `post.ts` | `POST` |
| `put.ts` | `PUT` |
| `patch.ts` | `PATCH` |
| `delete.ts` | `DELETE` |
| `head.ts` | `HEAD` |
| `options.ts` | `OPTIONS` |
| `agent.ts` | `POST` (streaming agent endpoint) |
Every file must export a default function (the handler). Any other named export is metadata — `meta`, `paramsSchema`, etc.
## Full routing tree example
```text theme={null}
server/routes/
get.ts → GET /
users/
_middleware.ts → scoped middleware for /users/**
get.ts → GET /users
post.ts → POST /users
[id]/
get.ts → GET /users/:id
files/
[...path]/
get.ts → GET /files/* (catch-all)
(admin)/ → route group — not part of the URL
dashboard/
get.ts → GET /dashboard
```
## Writing a handler
A handler is any function that accepts a `Context` and returns a value. Import `Context` from `anvil` for full TypeScript types:
```ts server/routes/get.ts theme={null}
import type { Context } from 'anvil-sdk';
export default function handler(ctx: Context) {
return {
name: 'my-api',
version: '1.0.0',
};
}
```
Handlers can be synchronous or async:
```ts server/routes/users/post.ts theme={null}
import { HttpError, type Context } from 'anvil-sdk';
export default async function handler(ctx: Context) {
const body = await ctx.body<{ name?: string }>();
if (typeof body !== 'object' || body === null || !body.name) {
throw new HttpError(400, 'Expected JSON body with a "name" field');
}
return ctx.json({ created: { id: Date.now().toString(36), name: body.name } }, { status: 201 });
}
```
## Auto-serialization
You don't need to call `ctx.json()` for every response. Anvil normalizes whatever your handler returns:
| Return value | Response |
| --------------------- | ------------------------------- |
| Plain object or array | `200 application/json` |
| `string` | `200 text/plain; charset=utf-8` |
| `null` or `undefined` | `204 No Content` |
| `Response` instance | Passed through unchanged |
This means the simplest possible handler is just a function that returns data:
```ts server/routes/users/get.ts theme={null}
export default function handler() {
return { users: [{ id: '1', name: 'Alice' }] };
}
```
## Route groups
Wrap a folder name in parentheses to create a **route group**. The group name is stripped from the URL — it exists only to organize your files or scope middleware.
```text theme={null}
server/routes/
(admin)/
dashboard/
get.ts → GET /dashboard (not GET /(admin)/dashboard)
settings/
get.ts → GET /settings
```
Route groups are useful for:
* Applying a `_middleware.ts` to a subset of routes without affecting their URLs
* Organizing large route trees into logical sections (e.g., `(public)/`, `(authenticated)/`)
## The `_middleware.ts` convention
A file named `_middleware.ts` in any route folder runs for all routes under that folder. Files starting with `_` are never treated as route handlers — they are reserved for middleware, context, and other conventions.
See [Scoped Middleware](/routing/middleware) for the full guide.
## The route manifest
Running `anvil build` generates a static route manifest at `.gen/routes.ts`. This file statically imports every handler and middleware, so your production bundle has no runtime filesystem scanning — startup is instant regardless of how many routes you have.
```ts .gen/routes.ts (generated — do not edit) theme={null}
// Generated by `anvil build`. Do not edit.
import type { Manifest, Middleware, RouteMeta } from 'anvil-sdk';
import * as r0 from '../server/routes/get.js';
import * as r1 from '../server/routes/users/get.js';
// ...
export const manifest: Manifest = {
routes: [
{
method: 'GET',
pattern: '/',
segments: [],
handler: r0.default,
middleware: [],
meta: metaOf(r0),
file: '../server/routes/get.js',
},
// ...
],
};
```
Never edit `.gen/routes.ts` directly. It is regenerated every time you run `anvil build` or `anvil dev` detects a change.
## Compile-time validation
Run `anvil lint` before deploying to catch routing problems at build time rather than at runtime:
* **Param schema keys** must match the actual folder params — if your folder is `[id]/` but your `paramsSchema` has `{ userId: z.string() }`, lint fails.
* **MCP-exposed schemas** must convert losslessly to JSON Schema (e.g., no `z.transform()` without a target type).
* **Case-insensitive collisions** — `Users/` and `users/` are the same on Windows but different on Linux; lint catches this so your app behaves the same on both.
```bash theme={null}
anvil lint # warns on issues
anvil lint --strict # exits non-zero on warnings too
```
# Middleware in Anvil JS: Scoped _middleware.ts Files
Source: https://anvil.thatdevguy.in/routing/middleware
Use _middleware.ts files to run middleware for all routes under a folder. Compose multiple functions, pass data via ctx.state, and add CORS with cors().
Middleware in Anvil is code that runs before (and optionally after) your route handler. You place it in a `_middleware.ts` file next to the routes it should affect — no central `app.use()` call, no manual import lists. Anvil wires it up automatically based on location.
## The `_middleware.ts` convention
A file named `_middleware.ts` in any `server/routes/` folder runs for every route under that folder. Root-level middleware (`server/routes/_middleware.ts`) runs for every request in the application, including unmatched paths (so static files and the observability dashboard can be served from there).
```text theme={null}
server/routes/
_middleware.ts ← runs for all routes + unmatched paths
get.ts
users/
_middleware.ts ← runs only for /users/**
get.ts
post.ts
[id]/
get.ts
```
### Root middleware example
The root `_middleware.ts` is a great place for logging, static file serving, and the built-in observability dashboard:
```ts server/routes/_middleware.ts theme={null}
import { serveStatic, type Middleware } from 'anvil-sdk';
const logger: Middleware = async (ctx, next) => {
const start = Date.now();
const res = await next();
console.log(`${ctx.method} ${ctx.path} → ${res.status} (${Date.now() - start}ms)`);
return res;
};
// Root middleware also runs for unmatched paths, so static files
// under public/ are served without needing a dedicated route.
export default [logger, serveStatic({ dir: 'public' })];
```
Exporting an **array** of middleware functions is supported. Anvil flattens single functions and arrays — you can mix and match in the same file.
## The onion (compose) model
Middleware wraps the handler in an "onion" pattern. Root middleware is the outermost layer; the middleware closest to the handler file runs innermost. Each function receives `ctx` and a `next` function to call the next layer.
```text theme={null}
Request
└─ root _middleware (outermost)
└─ /users _middleware
└─ route handler (innermost)
← response flows back out through each layer
```
Calling `next()` passes control inward. You can run code before the handler (pre-processing) and after it (post-processing):
```ts server/routes/users/_middleware.ts theme={null}
import type { Middleware } from 'anvil-sdk';
const scope: Middleware = async (ctx, next) => {
// Pre-processing: runs before the handler
console.log(`Users scope: ${ctx.method} ${ctx.path}`);
const res = await next(); // ← call the next layer
// Post-processing: runs after the handler returns
res.headers.set('x-scope', 'users');
return res;
};
export default scope;
```
## Composing multiple middleware functions
To apply several middleware functions to the same scope, export them as an **array** from `_middleware.ts`. Anvil flattens the array and runs each function in order:
```ts server/routes/users/_middleware.ts theme={null}
import type { Middleware } from 'anvil-sdk';
const timer: Middleware = async (ctx, next) => {
const start = Date.now();
const res = await next();
res.headers.set('x-response-time', `${Date.now() - start}ms`);
return res;
};
const scopeHeader: Middleware = async (ctx, next) => {
const res = await next();
res.headers.set('x-scope', 'users');
return res;
};
// Export an array — Anvil applies them outermost-first (timer wraps scopeHeader)
export default [timer, scopeHeader];
```
The `compose()` helper (exported from `anvil`) is used internally by the framework to build the full middleware chain at request time. You can also call it directly when you need to programmatically assemble a handler pipeline — for example, in tests or when creating reusable middleware factories:
```ts theme={null}
import { compose } from 'anvil-sdk';
// compose returns a (ctx: Context) => Promise — a fully-wired handler,
// not a Middleware. Use it when you want to call a pipeline directly, not as a
// _middleware.ts export.
const runChain = compose([timer, scopeHeader], myHandler);
const response = await runChain(ctx);
```
## Passing data with `ctx.state`
`ctx.state` is a mutable `Record` on every request context. Middleware can write values to it; the downstream handler (or the next middleware) reads them. This replaces the pattern of mutating `req` in Express.
### Auth guard example
```ts server/routes/(authenticated)/_middleware.ts theme={null}
import { HttpError, type Middleware } from 'anvil-sdk';
const authGuard: Middleware = async (ctx, next) => {
const token = ctx.headers.get('authorization')?.replace('Bearer ', '');
if (!token) {
throw new HttpError(401, 'Missing Authorization header');
}
// Validate the token — replace this with your real auth logic
const user = await verifyToken(token);
if (!user) {
throw new HttpError(403, 'Invalid or expired token');
}
// Store the authenticated user so handlers can access it
ctx.state.user = user;
return next();
};
export default authGuard;
```
Read `ctx.state.user` in any handler under the `(authenticated)/` route group:
```ts server/routes/(authenticated)/profile/get.ts theme={null}
import type { Context } from 'anvil-sdk';
export default function handler(ctx: Context) {
// ctx.state.user was set by the auth middleware
const user = ctx.state.user as { id: string; name: string };
return { profile: user };
}
```
## Short-circuiting
A middleware can return a response without calling `next()` to stop the chain entirely. This is how auth guards, rate limiters, and maintenance-mode middleware work:
```ts server/routes/_middleware.ts theme={null}
import { type Middleware } from 'anvil-sdk';
const maintenanceMode: Middleware = async (ctx, next) => {
if (process.env.MAINTENANCE === 'true') {
// Returns immediately — the handler is never called
return new Response(
JSON.stringify({ error: 'Service temporarily unavailable', status: 503 }),
{ status: 503, headers: { 'content-type': 'application/json' } },
);
}
return next();
};
export default maintenanceMode;
```
## Adding CORS
Anvil ships a `cors()` middleware helper with the same options as the popular `cors` npm package. Add it to your root `_middleware.ts`:
```ts server/routes/_middleware.ts theme={null}
import { cors, type Middleware } from 'anvil-sdk';
const logger: Middleware = async (ctx, next) => {
const start = Date.now();
const res = await next();
console.log(`${ctx.method} ${ctx.path} → ${res.status} (${Date.now() - start}ms)`);
return res;
};
export default [
cors({
origin: ['https://my-frontend.com', 'http://localhost:5173'],
credentials: true,
}),
logger,
];
```
Available `cors()` options:
| Option | Type | Default | Description | | |
| ---------------- | ---------- | ------------------------ | --------------------------------------------- | ----- | ------------------------------------------------------------------------------- |
| `origin` | \`'\*' | string\[] | (origin: string) => boolean\` | `'*'` | Allowed origins. Use an array for an allowlist or a predicate for custom logic. |
| `methods` | `string[]` | All standard methods | Allowed HTTP methods in preflight responses. | | |
| `allowedHeaders` | `string[]` | Reflects request headers | Allowed request headers. | | |
| `exposedHeaders` | `string[]` | — | Headers the browser is allowed to read. | | |
| `credentials` | `boolean` | `false` | Set `Access-Control-Allow-Credentials: true`. | | |
| `maxAge` | `number` | — | Preflight cache duration in seconds. | | |
# Durable Execution: Checkpoint and Resume Agent Runs
Source: https://anvil.thatdevguy.in/safety/durable-execution
Checkpoint every agent iteration so a crash or redeploy resumes from the last completed step. Completed side-effect tools are never re-executed on resume.
Long-running agent loops are vulnerable to server restarts, client disconnects, and unexpected errors. Anvil's durable execution system saves a checkpoint after every completed iteration — including the accumulated message history, token usage, and the results of any side-effect tool calls. If the process crashes or is redeployed, the run resumes from the last checkpoint without restarting from scratch or re-executing tools that have already run.
## How checkpointing works
The `Checkpointer` class persists `AgentCheckpoint` objects into any `StateStore` (in-memory for tests, SQLite for production). The agent runtime writes a checkpoint:
1. **After each completed iteration** — capturing the current message history and accumulated usage.
2. **When the run suspends for HITL approval** — storing the pending payload and the results of all tools that already ran.
On resume, `resumeAgent` loads the checkpoint, restores the message history and counters, and re-enters the loop at the point it left off. Tool calls recorded in the `approvals` map are replayed from their stored results rather than re-executed.
## Checkpointer API
```ts theme={null}
import { Checkpointer } from 'anvil-sdk/agent';
import { SqliteStateStore } from 'anvil-sdk/store';
const stateStore = await SqliteStateStore.open('.anvil/state.db');
const checkpointer = new Checkpointer(stateStore, 'run-abc-123');
```
The key-value store where checkpoints are persisted. Use `SqliteStateStore` in production and `MemoryStateStore` in tests.
A stable identifier for this run. Use a value you can look up later — a request ID, a database primary key, or a UUID generated at request time.
### Methods
Persist the current run state. Called automatically by the Anvil runtime when you pass `checkpoint` to `runAgent` or `streamAgent`; call it manually only if you are driving the loop yourself.
Retrieve the most recently saved checkpoint. Returns `undefined` if no checkpoint exists for this `runId`.
Remove the checkpoint after a run completes successfully. Call this in your cleanup logic to avoid unbounded growth of the checkpoint store.
## AgentCheckpoint fields
```ts theme={null}
interface AgentCheckpoint {
version: 1;
runId: string;
status: 'running' | 'suspended' | 'done';
messages: ModelMessage[];
iterations: number;
totalUsage: { inputTokens: number; outputTokens: number };
totalCostUsd: number;
approvals?: Record; // completed tool results keyed by call id
pending?: { callId: string; payload: unknown }; // set when suspended for HITL
finalText?: string;
updatedAt: number; // Unix ms of the last save
}
```
Current lifecycle state. `'suspended'` means the run is waiting for human approval; `'done'` means it completed (successfully or by hitting the iteration cap).
Tool call results already persisted. When the run resumes, any tool call whose `callId` appears in this map is **not** re-executed — its stored output is used directly. This prevents side-effect tools from running twice.
Set when `status` is `'suspended'`. Contains the call ID and the payload the tool passed to `ApprovalRequiredError`, which the approver sees in the HITL queue.
## Marking tools as side effects
Declare `sideEffect: true` on any tool that mutates external state — charges a card, sends an email, creates a database row. Anvil checkpoints after the tool completes and fences its result so a resume can never re-execute it:
```ts theme={null}
const chargeCard: AgentTool = {
name: 'chargeCard',
description: 'Charge a customer payment card.',
sideEffect: true,
execute: async ({ amount, customerId }) => {
const result = await paymentProvider.charge(customerId, amount);
return result.transactionId;
},
};
```
## Running an agent with a checkpoint
Pass `checkpoint` to `runAgent` or `streamAgent`. Anvil instantiates a `Checkpointer` internally and writes checkpoints throughout the run:
```ts theme={null}
import { runAgent } from 'anvil-sdk/agent';
import { SqliteStateStore } from 'anvil-sdk/store';
const stateStore = await SqliteStateStore.open('.anvil/state.db');
const runId = crypto.randomUUID();
const result = await runAgent({
client,
messages,
tools: [lookupOrder, chargeCard],
checkpoint: { store: stateStore, runId },
});
```
## Resuming after a crash
If the process was interrupted, call `resumeAgent` with the same `runId`. It loads the checkpoint, restores message history and usage counters, and re-enters the loop from the last saved state:
```ts theme={null}
import { resumeAgent } from 'anvil-sdk/agent';
const result = await resumeAgent({
client,
tools: [lookupOrder, chargeCard],
checkpoint: { store: stateStore, runId: 'run-abc-123' },
// Pass approval if resuming a HITL-suspended run:
// approval: { approved: true, reviewer: 'alice@example.com' },
});
```
`resumeAgent` returns immediately if the checkpoint has `status: 'done'` — handy for idempotent retry logic.
## Listing all persisted runs
Use `listRuns` to retrieve every run ID that has an active checkpoint. This is useful for building an admin queue of in-progress or suspended runs:
```ts theme={null}
import { listRuns } from 'anvil-sdk/agent';
const runIds = await listRuns(stateStore);
// ['run-abc-123', 'run-def-456', ...]
```
`listRuns` returns only run IDs, not the full checkpoints. Load a specific checkpoint with `new Checkpointer(store, runId).load()` to inspect its state.
## The state store
Durable execution uses `StateStore` — a simple key-value interface separate from `TraceStore`. The production default is `SqliteStateStore`:
```ts theme={null}
import { SqliteStateStore, MemoryStateStore } from 'anvil-sdk/store';
// Production
const store = await SqliteStateStore.open('.anvil/state.db');
// Tests
const store = new MemoryStateStore();
```
You can share one `SqliteStateStore` across multiple agent routes, since each run uses a unique `runId`-namespaced key.
Pair durable execution with [Human-in-the-Loop approvals](/safety/human-in-the-loop). When a tool suspends the run, the `suspended` checkpoint holds the full state — the HITL queue in `/_anvil` shows it, and `resumeAgent` resumes it with the approval decision.
# Guardrails: Content Filtering, PII Redaction and Tool Policy
Source: https://anvil.thatdevguy.in/safety/guardrails
Add centrally-enforced content filters, PII redaction, tool allowlists, and prompt-injection defences to every agent route in Anvil JS.
Guardrails let you enforce content policies and tool access rules at the agent level, not inside individual handlers. Pass a `guardrails` array to `defineAgent` and every request through that route is automatically screened — user input before it reaches the model, model output before it reaches the client, and tool calls before they execute. Anvil ships four built-in guardrails covering the most common needs; you can also implement the `Guardrail` interface directly for custom logic.
## The Guardrail interface
```ts theme={null}
interface Guardrail {
name: string;
/** Transform text (return new text) or throw GuardrailError to block. */
onText?(text: string, ctx: TextContext): string | void;
/** Decide whether a tool may run: allow / deny / approve. */
onToolCall?(ctx: ToolCallContext): ToolDecision | void;
}
```
* **`onText`** receives the raw text and a `TextContext` with `role: 'input' | 'output'`. Return a (possibly modified) string to continue, or throw `GuardrailError` to block the run entirely.
* **`onToolCall`** receives the tool name, its input, and a `tainted` flag indicating whether untrusted (tool or retrieved) content is already in the conversation. Return a `ToolDecision` to explicitly allow, deny, or route the call to human approval.
### ToolDecision
```ts theme={null}
type ToolDecision = { action: 'allow' | 'deny' | 'approve'; reason?: string };
```
The most restrictive decision across all guardrails in the chain wins: `deny` beats `approve`, `approve` beats `allow`.
### GuardrailError
Throw `GuardrailError` from `onText` to immediately stop the run:
```ts theme={null}
import { GuardrailError } from 'anvil-sdk/agent';
throw new GuardrailError('my-guardrail', 'Input contains disallowed content');
```
The `guardrail` property on the error holds the name string so downstream error handlers can identify which policy fired.
## Built-in guardrails
### contentFilter
Block or redact text matching custom regular expression patterns.
```ts theme={null}
import { contentFilter } from 'anvil-sdk/agent';
contentFilter({
deny: [/\bpassword\b/i, /\bsecret_key\b/i],
mode: 'block', // 'block' (default) | 'redact'
replacement: '[REMOVED]', // used only when mode: 'redact'
applyTo: ['input'], // 'input' | 'output' | both (default)
})
```
Patterns to screen. Each pattern is tested against the incoming text; the first match triggers the configured action.
What to do when a pattern matches. `'block'` throws `GuardrailError`; `'redact'` replaces every match with `replacement`.
Substitution string used in `'redact'` mode.
Which text direction to inspect. Restrict to `['input']` to screen only user messages, or `['output']` for model responses.
***
### redactPII
Automatically redact common PII patterns from input and/or output without needing to supply regexes:
```ts theme={null}
import { redactPII } from 'anvil-sdk/agent';
redactPII()
// or restrict the direction:
redactPII({ applyTo: ['input'] })
```
The following patterns are redacted:
| PII type | Replacement token |
| ----------------------------------------- | ------------------ |
| Email address | `[REDACTED_EMAIL]` |
| Payment card number (13–16 digits) | `[REDACTED_CARD]` |
| US Social Security Number (`NNN-NN-NNNN`) | `[REDACTED_SSN]` |
| US phone number | `[REDACTED_PHONE]` |
Which text direction to redact. Defaults to both.
***
### toolPolicy
Enforce an explicit allowlist, denylist, or approval requirement on tool names:
```ts theme={null}
import { toolPolicy } from 'anvil-sdk/agent';
toolPolicy({
allow: ['lookupOrder', 'getProductInfo'], // only these tools may run
deny: ['deleteAccount'], // always blocked
requireApproval: ['issueRefund'], // pauses run → HITL queue
})
```
When set, only tools whose names appear in this list may run. Any other tool call returns a `deny` decision.
Tools that are unconditionally blocked regardless of `allow`.
Tools that require human approval before executing. Matching tool calls receive an `approve` decision, suspending the run and adding it to the [HITL queue](/safety/human-in-the-loop).
***
### injectionGuard
Defend against prompt-injection attacks. Once untrusted content (from a tool result or a retrieval) has entered the conversation, the model may have been instructed to make malicious tool calls. `injectionGuard` detects this "tainted" context and gates subsequent tool calls:
```ts theme={null}
import { injectionGuard } from 'anvil-sdk/agent';
injectionGuard({
mode: 'approve', // 'block' | 'approve' (default) | 'allow'
allowlist: ['getWeather', 'lookupFAQ'], // always allowed even in tainted context
})
```
Action to take when a tool call arrives in a tainted context:
* **`'block'`** — denies the call outright with a `deny` decision.
* **`'approve'`** — routes the call to human approval via the HITL queue.
* **`'allow'`** — disables the guard (useful for temporarily turning it off without removing the guardrail).
Tool names that are always permitted even when context is tainted. Use this for idempotent, read-only lookups that carry no injection risk.
## Wiring multiple guardrails into defineAgent
```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import {
contentFilter,
redactPII,
toolPolicy,
injectionGuard,
} from 'anvil-sdk/agent';
import { openai } from 'anvil-sdk/llm';
export const post = defineAgent({
client: openai({ model: 'gpt-4o' }),
guardrails: [
// 1. Strip PII from both input and model output
redactPII(),
// 2. Block input that tries to override the system prompt
contentFilter({
deny: [/ignore previous instructions/i, /you are now/i],
mode: 'block',
applyTo: ['input'],
}),
// 3. Restrict which tools can run and require approval for refunds
toolPolicy({
allow: ['lookupOrder', 'getProductInfo', 'issueRefund'],
requireApproval: ['issueRefund'],
}),
// 4. Gate tool calls that arise from tainted context
injectionGuard({
mode: 'approve',
allowlist: ['lookupOrder'],
}),
],
tools: [lookupOrder, getProductInfo, issueRefund],
});
```
Guardrails run in the order they appear in the array. For `onText`, each guardrail's return value is passed as input to the next — they compose as a transform pipeline. For `onToolCall`, the most restrictive decision across all guardrails wins (deny beats approve, approve beats allow).
## Implementing a custom guardrail
```ts theme={null}
import type { Guardrail } from 'anvil-sdk/agent';
const lengthGuard: Guardrail = {
name: 'length-guard',
onText(text, ctx) {
if (ctx.role === 'input' && text.length > 10_000) {
throw new GuardrailError('length-guard', 'Input exceeds 10,000 characters');
}
},
onToolCall({ name }) {
if (name === 'runArbitraryCode') {
return { action: 'deny', reason: 'code execution not permitted' };
}
},
};
```
Return `void` or `undefined` from `onText` to pass the text through unchanged. Return `void` or `undefined` from `onToolCall` to abstain, letting other guardrails in the chain decide.
# Human-in-the-Loop: Pause and Approve Agent Actions
Source: https://anvil.thatdevguy.in/safety/human-in-the-loop
Suspend an agent run mid-execution, persist its state, and resume it after a human reviews and approves or rejects a pending action.
Human-in-the-loop (HITL) lets you pause an agent run at a critical decision point — before issuing a refund, sending an email, or executing a destructive operation — persist the full run state, and resume execution only after a human reviewer approves. The run is durably checkpointed in this suspended state, so the review can happen seconds or days later without losing any context.
## How the HITL flow works
A tool calls `meta.requestApproval(payload)` (or throws `ApprovalRequiredError` directly), passing a payload that describes what needs approval.
The agent runtime catches the error, saves an `AgentCheckpoint` with `status: 'suspended'` and the `pending` payload, and emits a `suspended` event to the response stream.
The `/_anvil` dashboard shows the pending run in the HITL queue with its payload. The reviewer approves or rejects from the UI — or your application calls `resumeAgent` programmatically.
`resumeAgent` loads the checkpoint, injects the approval decision as the tool result, and continues the agent loop from where it left off. The suspended tool is never re-executed.
## Triggering HITL from a tool
Use `meta.requestApproval` inside a tool's `execute` function. It is a typed shorthand that throws `ApprovalRequiredError` with the supplied payload:
```ts theme={null}
import type { AgentTool } from 'anvil-sdk/agent';
const issueRefund: AgentTool = {
name: 'issueRefund',
description: 'Issue a refund for an order.',
sideEffect: true,
execute: async ({ orderId, amount }, meta) => {
// Pause and ask for human approval before touching money.
meta.requestApproval({ action: 'refund', orderId, amount });
// Code below never runs until the run is resumed with an approval.
return await refundProvider.issue(orderId, amount);
},
};
```
Or throw `ApprovalRequiredError` directly if you need more control:
```ts theme={null}
import { ApprovalRequiredError } from 'anvil-sdk/agent';
throw new ApprovalRequiredError(meta.callId, {
action: 'refund',
orderId,
amount,
requestedBy: ctx.user.email,
});
```
### ApprovalRequiredError fields
The tool call ID that triggered the suspension. Anvil uses this to inject the approval result back into the message history when the run resumes.
Arbitrary data describing the pending action. This is what appears in the `/_anvil` dashboard and what your approval handler receives to make its decision. Keep it human-readable — it is the reviewer's primary signal.
## Wiring the checkpoint store
HITL requires durable checkpointing. Pass a `checkpoint` option to `runAgent` or `streamAgent` so the suspended state is persisted:
```ts theme={null}
import { runAgent } from 'anvil-sdk/agent';
import { SqliteStateStore } from 'anvil-sdk/store';
const stateStore = await SqliteStateStore.open('.anvil/state.db');
const runId = `refund:${orderId}:${Date.now()}`;
const result = await runAgent({
client,
messages,
tools: [lookupOrder, issueRefund],
checkpoint: { store: stateStore, runId },
});
if (result.suspended) {
console.log('Run suspended. Waiting for approval:', result.suspended.callId);
// The /_anvil dashboard now shows this run in the approval queue.
}
```
## Resuming with an approval decision
Call `resumeAgent` with the `runId` and the reviewer's `approval` value. The approval is injected as the result of the suspended tool call, and the agent loop continues:
```ts theme={null}
import { resumeAgent } from 'anvil-sdk/agent';
// Called by your approval webhook or dashboard action handler:
const result = await resumeAgent({
client,
tools: [lookupOrder, issueRefund],
checkpoint: { store: stateStore, runId: 'refund:order-99:1718000000' },
approval: {
approved: true,
reviewer: 'alice@example.com',
note: 'Verified with customer. Proceed.',
},
});
console.log('Run completed:', result.text);
```
If `approval` is `undefined` or carries a rejection value, the tool should inspect it in its `execute` body and handle accordingly — for example by returning an error string instead of calling the payment provider.
## resumeAgent options
`resumeAgent` accepts all the same options as `runAgent`, minus `messages`, `approvals`, and `resumeState` (which come from the checkpoint), plus two additional fields:
The same `StateStore` used when the run was originally started.
The stable run identifier used to locate the saved checkpoint.
The human reviewer's decision. Injected as the result of the tool call that was pending when the run suspended. The receiving tool can inspect this value in its `execute` body on the next invocation.
## Using toolPolicy for zero-code HITL
You can require approval for any tool without modifying the tool's `execute` function. Add `toolPolicy` to `guardrails` with the tool name in `requireApproval`:
```ts theme={null}
import { defineAgent, toolPolicy } from 'anvil-sdk/agent';
export const post = defineAgent({
client,
guardrails: [
toolPolicy({
requireApproval: ['issueRefund', 'deleteAccount'],
}),
],
tools: [lookupOrder, issueRefund, deleteAccount],
});
```
When the model calls `issueRefund` or `deleteAccount`, the guardrail returns an `approve` decision before the tool's `execute` function runs. The runtime suspends the run exactly as if the tool had thrown `ApprovalRequiredError`. This is useful for adding approval requirements to third-party tools you cannot modify.
## The HITL approval queue in /\_anvil
All runs with `status: 'suspended'` appear in the `/_anvil` dashboard approval queue. Each entry shows:
* The `runId` and the trace name
* The `pending.payload` — the description of the action needing approval
* The accumulated cost and token usage at suspension time
* Controls to approve or reject, which call `resumeAgent` on your behalf
The dashboard HITL queue reads suspended checkpoints from the `StateStore`. Make sure your `dashboardMiddleware` and your agent routes share the same SQLite file (`.anvil/state.db`) so the queue reflects live data.
## Handling rejection
When a reviewer rejects a pending action, call `resumeAgent` and pass a rejection value as `approval`. On resume, the tool's `execute` function runs again from the top with the suspended call fenced — the agent loop supplies the approval value as the tool result directly. Design your tool to check the approval value before performing the side effect:
```ts theme={null}
const issueRefund: AgentTool = {
name: 'issueRefund',
description: 'Issue a refund for an order.',
sideEffect: true,
execute: async ({ orderId, amount }, meta) => {
// First invocation: suspend and wait for a reviewer.
// meta.requestApproval always throws — execution does not continue past this line.
meta.requestApproval({ action: 'refund', orderId, amount });
},
};
```
On resume, pass the reviewer's decision as `approval` to `resumeAgent`. The approval value is injected as the tool result seen by the model. The model then generates a new response based on that result — for example, explaining to the user that the refund was rejected:
```ts theme={null}
// Reviewer rejected — pass a descriptive rejection object.
await resumeAgent({
client,
tools: [lookupOrder, issueRefund],
checkpoint: { store: stateStore, runId },
approval: { approved: false, reason: 'Amount exceeds policy limit.' },
});
```
The model receives `{ approved: false, reason: '...' }` as the tool result and crafts an appropriate response for the user.
`meta.requestApproval()` is typed as `never` — it always throws `ApprovalRequiredError` and never returns. Any code written after it in the same `execute` function body is unreachable and will not run on the initial invocation or on resume.