> ## Documentation Index
> Fetch the complete documentation index at: https://anvil.thatdevguy.in/llms.txt
> Use this file to discover all available pages before exploring further.

# Cost Governor: Cap Per-Request Token and USD Spend

> Cap per-request token usage and USD spend. Choose to block, degrade to a cheaper model, or route to human approval when a budget limit is hit.

Anvil's Cost Governor gates every model call in an agent run against a per-request spending budget. Set a `budget` on `defineAgent` and the runtime automatically tracks cumulative token usage and USD cost, checking the budget before each iteration of the agent loop. When a cap is breached you choose the response: throw an error, switch to a cheaper model, or route the run to human approval.

## Configuring a budget on defineAgent

Pass a `BudgetConfig` to the `budget` option on `defineAgent`. The governor is instantiated per request, so limits apply to a single agent run, not across all users.

```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { openai } from 'anvil-sdk/llm';

export const post = defineAgent({
  client: openai({ model: 'gpt-4o' }),
  budget: {
    maxTokens: 4000,
    maxUsd: 0.10,
    onBreach: 'block',
  },
  // ...
});
```

## BudgetConfig fields

<ParamField body="maxTokens" type="number" optional>
  Hard cap on the total number of tokens (input + output) consumed by this run. The governor checks cumulative usage before each model call; once the count equals or exceeds this value, the configured `onBreach` action fires.
</ParamField>

<ParamField body="maxUsd" type="number" optional>
  Hard cap on the total USD cost for this run. Computed from per-call `costUsd` values reported by the LLM client. Like `maxTokens`, this is checked before each model call.
</ParamField>

<ParamField body="onBreach" default="'block'" type="'block' | 'degrade' | 'approve'">
  Action to take when a cap is hit:

  * **`'block'`** — throws `BudgetExceededError` immediately, stopping the run and returning a 500 to the client.
  * **`'degrade'`** — signals that the run should continue on a cheaper model. The `onBreach` callback on `CostGovernor` receives the breach info so your code can switch models.
  * **`'approve'`** — suspends the run and routes it through the human-in-the-loop approval queue, just like a tool that throws `ApprovalRequiredError`.
</ParamField>

## BudgetExceededError

When `onBreach` is `'block'`, the runtime throws a `BudgetExceededError`. Catch it in error-handling middleware to return a structured response:

```ts theme={null}
import { BudgetExceededError } from 'anvil-sdk/trace';

// In a global error handler or try/catch around the agent run:
try {
  // ... agent logic
} catch (err) {
  if (err instanceof BudgetExceededError) {
    console.warn(
      `Budget exceeded: $${err.spentUsd.toFixed(4)} spent, ` +
      `${err.spentTokens} tokens used. ` +
      `Limit: maxUsd=${err.limit.maxUsd}, maxTokens=${err.limit.maxTokens}`
    );
    return Response.json({ error: 'Budget exceeded' }, { status: 429 });
  }
  throw err;
}
```

`BudgetExceededError` exposes three properties:

<ResponseField name="spentUsd" type="number">
  Total USD cost accumulated by this run at the time of the breach.
</ResponseField>

<ResponseField name="spentTokens" type="number">
  Total tokens (input + output) consumed by this run at the time of the breach.
</ResponseField>

<ResponseField name="limit" type="BudgetConfig">
  The `BudgetConfig` that was in effect — useful for building informative error messages.
</ResponseField>

## Using CostGovernor directly

`defineAgent` wires `CostGovernor` for you when you pass a `budget`. For custom agent loops built with `runAgent` or `streamAgent`, construct and pass the governor explicitly:

```ts theme={null}
import { runAgent } from 'anvil-sdk/agent';
import { CostGovernor } from 'anvil-sdk/trace';

const governor = new CostGovernor(
  { maxUsd: 0.05, maxTokens: 2000, onBreach: 'block' },
  (info) => {
    // Called for 'degrade' and 'approve' — not for 'block'.
    console.log(`Budget breach (${info.action}): $${info.spentUsd.toFixed(4)}`);
  },
);

const result = await runAgent({
  client,
  messages,
  governor,
  // ...
});
```

### CostGovernor methods

<ParamField body="governor.record(usage, costUsd?)" type="method">
  Accumulate usage from a completed model call. Called automatically by the Anvil agent runtime; call it manually only if you are driving the LLM client directly.
</ParamField>

<ParamField body="governor.assertWithinBudget()" type="method">
  Check the current spend against configured caps. Returns `'ok'` if within budget, throws `BudgetExceededError` if `onBreach` is `'block'`, or calls the `onBreach` callback and returns the action for `'degrade'` and `'approve'`.
</ParamField>

<ParamField body="governor.isOverBudget()" type="boolean">
  Returns `true` if the current accumulated spend has reached or exceeded any configured cap. Does not throw or trigger the breach action.
</ParamField>

<ParamField body="governor.spentUsd" type="number">
  Read-only accessor for the total USD cost accumulated so far.
</ParamField>

<ParamField body="governor.spentTokens" type="number">
  Read-only accessor for the total token count accumulated so far.
</ParamField>

## Tracking cost across LLM calls

The LLM client reports `costUsd` per generate call based on the model's pricing. The governor accumulates these values across every iteration of the agent loop. Aborted streams (client disconnects mid-response) are counted too, so a disconnect never creates a cost blind spot.

<Tip>
  Set both `maxTokens` and `maxUsd` together to guard against runaway loops independently of pricing changes. A `maxTokens` cap fires predictably regardless of the model's cost per token.
</Tip>

## Example: degrade to a cheaper model on breach

```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { CostGovernor } from 'anvil-sdk/trace';
import { openai } from 'anvil-sdk/llm';

const premiumClient = openai({ model: 'gpt-4o' });
const fallbackClient = openai({ model: 'gpt-4o-mini' });

export const post = defineAgent({
  // resolveClient picks the model dynamically based on request state
  resolveClient: (ctx) => ctx.state.useFallback ? fallbackClient : premiumClient,
  budget: {
    maxUsd: 0.20,
    onBreach: 'degrade',
  },
  // ...
});
```

When `onBreach` is `'degrade'`, the `CostGovernor` fires its `onBreach` callback with `{ action: 'degrade', spentUsd, spentTokens }`. Wire that callback to update request state or swap the active client.