> ## Documentation Index
> Fetch the complete documentation index at: https://anvil.thatdevguy.in/llms.txt
> Use this file to discover all available pages before exploring further.

# LlmClient: Provider-Agnostic Model Calls in Anvil JS

> LlmClient wraps Anthropic, OpenAI, and Gemini behind one interface. Configure fallback chains, retries, structured output, and observability hooks.

`LlmClient` is the provider-agnostic model client at the heart of Anvil's agent system. It owns driver dispatch, transient retries, fallback chains, cost accumulation, and structured-output enforcement — so your agent handlers call `generate`, `generateObject`, or `stream` without worrying about provider-specific details.

Import it from `anvil/llm`:

```ts theme={null}
import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';
```

## Constructor

```ts theme={null}
const client = new LlmClient(options: LlmClientOptions);
```

<ParamField body="drivers" type="ModelDriver[]" required>
  One or more driver instances that handle model calls. Each driver advertises which models it supports via `driver.supports(model)`. At least one driver is required. Add multiple drivers to cover different model families.
</ParamField>

<ParamField body="defaultModel" type="string">
  The model used when a `generate`, `generateObject`, or `stream` call omits the `model` field. If omitted here too, the call throws unless the request specifies a model explicitly.
</ParamField>

<ParamField body="fallback" type="string[]">
  An ordered list of model IDs to try after the primary model fails with a transient error (`RetryableModelError`). The client walks the list left-to-right, retrying `maxRetries` times per model before advancing. Each fallback model must be supported by one of the registered drivers.
</ParamField>

<ParamField body="maxRetries" type="number">
  Attempts per model before moving to the next in the fallback chain. Defaults to `2`.
</ParamField>

<ParamField body="retryBaseMs" type="number">
  Base backoff delay in milliseconds between retry attempts. The actual wait before each attempt is `retryBaseMs * attemptNumber`. Defaults to `50`. Set to `0` in tests to eliminate wait time.
</ParamField>

<ParamField body="onTrace" type="(event: TraceEvent) => void">
  Observability hook called after every model call — successful or failed. The `TraceEvent` carries the model name, provider, attempt number, token usage, cost, and whether the call hit the fallback chain. Use this to feed your own metrics or logging pipeline.
</ParamField>

## Drivers

Provider SDKs are optional peer dependencies loaded lazily at runtime — the Anvil kernel never pulls them in unless you instantiate the matching driver.

<Tabs>
  <Tab title="Anthropic">
    ```ts theme={null}
    import { AnthropicDriver } from 'anvil-sdk/llm';

    new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })
    ```

    Requires `@anthropic-ai/sdk` as a peer dependency. Supports all `claude-*` models (e.g. `claude-opus-4-8`, `claude-sonnet-4-5`, `claude-haiku-3-5`). Uses the top-level `system` parameter and adaptive thinking when `thinking: true` is set on the request.

    ```ts theme={null}
    import { LlmClient, AnthropicDriver } from 'anvil-sdk/llm';

    const client = new LlmClient({
      drivers: [new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY })],
      defaultModel: 'claude-opus-4-8',
    });
    ```
  </Tab>

  <Tab title="OpenAI">
    ```ts theme={null}
    import { OpenAIDriver } from 'anvil-sdk/llm';

    new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY })
    ```

    Requires `openai` as a peer dependency. Supports `gpt-*` models (`gpt-4o`, `gpt-4o-mini`) and `o`-series reasoning models (`o1`, `o3`, `o4-mini`). Uses the Chat Completions API with structured output via `response_format.json_schema`.

    ```ts theme={null}
    import { LlmClient, OpenAIDriver } from 'anvil-sdk/llm';

    const client = new LlmClient({
      drivers: [new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY })],
      defaultModel: 'gpt-4o',
    });
    ```
  </Tab>

  <Tab title="Gemini">
    ```ts theme={null}
    import { GeminiDriver } from 'anvil-sdk/llm';

    new GeminiDriver({ apiKey: process.env.GEMINI_API_KEY })
    ```

    Requires `@google/generative-ai` as a peer dependency. Supports all `gemini-*` models (e.g. `gemini-2.5-flash`, `gemini-2.5-pro`). Gemini function calls carry no IDs; the driver synthesizes stable IDs from the function name so the tool loop works correctly.

    ```ts theme={null}
    import { LlmClient, GeminiDriver } from 'anvil-sdk/llm';

    const client = new LlmClient({
      drivers: [new GeminiDriver({ apiKey: process.env.GEMINI_API_KEY })],
      defaultModel: 'gemini-2.5-flash',
    });
    ```
  </Tab>
</Tabs>

## Methods

### `client.generate(req)`

Make a single-turn model call and return the complete result.

```ts theme={null}
const result = await client.generate({
  model: 'claude-opus-4-8',  // optional if defaultModel is set
  system: 'You are a helpful assistant.',
  messages: [{ role: 'user', content: 'What is the capital of France?' }],
});

console.log(result.text);        // "Paris"
console.log(result.usage);       // { inputTokens: 14, outputTokens: 6 }
console.log(result.costUsd);     // 0.000042
```

<ResponseField name="text" type="string">
  The model's text response.
</ResponseField>

<ResponseField name="model" type="string">
  The model ID that produced the response (may differ from the request if the fallback chain was used).
</ResponseField>

<ResponseField name="provider" type="string">
  The provider name, e.g. `"anthropic"`, `"openai"`, `"gemini"`.
</ResponseField>

<ResponseField name="usage" type="{ inputTokens: number, outputTokens: number }">
  Token counts for this call.
</ResponseField>

<ResponseField name="costUsd" type="number | undefined">
  Estimated USD cost from the built-in pricing table. `undefined` if pricing data for the model isn't registered.
</ResponseField>

<ResponseField name="toolCalls" type="ToolCall[] | undefined">
  Tool invocations the model requested, when tools were provided and `stopReason` is `"tool_use"`.
</ResponseField>

***

### `client.generateObject(req, schema, opts?)`

Request structured output validated against a Zod schema. On validation failure the client automatically re-prompts the model with the validation error, up to `maxRepairs` times.

```ts theme={null}
import { z } from 'zod';

const SentimentSchema = z.object({
  sentiment: z.enum(['positive', 'neutral', 'negative']),
  confidence: z.number().min(0).max(1),
});

const result = await client.generateObject(
  {
    system: 'Classify the sentiment of the user message.',
    messages: [{ role: 'user', content: 'This product is amazing!' }],
  },
  SentimentSchema,
  { maxRepairs: 2, name: 'sentiment_analysis' },
);

console.log(result.object.sentiment);    // "positive"
console.log(result.object.confidence);  // 0.97
```

<ParamField body="req" type="GenerateRequest" required>
  The same request shape as `generate`. Do not set `responseFormat` manually — `generateObject` sets it automatically from the Zod schema.
</ParamField>

<ParamField body="schema" type="z.ZodType<T>" required>
  The Zod schema to validate the response against.
</ParamField>

<ParamField body="opts.maxRepairs" type="number">
  Maximum number of repair attempts after a validation failure. Defaults to `2`. Each attempt re-prompts the model with the Zod error message.
</ParamField>

<ParamField body="opts.name" type="string">
  Optional name for the JSON schema advertised to the model. Helps some providers choose a better response format.
</ParamField>

***

### `client.stream(req)`

Stream tokens from the model as they arrive. Yields `StreamEvent` objects.

```ts theme={null}
for await (const event of client.stream({
  messages: [{ role: 'user', content: 'Tell me a short story.' }],
})) {
  if (event.type === 'text') process.stdout.write(event.text);
  if (event.type === 'done') console.log('\nUsage:', event.result.usage);
}
```

<ResponseField name="type: 'text'" type="StreamEvent">
  A text fragment from the model. Concatenate these to build the full response.
</ResponseField>

<ResponseField name="type: 'done'" type="StreamEvent">
  The final event, carrying the complete `GenerateResult` with usage, cost, and stop reason.
</ResponseField>

<Note>
  Streaming does not retry mid-stream — bytes may already be on the wire. The client only falls back to the next model in the chain if the stream fails before yielding any events.
</Note>

## Fallback chain example

Register both Anthropic and OpenAI drivers, set Claude as the primary, and fall back to GPT-4o on transient errors:

```ts theme={null}
import { LlmClient, AnthropicDriver, OpenAIDriver } from 'anvil-sdk/llm';

const client = new LlmClient({
  drivers: [
    new AnthropicDriver({ apiKey: process.env.ANTHROPIC_API_KEY }),
    new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY }),
  ],
  defaultModel: 'claude-opus-4-8',
  fallback: ['gpt-4o'],
  maxRetries: 2,
  onTrace: (event) => {
    if (event.fallback) {
      console.warn(`Fell back to ${event.model} after primary model failed`);
    }
  },
});
```

The client tries `claude-opus-4-8` up to two times. If both attempts throw a `RetryableModelError` (rate limit, 5xx, connection error) it moves to `gpt-4o` and tries that up to two times. Non-retryable errors (bad request, auth failure) stop the chain immediately.

<Tip>
  Install only the provider SDKs you need. Adding `AnthropicDriver` to your drivers list does not pull in `@anthropic-ai/sdk` until the first request that routes through it.
</Tip>