> ## Documentation Index
> Fetch the complete documentation index at: https://anvil.thatdevguy.in/llms.txt
> Use this file to discover all available pages before exploring further.

# RAG Pipeline: Build Retrieval-Augmented Generation

> Index documents with Retriever, embed and store chunks in a VectorStore, then inject the most relevant context into your agent loop automatically.

Retrieval-Augmented Generation (RAG) lets your agents answer questions grounded in your own documents rather than relying solely on the model's training data. Anvil's `Retriever` handles the full pipeline: splitting documents into chunks, embedding them, storing vectors, and retrieving the most similar chunks for any query. The `retrievalContext` step wires this directly into the agent loop so relevant chunks appear in the system prompt automatically.

## Core concepts

<CardGroup cols={2}>
  <Card title="Retriever" icon="magnifying-glass">
    Orchestrates chunking, embedding, and indexing. Call `index()` to add documents and `retrieve()` to query them.
  </Card>

  <Card title="Embedder" icon="arrow-right-arrow-left">
    Converts text to vectors. Use `HashEmbedder` for dev/tests, or plug in an OpenAI/Gemini provider embedder for semantic similarity.
  </Card>

  <Card title="VectorStore" icon="database">
    Persists and queries `(id, text, embedding)` records. Choose `MemoryVectorStore` for tests or `SqliteVectorStore` for persistence.
  </Card>

  <Card title="retrievalContext" icon="bolt">
    A `ContextStep` that auto-retrieves chunks for the current query and appends them to the agent's system prompt before the model call.
  </Card>
</CardGroup>

## Setting up a Retriever

Import the pieces you need and assemble a `Retriever`:

```ts theme={null}
import { Retriever, HashEmbedder, SqliteVectorStore } from 'anvil-sdk/rag';

const embedder = new HashEmbedder(); // swap for OpenAI/Gemini in production
const store = await SqliteVectorStore.open(); // persists to .anvil/vectors.db

const retriever = new Retriever({ embedder, store });
```

### Constructor options

<Accordion title="RetrieverOptions">
  | Option     | Type           | Required | Description                         |
  | ---------- | -------------- | -------- | ----------------------------------- |
  | `embedder` | `Embedder`     | ✅        | Converts text to vectors            |
  | `store`    | `VectorStore`  | ✅        | Persists and queries vector records |
  | `chunk`    | `ChunkOptions` | —        | Chunking configuration (see below)  |
</Accordion>

## Indexing documents

Call `retriever.index(docs)` to chunk, embed, and store documents. It returns the number of chunks added.

```ts theme={null}
import type { IndexDoc } from 'anvil-sdk/rag';

const docs: IndexDoc[] = [
  {
    id: 'policy-001',
    text: 'Our refund policy allows returns within 30 days of purchase...',
    metadata: { source: 'handbook', section: 'refunds' },
  },
  {
    id: 'policy-002',
    text: 'Shipping times vary by region. Standard delivery takes 3–5 business days...',
    metadata: { source: 'handbook', section: 'shipping' },
  },
];

const chunksAdded = await retriever.index(docs);
console.log(`Indexed ${chunksAdded} chunks`);
```

### IndexDoc fields

| Field      | Type                      | Required | Description                                                      |
| ---------- | ------------------------- | -------- | ---------------------------------------------------------------- |
| `id`       | `string`                  | —        | Stable document identifier. Auto-generated as `doc-N` if omitted |
| `text`     | `string`                  | ✅        | Full document text to chunk and embed                            |
| `metadata` | `Record<string, unknown>` | —        | Arbitrary metadata returned with each result                     |

## Retrieving chunks

Call `retriever.retrieve(query, options?)` to get the top-K most similar chunks for a query string.

```ts theme={null}
const results = await retriever.retrieve('What is your refund policy?', { topK: 3 });

for (const result of results) {
  console.log(`[${result.score.toFixed(3)}] ${result.text}`);
  // [0.921] Our refund policy allows returns within 30 days...
}
```

Results are `VectorQueryResult[]` sorted by descending cosine similarity:

| Field      | Type                      | Description                           |
| ---------- | ------------------------- | ------------------------------------- |
| `id`       | `string`                  | Chunk identifier (`docId#chunkIndex`) |
| `text`     | `string`                  | Chunk text                            |
| `score`    | `number`                  | Cosine similarity (0–1)               |
| `metadata` | `Record<string, unknown>` | Metadata from the original `IndexDoc` |

### RetrieveOptions

| Option         | Type          | Default | Description                                               |
| -------------- | ------------- | ------- | --------------------------------------------------------- |
| `topK`         | `number`      | `4`     | Maximum number of chunks to return                        |
| `trace`        | `TraceHandle` | —       | Records a `retrieval` span visible in the Anvil dashboard |
| `parentSpanId` | `string`      | —       | Parent span for nested trace trees                        |

## Embedders

### HashEmbedder (dev/test)

`HashEmbedder` is a zero-dependency, deterministic embedder that uses a hashed bag-of-tokens approach. It requires no API key and no network calls, making it ideal for local development and CI. Because it is not semantic, replace it with a provider embedder before going to production.

```ts theme={null}
import { HashEmbedder } from 'anvil-sdk/rag';

const embedder = new HashEmbedder();        // defaults to 256 dimensions
const embedder512 = new HashEmbedder(512);  // increase dimensions for larger corpora
```

### Provider embedders

Any object that implements the `Embedder` interface works as a drop-in replacement:

```ts theme={null}
export interface Embedder {
  readonly dimensions: number;
  embed(texts: string[]): Promise<number[][]>;
}
```

<Tabs>
  <Tab title="OpenAI">
    ```ts theme={null}
    import { OpenAIEmbedder } from 'anvil-sdk/providers/openai';

    const embedder = new OpenAIEmbedder({
      model: 'text-embedding-3-small',
      apiKey: process.env.OPENAI_API_KEY,
    });
    ```
  </Tab>

  <Tab title="Gemini">
    ```ts theme={null}
    import { GeminiEmbedder } from 'anvil-sdk/providers/gemini';

    const embedder = new GeminiEmbedder({
      model: 'text-embedding-004',
      apiKey: process.env.GEMINI_API_KEY,
    });
    ```
  </Tab>
</Tabs>

## Vector stores

### MemoryVectorStore (tests)

In-process, zero-dependency brute-force cosine search. Data is lost when the process exits.

```ts theme={null}
import { MemoryVectorStore } from 'anvil-sdk/rag';

const store = new MemoryVectorStore();
```

### SqliteVectorStore (production)

Persistent SQLite-backed store. Requires the `better-sqlite3` optional peer dependency. Similarity search runs in JS over all stored records — appropriate for small-to-medium corpora embedded in an application.

```ts theme={null}
import { SqliteVectorStore } from 'anvil-sdk/rag';

const store = await SqliteVectorStore.open();               // .anvil/vectors.db
const custom = await SqliteVectorStore.open('./data/kb.db'); // custom path
```

<Note>
  Install `better-sqlite3` when you use `SqliteVectorStore`. The package loads lazily so it never breaks builds that use only `MemoryVectorStore`.
</Note>

## Custom chunking

`chunkText(text, options?)` splits text into overlapping chunks, preferring natural paragraph and sentence boundaries. You can call it yourself before indexing if you need fine-grained control.

```ts theme={null}
import { chunkText } from 'anvil-sdk/rag';

const chunks = chunkText(longDocument, {
  size: 600,    // max characters per chunk (default 800)
  overlap: 80,  // overlap between consecutive chunks (default 100)
});
```

Pass the same options directly to the `Retriever` via the `chunk` constructor option:

```ts theme={null}
const retriever = new Retriever({
  embedder,
  store,
  chunk: { size: 600, overlap: 80 },
});
```

## Using RAG in an agent

`retrievalContext` from `anvil/agent` is a `ContextStep` that retrieves chunks for the current user query and appends them to the agent's system prompt before each model call. This is the recommended way to integrate RAG into an agent loop.

### Complete example

```ts theme={null}
import { defineAgent } from 'anvil-sdk/agent';
import { retrievalContext } from 'anvil-sdk/agent';
import { Retriever, HashEmbedder, SqliteVectorStore } from 'anvil-sdk/rag';

// 1. Build the retriever
const embedder = new HashEmbedder();
const store = await SqliteVectorStore.open();
const retriever = new Retriever({ embedder, store });

// 2. Index your knowledge base once at startup (or on ingestion)
await retriever.index([
  { id: 'faq-1', text: 'Refunds are accepted within 30 days...', metadata: { section: 'refunds' } },
  { id: 'faq-2', text: 'Standard shipping takes 3–5 business days...', metadata: { section: 'shipping' } },
  { id: 'faq-3', text: 'Contact support at help@example.com...', metadata: { section: 'contact' } },
]);

// 3. Define an agent that retrieves context automatically
export default defineAgent({
  name: 'support-agent',
  model: 'gpt-4o-mini',
  system: 'You are a helpful customer support agent. Use the provided context to answer questions accurately.',
  context: [
    retrievalContext(retriever, { topK: 3 }),
  ],
  async run(ctx) {
    return ctx.complete();
  },
});
```

### Custom chunk rendering

Override how retrieved chunks appear in the system prompt with the `template` option:

```ts theme={null}
retrievalContext(retriever, {
  topK: 5,
  template: (chunks) =>
    `## Knowledge Base\n${chunks.map((c, i) => `${i + 1}. ${c}`).join('\n')}`,
})
```

<Tip>
  Pass the request's `trace` handle to `retriever.retrieve()` if you call it manually — this records a `retrieval` span in the Anvil dashboard so you can see exactly which chunks influenced each response.
</Tip>