RAG Pipeline: Build Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) lets your agents answer questions grounded in your own documents rather than relying solely on the model’s training data. Anvil’s Retriever handles the full pipeline: splitting documents into chunks, embedding them, storing vectors, and retrieving the most similar chunks for any query. The retrievalContext step wires this directly into the agent loop so relevant chunks appear in the system prompt automatically.

Core concepts

Retriever

Orchestrates chunking, embedding, and indexing. Call index() to add documents and retrieve() to query them.

Embedder

Converts text to vectors. Use HashEmbedder for dev/tests, or plug in an OpenAI/Gemini provider embedder for semantic similarity.

VectorStore

Persists and queries (id, text, embedding) records. Choose MemoryVectorStore for tests or SqliteVectorStore for persistence.

retrievalContext

A ContextStep that auto-retrieves chunks for the current query and appends them to the agent’s system prompt before the model call.

Setting up a Retriever

Import the pieces you need and assemble a Retriever:

import { Retriever, HashEmbedder, SqliteVectorStore } from 'anvil/rag';

const embedder = new HashEmbedder(); // swap for OpenAI/Gemini in production
const store = await SqliteVectorStore.open(); // persists to .anvil/vectors.db

const retriever = new Retriever({ embedder, store });

Constructor options

RetrieverOptions

Option	Type	Required	Description
`embedder`	`Embedder`	✅	Converts text to vectors
`store`	`VectorStore`	✅	Persists and queries vector records
`chunk`	`ChunkOptions`	—	Chunking configuration (see below)

Indexing documents

Call retriever.index(docs) to chunk, embed, and store documents. It returns the number of chunks added.

import type { IndexDoc } from 'anvil/rag';

const docs: IndexDoc[] = [
  {
    id: 'policy-001',
    text: 'Our refund policy allows returns within 30 days of purchase...',
    metadata: { source: 'handbook', section: 'refunds' },
  },
  {
    id: 'policy-002',
    text: 'Shipping times vary by region. Standard delivery takes 3–5 business days...',
    metadata: { source: 'handbook', section: 'shipping' },
  },
];

const chunksAdded = await retriever.index(docs);
console.log(`Indexed ${chunksAdded} chunks`);

IndexDoc fields

Field	Type	Required	Description
`id`	`string`	—	Stable document identifier. Auto-generated as `doc-N` if omitted
`text`	`string`	✅	Full document text to chunk and embed
`metadata`	`Record<string, unknown>`	—	Arbitrary metadata returned with each result

Retrieving chunks

Call retriever.retrieve(query, options?) to get the top-K most similar chunks for a query string.

const results = await retriever.retrieve('What is your refund policy?', { topK: 3 });

for (const result of results) {
  console.log(`[${result.score.toFixed(3)}] ${result.text}`);
  // [0.921] Our refund policy allows returns within 30 days...
}

Results are VectorQueryResult[] sorted by descending cosine similarity:

Field	Type	Description
`id`	`string`	Chunk identifier (`docId#chunkIndex`)
`text`	`string`	Chunk text
`score`	`number`	Cosine similarity (0–1)
`metadata`	`Record<string, unknown>`	Metadata from the original `IndexDoc`

RetrieveOptions

Option	Type	Default	Description
`topK`	`number`	`4`	Maximum number of chunks to return
`trace`	`TraceHandle`	—	Records a `retrieval` span visible in the Anvil dashboard
`parentSpanId`	`string`	—	Parent span for nested trace trees

Embedders

HashEmbedder (dev/test)

HashEmbedder is a zero-dependency, deterministic embedder that uses a hashed bag-of-tokens approach. It requires no API key and no network calls, making it ideal for local development and CI. Because it is not semantic, replace it with a provider embedder before going to production.

import { HashEmbedder } from 'anvil/rag';

const embedder = new HashEmbedder();        // defaults to 256 dimensions
const embedder512 = new HashEmbedder(512);  // increase dimensions for larger corpora

Provider embedders

Any object that implements the Embedder interface works as a drop-in replacement:

export interface Embedder {
  readonly dimensions: number;
  embed(texts: string[]): Promise<number[][]>;
}

OpenAI
Gemini

import { OpenAIEmbedder } from 'anvil/providers/openai';

const embedder = new OpenAIEmbedder({
  model: 'text-embedding-3-small',
  apiKey: process.env.OPENAI_API_KEY,
});

import { GeminiEmbedder } from 'anvil/providers/gemini';

const embedder = new GeminiEmbedder({
  model: 'text-embedding-004',
  apiKey: process.env.GEMINI_API_KEY,
});

Vector stores

MemoryVectorStore (tests)

In-process, zero-dependency brute-force cosine search. Data is lost when the process exits.

import { MemoryVectorStore } from 'anvil/rag';

const store = new MemoryVectorStore();

SqliteVectorStore (production)

Persistent SQLite-backed store. Requires the better-sqlite3 optional peer dependency. Similarity search runs in JS over all stored records — appropriate for small-to-medium corpora embedded in an application.

import { SqliteVectorStore } from 'anvil/rag';

const store = await SqliteVectorStore.open();               // .anvil/vectors.db
const custom = await SqliteVectorStore.open('./data/kb.db'); // custom path

Install better-sqlite3 when you use SqliteVectorStore. The package loads lazily so it never breaks builds that use only MemoryVectorStore.

Custom chunking

chunkText(text, options?) splits text into overlapping chunks, preferring natural paragraph and sentence boundaries. You can call it yourself before indexing if you need fine-grained control.

import { chunkText } from 'anvil/rag';

const chunks = chunkText(longDocument, {
  size: 600,    // max characters per chunk (default 800)
  overlap: 80,  // overlap between consecutive chunks (default 100)
});

Pass the same options directly to the Retriever via the chunk constructor option:

const retriever = new Retriever({
  embedder,
  store,
  chunk: { size: 600, overlap: 80 },
});

Using RAG in an agent

retrievalContext from anvil/agent is a ContextStep that retrieves chunks for the current user query and appends them to the agent’s system prompt before each model call. This is the recommended way to integrate RAG into an agent loop.

Complete example

import { defineAgent } from 'anvil/agent';
import { retrievalContext } from 'anvil/agent';
import { Retriever, HashEmbedder, SqliteVectorStore } from 'anvil/rag';

// 1. Build the retriever
const embedder = new HashEmbedder();
const store = await SqliteVectorStore.open();
const retriever = new Retriever({ embedder, store });

// 2. Index your knowledge base once at startup (or on ingestion)
await retriever.index([
  { id: 'faq-1', text: 'Refunds are accepted within 30 days...', metadata: { section: 'refunds' } },
  { id: 'faq-2', text: 'Standard shipping takes 3–5 business days...', metadata: { section: 'shipping' } },
  { id: 'faq-3', text: 'Contact support at help@example.com...', metadata: { section: 'contact' } },
]);

// 3. Define an agent that retrieves context automatically
export default defineAgent({
  name: 'support-agent',
  model: 'gpt-4o-mini',
  system: 'You are a helpful customer support agent. Use the provided context to answer questions accurately.',
  context: [
    retrievalContext(retriever, { topK: 3 }),
  ],
  async run(ctx) {
    return ctx.complete();
  },
});

Custom chunk rendering

Override how retrieved chunks appear in the system prompt with the template option:

retrievalContext(retriever, {
  topK: 5,
  template: (chunks) =>
    `## Knowledge Base\n${chunks.map((c, i) => `${i + 1}. ${c}`).join('\n')}`,
})

Pass the request’s trace handle to retriever.retrieve() if you call it manually — this records a retrieval span in the Anvil dashboard so you can see exactly which chunks influenced each response.

​Core concepts

Retriever

Embedder

VectorStore

retrievalContext

​Setting up a Retriever

​Constructor options

​Indexing documents

​IndexDoc fields

​Retrieving chunks

​RetrieveOptions

​Embedders

​HashEmbedder (dev/test)

​Provider embedders

​Vector stores

​MemoryVectorStore (tests)

​SqliteVectorStore (production)

​Custom chunking

​Using RAG in an agent

​Complete example

​Custom chunk rendering

Core concepts

Setting up a Retriever

Constructor options

Indexing documents

IndexDoc fields

Retrieving chunks

RetrieveOptions

Embedders

HashEmbedder (dev/test)

Provider embedders

Vector stores

MemoryVectorStore (tests)

SqliteVectorStore (production)

Custom chunking

Using RAG in an agent

Complete example

Custom chunk rendering