Add a Claude-Powered Chatbot to Your Website: a RAG Walkthrough

A chatbot that answers from your content is one of the most requested AI features on business sites, and the architecture behind it (retrieval augmented generation, RAG) is well within reach of one developer and an afternoon. This walkthrough builds the whole thing: content chunking, embeddings, retrieval, and a Claude-powered answer endpoint you can drop into an Astro or Next.js site.

This is the same architecture my site assistant product ships for clients, minus the production hardening covered at the end. If you want it done for you, that page has fixed prices; if you want to build it, everything follows.

Short answer: Split your content into chunks, turn each chunk into an embedding vector, store them (a JSON file is genuinely enough for a marketing site), then at question time: embed the question, find the most similar chunks, and have Claude answer using only those chunks, with links to the sources. The magic is not the model, it is the grounding: Claude answers from your retrieved content instead of its general memory, which is what keeps a site chatbot honest.

The architecture in one pass

1. Chunk: split your pages and posts into passages a few hundred words long
2. Embed: convert each passage to a vector that captures its meaning
3. Store: keep vectors + text + source URL together
4. Retrieve: embed the visitor’s question, find the top matching passages by cosine similarity
5. Answer: send question + passages to Claude with strict grounding rules; return the answer with sources

Two accounts are needed: an Anthropic API key for answers, and an embeddings provider. Anthropic does not ship its own embeddings endpoint; Voyage AI is the provider Anthropic recommends, and its free tier covers a marketing site comfortably.

Step 1: Chunk your content

For an Astro site with MDX content, your source material is already clean markdown on disk:

// chunk-content.mjs: MDX files -> chunks.json
import fs from 'node:fs';
import path from 'node:path';

const files = fs.readdirSync('src/content/blog').filter((f) => f.endsWith('.mdx'));
const chunks = [];

for (const file of files) {
  const raw = fs.readFileSync(path.join('src/content/blog', file), 'utf8');
  const body = raw.replace(/^---[\s\S]*?---/, '');       // strip frontmatter
  const slug = file.replace(/\.mdx?$/, '');
  const sections = body.split(/\n(?=## )/);              // split on H2 headings

  for (const section of sections) {
    const text = section.trim();
    if (text.length > 200) {
      chunks.push({ url: `/blog/${slug}`, text: text.slice(0, 4000) });
    }
  }
}

fs.writeFileSync('chunks.json', JSON.stringify(chunks, null, 2));
console.log(`${chunks.length} chunks ready`);

Heading-based splitting beats fixed-size splitting for marketing content: each chunk stays a coherent answer to one question, which is exactly what retrieval wants.

Step 2: Embed the chunks

// embed-chunks.mjs: chunks.json -> index.json (vectors included)
import fs from 'node:fs';

const chunks = JSON.parse(fs.readFileSync('chunks.json', 'utf8'));
const index = [];

for (let i = 0; i < chunks.length; i += 64) {
  const batch = chunks.slice(i, i + 64);
  const res = await fetch('https://api.voyageai.com/v1/embeddings', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.VOYAGE_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'voyage-3.5',
      input: batch.map((c) => c.text),
      input_type: 'document',
    }),
  });
  const { data } = await res.json();
  batch.forEach((chunk, j) => index.push({ ...chunk, vector: data[j].embedding }));
}

fs.writeFileSync('src/data/rag-index.json', JSON.stringify(index));
console.log(`Embedded ${index.length} chunks`);

Re-run both scripts whenever content changes; wiring them into your build pipeline makes the index self-updating.

Step 3: Storage, honestly

For a marketing site with a few hundred chunks, the JSON file you just wrote is the vector database. Cosine similarity over a few hundred vectors takes microseconds, ships with your site, and has zero moving parts. Reach for pgvector or a hosted vector store when you pass tens of thousands of chunks or need live updates without redeploys, not before.

Step 4: The answer endpoint

An Astro API route (a Next.js route handler is the same logic). Claude answers with strict grounding rules and returns sources:

// src/pages/api/ask.ts
import type { APIRoute } from 'astro';
import Anthropic from '@anthropic-ai/sdk';
import index from '../../data/rag-index.json';

export const prerender = false;

const client = new Anthropic(); // reads ANTHROPIC_API_KEY

const cosine = (a: number[], b: number[]) => {
  let dot = 0, na = 0, nb = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i]; na += a[i] * a[i]; nb += b[i] * b[i];
  }
  return dot / (Math.sqrt(na) * Math.sqrt(nb));
};

export const POST: APIRoute = async ({ request }) => {
  const { question } = await request.json();
  if (!question || question.length > 500) {
    return new Response(JSON.stringify({ error: 'Invalid question' }), { status: 400 });
  }

  // Embed the question
  const embedRes = await fetch('https://api.voyageai.com/v1/embeddings', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${import.meta.env.VOYAGE_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ model: 'voyage-3.5', input: [question], input_type: 'query' }),
  });
  const { data } = await embedRes.json();
  const qVector = data[0].embedding;

  // Retrieve top 4 chunks
  const top = index
    .map((chunk) => ({ ...chunk, score: cosine(qVector, chunk.vector) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, 4);

  // Ask Claude, grounded in the retrieved chunks
  const response = await client.messages.create({
    model: 'claude-opus-5',
    max_tokens: 1024,
    system: [
      'You are the assistant for this website. Answer questions using ONLY the provided context passages.',
      'If the context does not contain the answer, say you do not know and suggest contacting the site owner.',
      'Never invent prices, features, or policies. Keep answers under 150 words.',
      'End your answer by listing the source URLs you used.',
    ].join(' '),
    messages: [{
      role: 'user',
      content: `Context passages:\n\n${top
        .map((c) => `[Source: ${c.url}]\n${c.text}`)
        .join('\n\n---\n\n')}\n\nQuestion: ${question}`,
    }],
  });

  const answer = response.content
    .filter((block) => block.type === 'text')
    .map((block) => block.text)
    .join('');

  return new Response(
    JSON.stringify({ answer, sources: [...new Set(top.map((c) => c.url))] }),
    { headers: { 'Content-Type': 'application/json' } }
  );
};

The system prompt is doing the guardrail work: answer only from context, admit ignorance, never invent prices, cite sources. Those four rules are the difference between an assistant and a liability.

Step 5: A minimal widget

Any form that POSTs to the endpoint works:

const res = await fetch('/api/ask', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ question: inputValue }),
});
const { answer, sources } = await res.json();

Style it to your brand, render the sources as links, and you have a working site assistant. On a Webflow site, the widget embeds the same way; the endpoint just lives on Netlify or Vercel functions instead.

Production hardening (the part clients pay for)

The walkthrough above works. Before real visitors use it, add:

• Rate limiting on the endpoint, or your API bill becomes someone’s toy
• Streaming responses for perceived speed on longer answers
• Question logging so you learn what visitors actually ask (and what your content fails to answer)
• Evaluation against a set of real questions before launch, re-run when content changes
• Cost controls: for high-volume widgets, claude-haiku-4-5 cuts answer costs to a fifth; keep claude-opus-5 where answer quality sells your services
• Prompt caching on the system prompt if traffic is steady, cutting input costs further

This list is exactly what separates my $1,500 site assistant build from the tutorial: the architecture is the same, the hardening and testing are the product.

What it costs to run

Real numbers for a marketing-site assistant: embeddings for a few hundred chunks cost cents and re-embed rarely. Answer traffic is the variable: at roughly a thousand questions a month with retrieved context, expect $20 to $100 per month in API usage depending on the model, most sites landing at the low end. There is no per-seat fee and no platform subscription; the infrastructure is your site plus two API keys you own.

FAQ

Which Claude model should I use?

Start with claude-opus-5 and measure: for a services site, one great answer is worth more than ten cheap ones. If volume grows and answers stay simple, claude-haiku-4-5 handles grounded Q&A well at a fifth of the cost. The architecture does not change when you swap the model string.

Will it hallucinate?

Less than you fear, if grounding is strict. The system prompt restricts answers to retrieved context, requires an honest “I do not know,” and bans invented specifics; retrieval quality then becomes your main lever, which is why chunking by heading matters. Test with questions your content cannot answer and verify it declines.

Can this work on a Webflow site?

Yes. The widget is a small embed, and the endpoint runs on Netlify or Vercel functions. Your content comes out through the Webflow API for chunking, the same export approach as my migration guide uses, minus the migration.

Should I build or buy?

Build it if you have a developer and this post made sense: the architecture is honest work but not hard. Buy it when you want the hardening, evaluation, and upkeep handled: my AI integration services ship this from $1,500 fixed with an optional care plan, and the audit tells you honestly whether a chatbot will even help your site.