A RAG chatbot answers in two steps. First it retrieves the passages of your own content that match the question: product records, policy pages, uploaded documents. Then it writes an answer from those passages only, and says it does not know when nothing relevant was found. That is what makes it usable on a store, where a confident guess costs money. Below: one question walked through both steps, the terms explained, and where retrieval still fails.
One question, end to end
A shopper types "can I return sale items?". Here is what happens inside a retrieval-based bot such as Vatdi in the second or so before the answer appears.
- The question is turned into a search. Two searches, in fact: a meaning-based one (the question is converted into a numeric representation, an embedding, and compared with the embeddings of every chunk of your content) and a keyword one (the literal words "return" and "sale"). Combining them catches both the paraphrased match and the exact-term match.
- The best-matching chunks are retrieved. Say the top results are a paragraph from your returns page ("Sale items can be returned within 14 days if unworn, with tags") and a line from your FAQ. Only these passages go forward.
- The model writes from those passages. It is instructed to answer using the retrieved text and nothing else. The reply: "Yes. Sale items can be returned within 14 days as long as they are unworn and still have their tags."
- If nothing matched, it declines. Had your site never mentioned sale returns, the bot would say it is not sure and offer a person, rather than composing a plausible policy.
The pattern was named in a 2020 research paper (Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"). The shorter definition is on our what is a RAG chatbot page.
The terms, in plain words
| Term | What it means here | Why you care |
|---|---|---|
| Chunk | A piece of your content small enough to search and quote: a paragraph, a product record, an FAQ entry | Content written in short, single-topic pieces retrieves precisely; long mixed pages retrieve vaguely |
| Embedding | A list of numbers that represents the meaning of a chunk or a question, so similar meanings land near each other (see OpenAI's embeddings guide) | Lets "do you take stuff back" find the returns page |
| Keyword search | Matching the literal words | Catches product codes, names and exact terms that meaning search can blur |
| Index | The stored, searchable set of chunks and embeddings for your store | Updated when the catalogue syncs, pages are re-crawled or documents replaced |
| Grounding | Instructing the model to answer only from the retrieved passages | The reason a RAG bot can say "I don't know" |
| Generation | The model writing the reply | Vatdi uses OpenAI's GPT-4o mini; the model matters less than the passages it is given |
Why RAG beats the alternatives for a store
- Versus a generic AI chat: a general model answers from training data and fills gaps with plausible text. It does not know your returns window; it will state one anyway.
- Versus fine-tuning a model on your content: fine-tuning bakes yesterday's content into the model and cannot cite a source. Prices change weekly; a retrieval index updates with the sync, and every answer traces to a passage.
- Versus rule-based flows: flows handle the paths someone built. Retrieval handles the phrasing nobody anticipated, as long as the answer exists in your content. The trade-offs are laid out in AI chatbot vs rule-based chatbot.
The cost of these advantages is a dependency: the bot is exactly as good as the passages it can retrieve, which is why the rest of this guide is about failures and content.
Where a RAG chatbot still fails, and the fix for each
| Symptom | Usual cause | Fix |
|---|---|---|
| "I don't know" to a question you have answered somewhere | The answer is in an image, a PDF table image or a page not crawled | Put the fact in text; crawl the page or upload the document |
| Correct-sounding but outdated answer | An old promotion or policy page is still in the index | Remove retired pages from the crawl; keep prices only in the synced catalogue |
| Answer mixes two products or two policies | One long page covers several topics, so one chunk holds both | One question per heading; one product per record |
| Short message gets a generic reply ("ok", "medium") | Too little to retrieve on; the previous turn's context was needed | Good bots carry the conversation context; test follow-ups explicitly and set a fallback language |
| Right product, wrong variant detail | Attribute stored in a description sentence rather than a field | Move attributes (size, material, compatibility) into structured fields the sync reads |
| Confident answer to a question you cannot answer | A loosely related passage was retrieved and treated as relevant | Add the explicit statement ("we do not ship to X") so retrieval finds the true answer; test negative cases |
Most of these are content problems dressed as AI problems. Our knowledge base guide shows the before-and-after for shipping, returns and sizing pages, and chatbot training data best practices covers formats and update cadence.
Writing content that retrieves well
- One question per heading, the number in the first sentence. "Can I return sale items? Yes, within 14 days, unworn, with tags." That paragraph becomes one clean chunk.
- Keep each fact in one place. Prices in the catalogue only; shipping times on the shipping page only. Duplicates drift, and retrieval may pick the stale copy.
- Name things consistently. If the product is the "Trailhead 40L" on the product page, do not call it "the big backpack" in the FAQ.
- State the negatives. "We do not ship to the US" retrieves; silence does not.
- Prefer structured fields to prose for attributes, so the catalogue sync carries them and the product card can show them.
How to test retrieval on your store
Pick twenty questions: ten you know the content answers, five you know it does not, five phrased the way a shopper types rather than the way your page is written ("do u take returns on sale stuff"). Ask them in the live widget. Pass conditions: the ten are answered from the right page (you can tell because the numbers match), the five are declined with an offer of a person, and the five paraphrases still find the right passage. Vatdi shows the sources it used and grades each conversation 0–10, so a failed test points to the chunk to fix. Re-run the set after every content change; the technique list in advanced chatbot techniques builds a weekly loop around it. How the training sources connect is in how to train a chatbot on your data and RAG training; all of it is included on every plan (pricing).
Frequently asked questions
What does RAG stand for, and do I need to understand it to use a chatbot?
Retrieval-augmented generation: retrieve the relevant passages from your own content, then generate the answer from them. You do not need the theory, but one consequence matters daily: the bot can only answer what your content states, and it should decline otherwise. That turns chatbot quality into a content task you control rather than a model setting you cannot.
Is a RAG chatbot the same as training ChatGPT on my data?
No. Training (fine-tuning) changes the model's weights with your content and cannot cite sources or stay current without retraining. RAG leaves the model unchanged and instead gives it the matching passages at question time, so an updated page changes the next answer and every reply traces to a source. For a store with weekly price changes, retrieval is the practical choice.
Why does the bot sometimes say "I don't know" when the answer is on my site?
Usually because the fact is not in retrievable text: it is in an image, a PDF rendered as a picture, a page that was never crawled, or a long page where the relevant sentence sits among unrelated ones. Put the fact in plain text under its own heading, crawl the page or upload the document, and ask again. If the phrasing is the issue, add the shopper's wording to an FAQ entry.
How often is the index updated?
The catalogue updates through the plugin sync as products change; pages are re-crawled on the schedule you set; uploaded documents change when you replace them. There is no training run to wait for. The habit that matters is checking after a change: edit the returns page, ask the related question the same day, confirm the new wording appears in the answer.
Can retrieval leak content I did not mean to publish?
It answers from whatever you give it, so yes if you upload the wrong file. Keep the sources to public pages, product data and documents meant for customers; never upload customer lists, order exports or staff manuals. Review the source list quarterly. Data handling by the vendor is a separate question, covered in our GDPR checklist and Vatdi's trust page.
Does a bigger AI model fix retrieval problems?
No. If the right passage was not retrieved, no model can answer from it; if a stale passage was retrieved, a stronger model states the stale fact more fluently. Model quality affects tone and reasoning over the passages it is given. The fixes for wrong answers are almost always in the content and its structure, which is why this guide spends so long on them.
