What Is a RAG Chatbot? Retrieval-Augmented Generation Explained

The word "chatbot" covers three different machines, and the difference explains why RAG exists.

So "RAG vs chatbot" is not really a contest between two products. RAG is an architecture that turns a general language model into one that answers from a specific body of text. The model itself is not changed; the model is given the right material to read before it answers, every single time.

Every production RAG system does roughly the same nine things. Two of them happen when data changes; the rest happen on every question.

1. Ingest sources. Product catalogue (through a platform plugin or feed), website pages (crawl), uploaded files (PDF, DOC, CSV, Markdown), pasted text and FAQ lists. Each source becomes a set of records with metadata such as URL, price, stock and last-updated time.
2. Clean and chunk. Navigation, cookie banners and boilerplate are stripped. Long pages are split into chunks small enough to be specific but large enough to keep meaning, typically a few hundred words, with overlap so a sentence at a boundary is not lost.
3. Embed. Each chunk is converted into an embedding, a numeric vector that represents its meaning, so that "does it fit a 15-inch laptop" and "compatible with 15" notebooks" land close together.
4. Store in a searchable index. Vectors go into a vector index. Good systems also keep a keyword index alongside it and run hybrid search, because semantic search alone is poor at exact tokens like SKUs, model numbers and brand names.
5. Retrieve top matches per question. The visitor's question is embedded the same way and compared against the index; the best-scoring chunks, often five to twenty, come back.
6. Filter and rank. Structured filters are applied on top of similarity: only products under the stated budget, only items in stock, only this store's data. Results may be re-ranked by a second model or by business signals such as units sold.
7. Build the prompt. The retrieved snippets, the conversation so far and the instructions ("answer only from the passages below; if they do not contain the answer, say so") are assembled into a prompt for the language model.
8. Generate. The model writes the reply in the visitor's language. Its job is phrasing, not knowledge; the knowledge is in the snippets.
9. Show sources or product cards. The reply is returned with the products or pages it drew on, as cards with image, price and link, or as citations, so the visitor and the merchant can check it.

Steps 1 to 4 run when the catalogue or a document changes; steps 5 to 9 run in a second or two for each message.

Fine-tuning means further training a language model on your own text so that its weights absorb it. It sounds like the more thorough approach. For a store it is almost always the wrong one.

Fine-tuning has a real use: teaching a model a tone, a format or a specialised vocabulary. It is a poor way to store facts that change. For facts, retrieval wins on freshness, cost, auditability and the ability to forget.

RAG reduces hallucination; it does not make a bot correct by default. These are the failure modes you should ask any vendor about, and test for yourself.

• Poor chunking. If a product's price ends up in a different chunk from its name, the bot retrieves the name and invents the price. Fix: chunk by record (one product, one policy clause) with metadata attached, not by fixed character count alone.
• Stale index. The sale ended on Sunday; the index still says 20% off on Tuesday. Fix: sync on change, and put the last-sync time where the merchant can see it.
• Retrieval misses on very short queries. "Blue?" or a bare model number gives an embedding little to work with. Fix: hybrid keyword search, using the conversation history to expand the query, and asking a clarifying question instead of guessing.
• Answering when nothing relevant was retrieved. The most damaging failure. The retrieved chunks are off-topic, the model answers anyway from general knowledge, and the visitor is told a returns window that does not exist. Fix: a relevance threshold, and a prompt rule that forces "I do not know, would you like to speak to someone?" when nothing passes it.
• Wrong-store data leaks. On a multi-tenant platform with weak isolation, a query on one store can retrieve another store's products. Fix: a hard store filter enforced at the index level on every query, not a soft instruction in the prompt.
• Over-long context. Stuffing 40 chunks into the prompt makes the model pick the wrong one and costs more. Fix: retrieve more, re-rank, pass fewer.
• Language mismatch. The visitor asks in Greek, the index is in English. Fix: multilingual embeddings so meaning matches across languages, and generation in the visitor's language.

None of these are exotic. They are the difference between a demo and a bot you can leave alone with customers.

Do not judge a RAG bot on five demo questions. Build a small test set and score it the same way every time.

1. Collect 50 to 100 real questions. Take them from chat logs, support email and search queries on the site. Include short ones, misspelled ones, ones in other languages, and at least ten the bot should refuse because the answer is not in your data.
2. Write the expected answer for each, or at least the facts it must contain and the facts it must not invent.
3. Run every question and score two things 0–10: grounding (did the answer come only from your data, with the right product or page attached?) and correctness (is it right?). A fluent, wrong answer scores 0 on correctness however good it sounds.
4. Check the refusals. The ten unanswerable questions should produce "I do not know" or a handover, not a guess. A bot that scores 9 on the answerable set and invents answers for the unanswerable set is not ready.
5. Re-run after every knowledge change. A new catalogue sync, a new policy document or a prompt tweak can silently break a question that used to pass. Keep the scores in a sheet and compare.
6. Read low-scoring transcripts weekly in production. Real visitors phrase things you did not think of. Each low score is either missing data or a retrieval gap, and both are fixable.

A platform that grades conversations for you speeds this up but does not replace your own test set; you know which answers are wrong, the grader only knows which look unsupported.

Most RAG problems are data problems. Before blaming retrieval, run through this list:

• One source of truth per fact. If the returns window appears in three documents with three numbers, the bot will pick one at random. Delete or correct the extras.
• Product titles that mean something. "Item 4471" retrieves badly; "Oak dining table 180 cm, seats 6" retrieves well.
• Prices, stock and variants come from the catalogue sync, never from a pasted PDF that will go stale.
• Policies as clean text or a document, not a screenshot. Image-only PDFs have nothing to embed.
• Remove discontinued products from the index, not just from the storefront menu.
• Keep FAQs short and one question per entry; a 40-question FAQ page as a single chunk answers nothing well.
• Date-stamp promotions inside the text ("valid until 30 November 2026") so the model can see they have ended even if the sync is late.
• Check what the crawler actually captured: cookie banners, menus and footers crowd out the content if they are not stripped.
• Re-test the 50–100 question set after each clean-up.

The guide to training a chatbot on your data goes through source types and file formats in more detail.

Almost every chatbot vendor now says its bot is "trained on your data". In nearly all cases this means RAG, not fine-tuning: your content is indexed and retrieved at question time, and the underlying model is a general one from OpenAI or another model provider. That is the right design for a store, so the phrase is not dishonest, but it hides the questions that actually separate products. Ask any vendor:

• Is retrieval semantic only, or hybrid with keyword search? (SKUs and model numbers need keywords.)
• How is the product catalogue kept current: a native plugin sync, a feed, or a periodic crawl?
• Can the bot apply filters such as price and stock, or only text similarity?
• What happens when nothing relevant is retrieved? Ask for a demo of the bot refusing.
• Is store data isolated at the index level?
• Are the retrieved sources shown to the visitor, and logged for you?
• Which model generates the answer, and is your data used to train it? (Look for API use with no training on your content.)
• How can you test it: is there a way to run your own question set and see scores?

Chatbase, Botpress, Tidio's Lyro, Intercom's Fin and Vatdi all answer these differently; the pricing units differ too, from flat plans to per-resolution fees. Comparing the answers tells you more than the phrase on the homepage. For a broader comparison see the best AI chatbots for e-commerce.

For transparency, this is how the pipeline above maps onto Vatdi, stated plainly.

• Ingest: native catalogue sync through plugins for WooCommerce, OpenCart 3 and 4, PrestaShop, Magento 2, Shopware 6, Joomla and Drupal Commerce; website crawl and pasted URLs; PDF, DOC, DOCX, CSV, TXT and Markdown uploads up to 10 MB each; manual text; FAQ import from CSV or Excel. On Shopify the app indexes pages, collections and policies plus the dashboard knowledge base.
• Retrieve: hybrid search, vector plus keyword, over the store's own content; catalogue results carry price, sale price and stock so the answer and the cards reflect them.
• Generate: OpenAI GPT-4o-mini writes the reply from the retrieved snippets in the visitor's language (95+ languages, auto-detected). Requests go to OpenAI via API and are not used to train models.
• Show sources: product cards with image, price, sale price while a sale is on, link and optional stock.
• Refuse when unsupported: when retrieval finds nothing relevant the bot says it does not know and can offer a human, rather than guessing.
• Isolation: per-store data isolation, credentials encrypted at rest, details on the trust page.
• Evaluation: every conversation is graded 0–10 with plain-English advice on what to fix, and per-answer thumbs and a 1–5 visitor rating feed the same reports.

Limits that matter for RAG: knowledge items are capped at 100 on Free, 500 on Starter and unlimited on Grow; synced products at 200, 2,000 and 10,000. Every plan has the full pipeline; see pricing and features. Vatdi does not offer an API or webhooks, so it is a hosted RAG chatbot for a website, not a framework for building your own.