An AI customer service bot earns its place on the contacts that have an answer in your content or your order data, and loses trust the moment it improvises on the rest. That line is the whole design. Below: what a retrieval-based bot handles well, what it must hand to a person and how, the ten instructions that keep it honest, what our own conversation data shows about the split, and the numbers to watch each week.
What "AI customer service bot" means here
Not a scripted menu, and not a general chat model let loose on your site. The kind that works for service is retrieval-based: for each message it searches your synced products, pages, documents and (where connected) order records for the passages that match, writes an answer from them, and says it does not know when nothing matches. The method is described in Lewis et al., 2020; the practical version is on Vatdi's AI chatbot for customer support page. Everything below assumes that kind of bot; a menu bot cannot do the first half, and a general model cannot be trusted with the second.
What it handles well
| Contact | Where the answer lives | What the bot does |
|---|---|---|
| Where is my order? | The order record | Asks for order number + email or phone, verifies both, returns the live status |
| Shipping time, cost, destinations | Shipping page, zones table | Quotes the range and price for the destination, in the shopper's language |
| Returns and exchange rules | Returns page | States the window and conditions; hands over if the shopper wants to start a return that needs a decision |
| Product fit, compatibility, stock | Catalogue attributes and variants | Answers from the fields and shows the product card |
| Payment methods, hours, location | FAQ and contact pages | Quotes the page |
| Warranty terms | Warranty page | States the terms; a claim goes to a person |
| "How do I…" questions | Help articles, PDFs | Gives the steps and the link |
In our data from 884 real store conversations over 90 days (method in our conversation statistics), order status and shipping were the two largest topic groups, and both have answers in data rather than judgement; that is why they are the first things to connect.
What it should hand over, every time
- Refunds already in progress and anything about money that has moved.
- Damaged, missing or wrong items. The bot can confirm the order and collect photos or details; a person decides.
- Payment disputes and chargebacks.
- Complaints and anyone who is upset. A bot can acknowledge; it cannot resolve.
- Any explicit request for a person, without a second bot attempt.
- Anything the content does not cover. "I don't know, let me get someone" beats a plausible guess every time.
The failure that costs stores customers is not the bot declining too often; it is the bot attempting one of these. Set them as explicit rules before launch.
How the handover should work
Four properties matter: the trigger is recognised on the first request; the bot tells the customer what happens next and when; the conversation moves to a team inbox with the full transcript; and a person replies in the same thread. Outside agent hours the widget says when someone will reply and collects contact details rather than promising an immediate answer. Vatdi's inbox uses the states Needs you, Handling, Team handling, Missed and Resolved, with email and browser alerts and unlimited teammates; details in what chatbot human handover is and the team inbox.
Ten instructions that keep a service bot honest
- Answer only from the store's content and order data; never guess a price, date or policy.
- When the content does not cover it, say so and offer a person.
- Require the order number plus the email or phone on the order before showing any order detail.
- Never ask for card numbers, passwords or identity documents.
- Give delivery times as the range on the shipping page, never a specific date.
- Offer a coupon only when the conversation calls for it, and always with its conditions.
- Escalate refunds in progress, damaged goods, disputes and complaints immediately.
- On any request for a person, hand over without a second attempt.
- Keep product names as written; do not translate them.
- State the agent hours honestly when nobody is available, and collect contact details.
These are content and settings, not code; on Vatdi they are the assistant's instructions plus the handover and agent-hours settings.
What our own data shows about the split
In our data, 3.8% of conversations included a request for a person and 2.3% were assigned to one, and the average conversation grade from our own strict judge was 5.41 out of 10, with low scores traced overwhelmingly to content gaps rather than model errors. Read together: most store questions had an answer in the store's content, the bot rarely needed to hand over, and where it fell short the fix was a missing or unclear page. That is the honest shape of a retrieval-based service bot on small stores, and the reason the weekly review below is about content.
The numbers to watch weekly
| Number | Healthy pattern | Warning sign |
|---|---|---|
| Answered without a person | Rises as content improves | Rises while the rating falls: the bot is stonewalling |
| Handover requests vs assigned | Close to equal | A gap: requests arriving outside hours or not picked up |
| Missed handovers | Near zero | Each one is a customer who asked for you and did not get you |
| Visitor rating and thumbs | Stable or rising with volume | Low ratings clustered on one topic: a page to fix |
| Conversation grade | Low scores becoming next week's content edits | The same low scores three weeks running |
Vatdi grades every conversation 0–10 with plain-English advice and shows these numbers on every plan; the wider set is in AI chatbot KPIs to track, and the deflection side specifically in AI chatbot for ticket deflection.
Setting it up, in order
- Connect the order system and the catalogue through the plugin (WooCommerce, OpenCart, PrestaShop, Magento, Shopware, Joomla, Drupal; on Shopify, pages and policies only).
- Rewrite the shipping and returns pages one question per heading, with the number in the first sentence.
- Enter the ten instructions above and set agent hours.
- Put "Talk to a person" in the quick replies.
- Run twenty real questions and five unanswerable ones; fix the content behind every miss.
- Review low-graded conversations weekly for a month, then monthly.
Choosing between tools for this job is covered in the customer-service scoring method; the split between bot and live chat in chatbot vs live chat; and the honest limits in can an AI chatbot replace customer service. Everything here is on every Vatdi plan, including Free (pricing); check the widget for keyboard and screen-reader access as well, since a service channel must work for everyone (the W3C's accessibility fundamentals are the baseline).
Frequently asked questions
What can an AI customer service bot handle without a human?
Anything with an answer in your content or order data: order status (with verification), shipping times and costs, returns and warranty rules, product fit, compatibility and stock, payment methods, hours and how-to steps. In our data those categories made up most store conversations. It should not attempt refunds in progress, damaged goods, disputes, complaints or any request for a person; those are handover triggers.
How does the bot know when to hand over?
Through explicit rules you set: topics that always escalate (refunds in progress, damaged or missing items, payment disputes, complaints), any request for a person, and any question the content does not cover, where the bot says it does not know and offers someone. Good tools also let you set agent hours so the handover message is honest about when a reply will come.
Will customers be annoyed by a bot before reaching a person?
They are annoyed by bots that guess, loop or hide the exit, not by bots that answer the shipping question in seconds and hand over on the first request. Put "Talk to a person" in the quick replies, escalate immediately on the trigger topics, and never make the bot try twice. Watch the visitor rating weekly rather than assuming either way.
Is an AI customer service bot safe with order and personal data?
It should require two facts the customer already knows, the order number plus the email or phone on the order, before showing any detail, and it should never ask for card numbers, passwords or identity documents. Beyond that, check the vendor's data handling: encryption, retention, sub-processors, export and deletion. Vatdi documents its practices on its trust page and rate-limits order lookups.
What happens to the bot's mistakes?
They become content edits. On a retrieval-based bot, a wrong answer almost always traces to a missing, stale or unclear page, so the fix is to correct the page and re-ask the question the same day. Vatdi grades every conversation 0–10 with advice on what to fix, which turns the weekly review into a short list rather than a hunt through transcripts.
Does this need a paid plan?
Not on Vatdi: answers from your content, order lookup through the plugins, handover with agent hours, the team inbox, conversation grading and reports are on every plan, including Free with 15 conversations a month. As of September 2026 the paid plans ($4.49 for 150 conversations, $7.49 unlimited) differ only in the conversation cap and badge removal.
