AI customer service is good at the routine stuff: opening hours, delivery areas, order status, “do you have this in stock?”. Where it goes wrong is when it doesn’t know what it doesn’t know, and keeps answering when a person should have stepped in. This playbook covers how to set up AI customer service that answers well, hands off at the right moment, and gives your team enough context to pick up without asking the customer to repeat themselves.
Why the handoff matters more than the answers
Most people judge an AI assistant by how clever its answers are. Customers judge it by what happens when it can’t help.
A good handoff means:
- The AI recognises that a conversation needs a person.
- It tells the customer, honestly, what happens next.
- It passes a short summary to the right teammate, so nobody has to scroll back through twenty messages.
A bad handoff is an AI that keeps guessing, or a customer who explains their problem twice. Both cost you more than the routine questions the AI saved.
Rules matter here too. WhatsApp’s Business Messaging Policy says businesses using automation must have prompt, clear and direct escalation paths, such as transferring to a live agent. Meta’s Messenger Platform and IG Messaging API policy covers disclosing automated experiences. A handoff isn’t just good service, it’s part of using these channels properly.
Step 1: Build a knowledge base the AI can actually use
An AI assistant is only as good as what you give it. Start with the questions you already answer every day.
What to include:
- Your FAQs, written as questions and answers
- Policies: delivery, returns, refunds, cancellations
- Product or menu details, with prices
- Opening hours, locations and contact details
- Documents you already have, like a delivery zones PDF or a price list
- Your website, if it’s up to date
How to write it so the AI gets it right:
- One fact, one place. If your delivery cut-off is in three documents with three different times, the AI will pick one at random. Fix the source.
- Be specific. “We deliver to Riverside until 7 pm, free over $15” is better than “We deliver to most nearby areas.”
- Date anything that changes. Seasonal menus, holiday hours and promotions go stale fast.
- Write down what you don’t do. “We don’t ship outside the city” stops the AI promising something you can’t deliver.
For Kettle & Co., the most useful single document turned out to be a one-page delivery zones PDF. It answered the question customers ask most (“Do you deliver to Riverside today?”) with areas, times and prices in one place.
Step 2: Make the AI show its sources
Every answer should be traceable to something in your knowledge base. That does two things: it lets your team check the answer quickly, and it gives you a clear rule for when the AI should stop.
The rule: if there’s no source, there’s no answer. When the knowledge base doesn’t cover a question, the AI should say it will pass the question to a teammate, not improvise.
In Open Motives, the AI Employee answers from your knowledge base with sources, so a draft shows where it came from: “AI draft · from Delivery zones.pdf”. If the source is wrong, you fix the source, and every future answer improves.
Step 3: Choose draft-for-approval or auto-send
There are two sensible ways to run AI replies, and you can use different modes for different channels or assistants.
| Draft for approval | Auto-send | |
|---|---|---|
| How it works | AI writes the reply, a teammate checks and sends | AI sends the reply directly |
| Best for | New setups, complex products, high-value customers | Well-covered routine questions, out-of-hours replies |
| Speed | As fast as your team approves | Immediate |
| Risk | Low | Depends on your knowledge base and handoff rules |
| What you learn | Exactly where the AI is wrong, from the edits | Where it’s wrong, from reviews and complaints |
Our advice: start with draft for approval, even if you plan to auto-send later. A couple of weeks of drafts shows you exactly where the knowledge base has gaps, and your team builds trust in the answers. Move to auto-send for the topics where drafts almost never need changing.
In Open Motives, each AI Employee is set to either send automatically or draft for approval. Separately, the Copilot in the reply box only fills the box: nothing is sent until someone presses Send.
Step 4: Decide what counts as sensitive
Write down the topics where the AI should always hand off, whatever it thinks it knows. A starter list:
- Complaints and refunds. Someone upset about a late order wants a person.
- Health and safety. For a bakery, that means allergies. “Is this nut-free?” should never get a guess.
- Money and accounts. Payment problems, chargebacks, billing disputes.
- Legal or privacy requests. Data requests, threats of legal action, anything involving a contract.
- Personal circumstances. Bereavement, illness, anything where tone matters more than speed.
- An explicit request for a person. “Can I talk to someone?” should always work, first time.
- Strong negative sentiment. Anger or frustration, even about a routine topic.
Then add your own. Look through past conversations for the ones you’d never want a machine to handle alone.
Open Motives includes workflow templates for two of these: “Escalate unhappy customers” (negative sentiment goes to a person) and “Hand off on request” (for “Can I talk to someone?”).
Step 5: Hand off with a summary
When the AI hands off, the teammate who picks up should be able to reply within a minute. That means a summary, not a transcript.
A good handoff summary includes:
- Who: the customer’s name and channel
- What they want: one sentence
- What’s been said: key facts and anything the AI already told them
- Why it’s handed off: “Allergy question” or “Customer asked for a person”
- What’s needed next: “Confirm whether the cardamom buns contain nuts”
What the customer should hear:
Thanks, Sam. That’s a question for one of our team, so I’ve passed it on with the details. Someone will reply here shortly.
Keep it honest. Don’t promise “in two minutes” if your team is off shift. If you’re closed, say when you’ll be back.
In Open Motives, a sensitive request is detected, the conversation is handed off with an AI summary, and routing sends it to someone on shift. You can also send handoff alerts to Slack or Microsoft Teams.
Step 6: Limit what the AI can do
Answering questions is one thing. Taking actions, such as creating an order, changing a booking or issuing a refund, is another.
Start narrow:
- Let the AI read what it needs: order status, delivery slots, stock.
- Make any write action wait for a teammate’s approval.
- Review the list of allowed tools every few months.
In Open Motives, write actions on connected apps always wait for a teammate, and the approval card shows every detail, so what you approve is exactly what runs.
Step 7: Tell customers they’re talking to AI
Be upfront. A short line at the start (“I’m the Kettle & Co. assistant, an AI. I can help with orders and deliveries, or pass you to the team.”) sets expectations and makes the handoff feel natural rather than like a failure. Meta’s messaging policy requires disclosure where the law does, and recommends it everywhere.
How to measure AI customer service
Track these monthly. None of them need special tools, just consistent counting.
| Metric | What it tells you |
|---|---|
| Handoff rate | Share of AI conversations passed to a person. Too low can mean the AI is guessing. |
| Resolved without handoff | Conversations the AI closed that didn’t come back |
| Reopen rate | AI-closed conversations where the customer came back with the same issue |
| Draft edit rate | How often teammates change AI drafts before sending, and what they change |
| Time to human | How long a handed-off customer waits for a person |
| Customer satisfaction (CSAT) | How customers rate AI-handled versus human-handled chats |
Look at the numbers together. A falling handoff rate is only good news if the reopen rate and CSAT hold steady. Open Motives includes SLA timers and CSAT for support teams, so time to human and satisfaction sit in the same place as the conversations.
A 30-day rollout plan
Week 1: Knowledge. Collect your FAQs, policies and documents. Fix contradictions. Write the sensitive-topics list.
Week 2: Drafts only. Turn on the AI in draft-for-approval mode on one channel. Have the team note every edit they make.
Week 3: Fix and expand. Update the knowledge base based on the edits. Add a second channel, still in draft mode.
Week 4: Auto-send the safe topics. Switch to auto-send for topics where drafts rarely change, such as hours and delivery areas. Keep everything else on drafts. Review your metrics at the end of the month.
Frequently asked questions
Should AI customer service reply automatically?
Only for topics where it has proven reliable. Start in draft-for-approval mode, review the edits your team makes, and switch routine, well-covered topics to auto-send once drafts rarely need changing.
How do I stop an AI assistant making things up?
Give it a clear, consistent knowledge base, require it to answer only from sources, and tell it to hand off when the knowledge base doesn’t cover a question. Review drafts and fix the source whenever an answer is wrong.
When should a chatbot hand off to a human?
Whenever the customer asks for a person, shows strong frustration, or raises a sensitive topic such as complaints, refunds, health and safety, payments or legal matters. It should also hand off whenever it can’t find the answer in its knowledge base.
Which AI model should I use for customer service?
Most current models from major providers handle customer service well, so pick based on cost, speed, language support and data requirements. Open Motives works with 11 AI providers, including OpenAI, Anthropic, Google Gemini and Mistral, using your own key or included credits where a plan has them.
Wrapping up
The difference between AI customer service people tolerate and AI customer service people like is the handoff. Give the AI a tidy knowledge base, make it show its sources, start with drafts, write down what’s sensitive and hand off with a proper summary. Then measure, and keep adjusting.
If you’d like to talk through how this would work for your team, get in touch.