A chatbot can sound helpful while giving an answer your business cannot use. If customers or employees need answers from your policies, service documents or internal guidance, the first question is where those answers will come from.
Retrieval-augmented generation, usually called RAG, is one way to ground an AI response in selected source material. It is useful for some knowledge problems, but a reliable assistant needs more than a document upload.
RAG and chatbots solve different parts of the problem
A chatbot is the conversational interface. RAG is a method that can work behind that interface: retrieve relevant material, provide it as context to a language model, and generate an answer using that context. They are not competing product categories. A chatbot can use RAG, predefined rules, live API data, or a combination.
A simple guided chatbot may be enough for a short enquiry form or a fixed set of opening-hours questions. RAG becomes worth assessing when the answer depends on a larger collection of approved documents that changes over time. It does not guarantee that every generated answer will be correct.
Choose the smallest approach that fits the question
| Business question | Useful starting point |
|---|---|
| What information should a new lead provide? | A guided form or rules-based conversation. |
| What does our cancellation policy say? | Retrieval from approved, current policy documents, with a source reference. |
| Is this room available on these dates? | An authorized booking-system lookup; a document index alone cannot establish live availability. |
| Can you approve an exception? | A human review step with the relevant context. |
For example, a travel agency could use approved package documents to explain inclusions and a booking API to check availability. Those are separate information paths. A knowledge answer should never silently become a promise of availability or price. This is an illustrative workflow, not a claim about a deployed client system.
Prepare the knowledge before connecting the model
Start with a small collection that has a clear owner. Remove obsolete versions, label effective dates, and resolve contradictions. Scanned PDFs may need text extraction; large files need meaningful sections so retrieval can return a relevant passage rather than an entire manual.
- Define the source of truth: decide which policy or system wins when information conflicts.
- Preserve context: keep titles, section names, dates and source links with retrieved passages.
- Plan updates: specify how revised and deleted documents leave the searchable index.
- Assign ownership: somebody must review content gaps and unanswered questions after launch.
Permissions and escalation belong in the design
An employee assistant may need different answers for different roles. Enforce access restrictions when retrieving content, rather than relying on a prompt to hide restricted text after it has already reached the model. A customer-facing assistant should search only material approved for that audience.
Source documents and user messages should also be treated as untrusted input. Retrieved instructions must not gain authority to change system behavior or authorize external actions. Keep tools and actions behind explicit application permissions.
When a source is missing, conflicting or too weak, the assistant needs an honest fallback and a route to a person. Showing a citation helps a user inspect an answer, but the citation must actually support it. Displaying a link by itself is not evidence of correctness.
Evaluate a focused pilot before expanding
Collect representative questions from the team, including ambiguous questions and requests outside the allowed scope. Record the expected source and acceptable answer. Test retrieval separately from the written response: a fluent summary cannot repair a missing or unauthorized source.
- Choose one audience and one document collection.
- Agree which questions the assistant may answer and which it must escalate.
- Test factual support, citation relevance, permission boundaries, latency and operating cost.
- Review failures with the person who owns the source material.
- Expand only after the pilot meets the acceptance criteria agreed for that workflow.
Measure useful outcomes such as correctly resolved questions and time to reach a human when needed. Conversation count alone does not show whether the assistant helped the business.
What to bring to a discovery conversation
Bring sample questions, approved documents, the intended audience and any systems the assistant needs to read. Also identify sensitive content, the person responsible for updates and the situations that require human approval. These inputs make an assessment more useful than choosing a model first.
Explore our AI automation services and AI knowledge assistant approach, or read about state and review in agent workflows.
Further reading
Microsoft’s overview of retrieval-augmented generation explains the grounding pattern and retrieval challenges. The planning recommendations above are Delphro’s guidance; the right implementation depends on the business and its data.
