SHORT ANSWER
RAG (retrieval-augmented generation) is a way to connect a language model to your own documents. Documents are split into chunks, converted into vectors and stored in a database; when a question comes in, the system finds the most relevant chunks, and only those reach the model as context. This grounds answers in company content and allows sources to be cited. RAG makes sense where knowledge is scattered across files and changes often.
- The model is not fine-tuned — documents are attached to the question as context.
- Each answer can point to the file and passage it came from.
- Updating knowledge means replacing a file, not retraining.
- Answer quality depends mainly on document quality and how documents are split.
- Permissions must be enforced during retrieval, not in the prompt text.
In plain language: what happens under the hood
Imagine an assistant who runs to a binder before answering, pulls out the two right pages and speaks only based on those. That is RAG: a search engine attached to a model.
On its own, the model does not know your price list from last week. But if it gets that specific page of the price list, it will answer correctly and show where the information came from.
When it pays off
It pays off when people in the company waste time searching: warranty terms, complaints procedures, a rate for a specific client, a machine manual. One assistant instead of five questions on Slack.
It does not pay off when there are ten documents and they all fit in one PDF. Then a good search tool or a simple FAQ page is enough.
Where it fails
The most common problem is messy files: three versions of a price list, two “final” folders, scans with no text. The assistant will not tidy this up for you — it will reproduce the mess, only faster.
The second problem is permissions. If everyone queries the same index, someone will eventually pull up a contract they should not see. The filter must work at the retrieval stage.
How to start at low cost
Choose one set of documents and one question that comes up every day. Build an assistant just for that, for a week, and measure the time saved. You can expand it later — or discard it without regret.