RAG: How AI makes your company's knowledge usable

Large language models know an astonishing amount—just nothing about your company. No model knows your offers, your manuals, or the decision from the last meeting. And when it has to guess, the answer often still sounds convincing. The data protection conference describes this phenomenon precisely:

"AI hallucinations occur when AI models generate information that sounds plausible but is not supported by their training data."

Retrieval-Augmented Generation, or RAG for short, is the established answer to this. Instead of letting the model answer from memory, the appropriate excerpt from your own knowledge base is provided with every question. The model then formulates an answer.on the basis of these documents— not from the murky sea of its training data.

How RAG works

The term originates from aFoundational paper by Lewis et al. (2020); the authors observed that RAG models generate "more specific, diverse, and factually accurate" language. The process consists of five steps:

1. Processing— Your documents are broken down into small sections ("chunks") and stored as vectors (embeddings) in a vector database.

2. Retrieval— for a question, the system identifies the semantically closest passages — not by keyword, but by meaning.

3. Enrichment— the passages found will be attached to the prompt.

4. Generation— the LLM formulates the answer based on these pieces of evidence.

5. Receipt— the sources are provided, so that every statement can be verified.

Why RAG is the key for companies

RAG solves three problems at once:

• Up-to-dateness without retraining— new documents simply land in the knowledge base; the model itself does not need to be retrained.

• Fewer hallucinations — grounding in actual evidence corresponds to the principle of accuracy (Art. 5 Para. 1 lit. d GDPR). It remains important: RAG is, according to the DSK, "one of several mitigating measures" — not a guarantee.

• Traceability — source citations turn a black box into a verifiable answer.

The data protection advantage — and the pitfalls

The DSK RAG Guidance (October 2025) identifies an often overlooked advantage: reference data can be deleted, updated, and provided as information — unlike knowledge that has been firmly trained into a model. This allows a company to uphold data subject rights. And, also DSK:

"This can avoid the transfer of personal data to online operators of large language models."

Two points are central, however. First: access rights belong at the level of the knowledge base, not in the model. An LLM cannot control itself who is allowed to see which passage — the authorization must apply before the query. Second: RAG does not cure an unlawful model. Anyone using a model that was trained on unlawfully collected data does not make it lawful through RAG.

Conclusion

RAG transforms a general language model into an assistant that knows your company — with verifiable answers and without handing over your data to external providers. It is the bridge between the power of large models and the real requirements for data protection and reliability. Which criteria a platform must fulfill for this is shown in our post GDPR-compliant AI: the 7 selection criteria; the overall overview is provided by the Guide to the AI Platform for Medium-Sized Businesses.

Sources

• DSK — RAG Guidance (2025)

• DSK — Press release on the RAG Guidance

• Lewis et al. — Retrieval-Augmented Generation (2020)

• AWS — What is RAG?