RAG (Retrieval-Augmented Generation)

RAG stands for Retrieval-Augmented Generation. Before generating an answer, the application searches a connected knowledge base for relevant content.

The retrieved passages are provided to the language model as context alongside the request. The model formulates its response using this additional material.

How RAG works technically

In a typical RAG setup, documents are first divided into suitable passages. These may come from wikis, manuals or issue trackers.

The passages are often stored as embeddings in a vector database and supplemented with the required metadata.

When a request arrives, the system searches for relevant passages. It may compare the request embedding with stored content and supplement the search with filters or additional ranking methods.

The selected passages are passed to the language model as context. Ideally, the application also identifies the sources used.

Why RAG matters in practice

Internal documents and current product information are often absent from the base model. RAG can provide this material at runtime without changing model parameters.

The knowledge base can be updated independently of the model. For changing factual knowledge, this approach is generally easier to maintain and trace than fine-tuning.

Where the technique runs into limits

RAG can reduce hallucination, but cannot eliminate it. Irrelevant or contradictory results may still lead to incorrect conclusions.

Quality depends on document preparation, retrieval and result selection. A robust system also needs rules for outdated content, conflicting sources and traceable attribution.

For sensitive documents, assess which data is sent to the embedding model, vector database and language model.

Contracts, data classification and technical safeguards determine whether an external service, controlled cloud environment or privately operated infrastructure is appropriate.