Retrieval-augmented generation (RAG)

Retrieval-augmented generation, usually shortened to RAG, is answering from a selected body of documents rather than from training alone. The quality of the answer depends on which documents were selected, which makes the corpus the real design decision.

Key takeaways

A RAG system retrieves passages first, then generates an answer from them.
The quality of the answers depends on which files go in.
A vector database stores the passages a RAG system retrieves.

What is an example of RAG?

A support team loads its product manuals and resolved support tickets into a RAG system. When an agent asks how to reset a device, the system finds the matching manual section and ticket, and the model writes an answer from them, with links to the sources. If last year’s manual is still in the set, the answer may describe a button that no longer exists, which is why source selection matters.

What is the difference between RAG and a large language model?

A large language model, or LLM, answers from what it learned during training, which stops at a certain date and does not include an organization’s own files. RAG adds a step before the answer: it looks up the most relevant passages in a selected set of documents and gives them to the model, so the answer is based on current, internal information and can point to its sources. RAG does not replace the model. It decides what the model reads before it answers.

What is the difference between RAG and fine-tuning?

Fine-tuning retrains a model on an organization’s data, so the knowledge becomes part of the model itself. RAG leaves the model unchanged and looks up documents each time a question is asked. RAG is usually faster to set up, easier to keep current, since new documents only need to be added to the set, and easier to control, since removing a document removes it from answers. Fine-tuning suits teaching a model a style or a specialized task, rather than facts that change.

What is RAG data preparation?

Retrieval-augmented generation (RAG) lets a model answer from a selected body of documents, called the corpus, rather than from training alone. Preparation is the work of choosing that corpus: selecting current and authoritative files, excluding superseded versions and restricted material, and refreshing the selection as the estate changes. Most of the effort is in the exclusions.

What is a vector database, and where do files fit?

A vector database stores numeric representations of text or images, called embeddings, so a system can find passages by meaning rather than by exact words. RAG systems use it to retrieve relevant passages before generating an answer. Files are the source: documents are split into passages, converted to embeddings, and loaded into the vector database. The quality of the answers depends on which files went in, so selecting and describing those files comes first.

Which files should go into a RAG system?

Only files that should be allowed to answer a question. That means current versions rather than drafts and old copies, documents the intended users are allowed to see, and content from owners who can vouch for it. Leaving out duplicates, outdated files, and anything sensitive matters as much as choosing what goes in, because the model will quote whatever it retrieves.

Why does RAG depend on data management?

Because a RAG system can only be as good as the documents it retrieves from. If the source set includes outdated versions, duplicates, drafts, or files the user should not see, the answers will repeat those mistakes, with a confident tone and a link as proof. Data management decides which files are current, who owns them, which ones are sensitive, and when they change, which is exactly what a RAG system needs to know about its sources.

What drives the cost of a RAG system?

Mostly the amount of data it has to handle. Every document is split into passages and turned into embeddings, which costs compute; the embeddings are stored in a vector database, which costs storage; and every change means processing again. Sending a whole file share instead of a curated set multiplies all three, and also adds duplicates and outdated files that make answers worse. In a Snowflake survey, storage and compute costs pushed AI projects over budget at 60% of organizations. Selecting a smaller, current set first is usually the cheapest improvement a RAG project can make.