RAG vs CAG: Should You Retrieve Sources or Cache Context?

RAG, retrieval-augmented generation, searches external sources for material relevant to each question. CAG, cache-augmented generation, prepares a manageable body of source material in the model's context and reuses the work of processing it across questions. RAG suits a broad or changing collection; CAG is worth testing when the same bounded sources support many questions.

Neither approach automatically knows what your team approved or where yesterday's work stopped. A shared work record such as fenbs can hold that current context, but your assistant must deliberately read it. The architectural choice is how to supply evidence, followed by how to keep evidence and work state current.

What is the practical difference?

These are starting points for testing, not universal performance rankings. A small source set can still contain conflicting instructions, and a large collection may have simple questions that retrieval answers well. For the broader vocabulary, see the context engineering guides.

How does each approach get evidence into an answer?

In a typical RAG application, software searches a collection and passes selected passages to a model with the question. The model writes an answer from that supplied material. The original RAG research↗ (opens in a new tab) combined a language model with a searchable external index; practical systems can use different search methods and source stores.

Imagine an assistant answering questions about product installation guides. It searches for the relevant product and installation condition, then supplies the matching sections. If it retrieves the standard installation but misses the exception for an older model, the answer may be fluent and wrong. Inspecting the selected evidence is part of diagnosing that failure.

The CAG paper↗ (opens in a new tab) explores preloading relevant documents and preparing a key-value cache, the model's reusable internal processing state. Questions then use that prepared context without a new document search. Its central premise is a knowledge set small enough to handle together, rather than an unlimited library placed in a cache.

For the installation example, a CAG package might contain the complete approved guide for one product family. Selection still happened: someone chose that family, its revisions, and its supporting material. CAG moves selection earlier; it does not remove the responsibility to decide which evidence belongs. The RAG introduction explains the retrieval stages in more detail.

When is the whole source set manageable enough?

Start with the smallest complete evidence set for the task. Count more than the source files: your application also needs instructions, the question, any conversation history, and space for the answer. A context window is the amount of material a model can handle in one interaction, and merely fitting within it does not prove reliable understanding.

Include the hardest legitimate question in that check. An assistant may answer a simple definition from a long specification while missing an exception that appears far away. Ask for a comparison, an exception, and a case where two documents disagree. Keep the source identifiers visible so you can tell whether the answer used the controlling passage.

If the complete package underperforms selected evidence, making its cache more efficient will not fix the reading problem. That is a reason to narrow or reorganize the supplied material and rerun the questions. The useful limit is the source set your application handles reliably, not just the amount it accepts.

Say a small design studio wants an assistant for one approved website specification. If the specification and its exceptions form a compact, stable package, supplying them together is a sensible baseline. If the studio wants questions answered across every client, proposal, and revision, selecting relevant material becomes a larger part of the job.

• Try prepared context when the scope has a clear boundary and questions repeatedly use the same sources.
• Favor retrieval when a question should select a small portion of a much larger collection.
• Test both when questions depend on scattered details, exceptions, or comparisons across documents.
• Reduce the scope when neither approach can identify which source version should control the answer.

A document package describes the work, while a current decision says which part now applies. Keep the approved scope and unresolved questions in fenbs project context so the next assistant can read the same boundaries before using either approach.

Is provider prompt caching the same thing as CAG?

No. CAG describes a way to answer from prepared knowledge without a fresh retrieval step. Prompt caching is a provider capability that reuses processing for repeated input. You can use caching with a CAG-style package, with the stable instructions around a RAG application, or with another repeated prompt entirely.

For example, Anthropic's prompt-caching documentation↗ (opens in a new tab) describes exact matching of reusable prompt segments and recommends placing stable content before changing content. Cache requirements, lifetimes, supported models, and usage reporting belong to the provider's documentation. Do not assume a saved document remains available indefinitely because one request used a cache.

A cache hit is evidence that processing was reused, not evidence that an answer is correct. A cache miss is not necessarily a quality failure: the application may process the same complete sources again. Treat reuse, answer accuracy, and source freshness as separate measurements so a useful optimization cannot hide an outdated source package.

How should updates and permissions change the design?

Define an update rule before choosing either architecture. For RAG, ask how a corrected source reaches search results and how deleted material disappears. For CAG, ask which source changes require a new package and cache. Give packages or searchable snapshots a version that you can record during testing and identify later.

A useful test is to change an approved requirement and repeat the old question. The answer should use the replacement, explain any unresolved conflict, and stop presenting the retired rule as current. Include a deletion test as well: removing a source from the original folder is insufficient if a searchable or cached copy remains usable.

Apply access rules before material reaches the model. A shared context package must contain only information every intended reader may use. When access varies by client, team, or document, consider separate authorized packages or retrieval that enforces the caller's access. Asking the model to conceal an unauthorized passage after receiving it is the wrong boundary.

Which approach should you choose after testing?

Compare the same questions against the same approved source versions. Include direct lookups, exceptions, questions needing two documents, and questions the sources cannot answer. Add at least one outdated-source trap and one question from a reader with restricted access. You need to know whether the system can decline as well as whether it can answer.

1. Write down the expected evidence and acceptable answer before running either approach.
2. Check whether the necessary passages reached the model and whether the answer actually follows them.
3. Measure complete response time, including preparation or refresh work when it applies.
4. Compare first-use requests with repeated requests and record whether caching was used.
5. Repeat the test after a source correction and an access change.

Choose the simpler setup that meets your requirements on those tests. Prepared context can remove a retrieval failure point, while retrieval can avoid sending an entire collection. A combined design may keep stable instructions reusable and retrieve changing evidence. Neither label is a reason to accept unsupported answers or skip source maintenance.

Keep learning from a session separate from document retrieval; agent memory vs RAG covers that distinction. Save the tested scope and remaining work in fenbs so another assistant can continue without treating an experimental result as an approved decision.

How to keep this work current with fenbs?

fenbs holds the reviewed work record around your knowledge system. It does not run a retrieval pipeline, provide vector search, or manage your model provider's cache. If you only need to ask questions about a single document, a document assistant may be enough; fenbs helps when decisions, tasks, and progress need to survive across sessions.

1. Create a free account, then add a project for the knowledge task.
2. Record the scope and what a person approved in Decisions and rules; keep uncertain findings in context notes.
3. Add source locations to Links and listings, with status and dates that identify the material you reviewed.
4. Connect your assistant using an integration guide, and ask it to read project context before working.
5. Track failed questions as tasks, then save the test result and Where we left off before ending the session.