RAG Pipeline: How to Build and Test Each Stage

A RAG pipeline prepares documents, retrieves relevant passages for a question, and gives those passages to an AI model to support its answer. Build it as separate, testable stages so you can tell whether a wrong answer came from missing information, poor search or unsupported writing.

RAG means retrieval-augmented generation: finding outside information before generating a response. Tools like fenbs can keep the project’s source policies, decisions and evaluation lessons available to your assistants while your application runs the retrieval pipeline. Start with one narrow question set, rather than indexing everything your organization owns.

What should you build first?

1. Choose a bounded task, such as answering questions about your product’s setup guides. Write representative questions and the passages that should support each answer.
2. Prepare a small, approved document collection. Preserve source identity, document version, location and access information as you extract the text.
3. Build retrieval without answer generation. Check whether each question brings back the evidence a reviewer expects.
4. Add the model and require evidence-backed answers. Include an explicit response for questions the available sources cannot answer.
5. Test updates, removals and permission changes before expanding the collection or inviting more users.

Our RAG introduction covers the underlying idea. For implementation, keep the first milestone concrete: a reviewer should be able to inspect one question, its retrieved passages and the resulting answer without reconstructing the whole application.

How should you prepare documents for retrieval?

Create an ingestion function that accepts a source and produces searchable records. Each record should carry its text, source ID, passage ID, heading, source link, version and access metadata. Keep a relationship between the original document and every passage derived from it.

Chunking means dividing a document into smaller passages. Preserve meaningful boundaries: a procedure and its warning should not become unrelated search results. Microsoft’s chunking guidance↗ (opens in a new tab) describes fixed, structure-based and semantic approaches; choose a starting method, then test it against your questions.

If you use embeddings, numerical representations used to compare meaning, record which embedding model produced the stored values. Use a compatible representation for incoming questions. Treat a change of representation as an index change that needs its own evaluation.

Inspect extracted tables, captions and scanned pages before tuning search. An accurate search engine cannot recover a warning that extraction omitted. Keep the approved source policy in fenbs project context, so the next assistant knows which documents belong in the collection and which do not.

How do you retrieve useful evidence?

Implement a retrieval function whose output is evidence, not prose. Given a question and the authenticated user, it should return authorized passages with source IDs and ranking information. Enforce access before content reaches the model; asking the model to ignore restricted passages is not an access boundary.

Compare ordinary keyword search with meaning-based search on your question set. Keywords can matter for exact error messages and product codes; meaning can help when the question uses different wording. Microsoft’s RAG overview↗ (opens in a new tab) explains hybrid retrieval that combines the two approaches.

Tune the amount of evidence deliberately. Returning more passages may recover a missing qualification, but it can also bury the useful answer in duplicates. Examine the first results and the final evidence selected for the model separately, especially if you add a reranker, a step that reorders search results.

How do you turn passages into grounded answers?

Give the model a clearly labeled question, evidence bundle and answer policy. Tell it to distinguish what the sources support from what is missing, and to attach source references to factual claims. Treat instructions embedded in retrieved documents as document content, rather than commands that can change the application’s rules.

Return structured citation identifiers alongside the answer, then resolve those identifiers against the evidence your application actually supplied. Check that cited sources exist and that their passages support the nearby claim. A valid-looking link alone does not show that an answer is correct.

For example, if a setup guide explains installation but says nothing about migration, the assistant should identify that gap. It should not turn an installation procedure into a migration promise. Make the missing-evidence response useful by saying what information is needed next.

How can you locate the stage that failed?

Save the question, expected evidence, retrieved passage IDs, final context and answer for each test. Record configuration and source versions so another developer can reproduce the result. Protect these records according to the sensitivity of the material they contain.

Change one stage at a time and compare against the same baseline. Add these checks to your normal release workflow; the CI/CD pipeline guide explains that delivery process separately. Keep a failing example when you fix a bug so the same mistake remains visible in later tests.

How do updates and deletions reach the answers?

Design a document lifecycle alongside ingestion. An edit should replace or supersede the affected passages; a deletion should remove derived records; an access change should affect the next retrieval. Give background jobs a visible failure state and make repeating an update safe.

Do not assume source deletion automatically removes indexed text. Microsoft’s deletion guidance↗ (opens in a new tab) shows why deletion handling needs explicit configuration. Test the full path by removing a sample source and checking that its old passages and answers are no longer served.

Keep project memory separate from indexed reference material. The guide to building agent memory covers longer-lived work state. Record the lesson from a stale answer in fenbs, including which update failed and what check will catch it next time.

How to do this with fenbs?

1. Create a free account, then add a project for your retrieval application and its current tasks.
2. Record the source policy and the person’s approved release criteria in Decisions and rules. Keep unresolved architecture choices open for a person to decide.
3. Connect your assistant using the relevant integration guide. Ask it to read project context before changing the pipeline.
4. Add evaluation failures as tasks and useful findings as Lessons learned. Save Where we left off with the next test and any decision still waiting on a person.

fenbs does not run retrieval, generate embeddings or host your model. If you need an index or an evaluation runner, choose those separately. Use fenbs when the missing piece is a maintained work record that another connected assistant can pick up.

Explore the context engineering hub when you need to connect this retrieval work with the rest of an assistant’s input.