RAG Tutorial: Build a Production-Ready Retrieval-Augmented Generation Pipeline

Overview

Retrieval-augmented generation, or RAG, separates an AI application’s knowledge from the language model itself. Instead of asking a model to answer from its training data or a long unstructured prompt, the application retrieves relevant passages from a controlled collection and supplies them as context.

A dependable RAG system has two related workflows:

1. Indexing: collect documents, extract text, divide it into useful passages, create embeddings, and store the passages with searchable metadata.
2. 回答 generation: interpret a user question, retrieve and optionally rerank relevant passages, construct a constrained prompt, and return an answer with citations or an explicit indication that the evidence is insufficient.

The model is only one part of the application. Retrieval quality, document freshness, access control, prompt design, latency, and observability all affect the result. Treat each handoff as a testable interface rather than hiding the entire process behind one model call.

Before implementation, define the intended answer types. A support assistant may need short answers with source links, while an internal search tool may need passages and metadata more than fluent prose. This decision influences chunking, retrieval, citation format, and evaluation.

Step-by-step workflow

1. Ingest and normalize source documents

Start with the documents the application is allowed to use: help pages, policies, technical manuals, tickets, or structured records. Preserve the source identifier, title, URL or path, section heading, document version, timestamp, and permission information.

Normalize the extracted text before creating chunks. Remove navigation elements, repeated footers, broken encoding, and irrelevant markup, but do not discard headings or list structure. Tables require special care: convert them into a representation that preserves row and column relationships instead of flattening values into ambiguous text.

Make ingestion repeatable. A scheduled or event-driven job should be able to identify new, changed, and deleted documents without rebuilding the entire index unnecessarily. Store a content hash or equivalent change marker so that updates can be traced.

2. Choose a chunking strategy

Chunking determines what the retriever can return as evidence. Begin with document structure: split at headings, paragraphs, list boundaries, or logical sections where possible. Only then apply length limits to unusually large sections. Keep a small amount of overlap when a concept may continue across boundaries, but avoid repeating large portions of every document.

Each chunk should be understandable when shown on its own. Include useful context in metadata or in the chunk text, such as the document title and section path. For a deeper comparison of size, overlap, and retrieval trade-offs, see RAG chunking strategies compared.

3. Create embeddings and store searchable records

Generate an embedding for each chunk using a model appropriate to the language and content of the collection. Store the vector alongside the original text and metadata. Keep the embedding model identifier and preprocessing version with the record; changing either can make old and new vectors difficult to compare.

A vector index is useful for semantic similarity, but it is not always sufficient. Exact terms such as product codes, error messages, or policy identifiers may benefit from lexical search. A hybrid retriever can combine semantic and keyword signals when the collection contains both natural-language questions and precise technical terms.

4. Retrieve candidates for each question

At query time, normalize the question only as much as needed. Preserve important identifiers, filters, and user intent. Retrieve a candidate set larger than the final context allowance, then apply metadata filters such as tenant, department, product, date, or document permissions before the model sees any text.

Reranking can improve ordering when the initial search returns broadly related passages. A reranker evaluates the question and each candidate together, allowing the application to pass only the strongest evidence to the generation step. The right number of candidates depends on the document domain, context budget, and latency target, so measure it rather than selecting a universal value.

5. Construct a grounded generation prompt

Give the model a clear task, the retrieved context, and rules for uncertainty. A useful instruction might require the model to answer only from the supplied evidence, distinguish direct evidence from inference, cite the source identifier for each material claim, and say that the information is unavailable when the context does not support an answer.

Keep retrieved content clearly separated from instructions. Treat documents as data, not as authority to change the application’s behavior. Define a stable output shape when downstream code needs one, such as an answer, citation list, confidence label, and refusal reason. The system prompt examples for reliable AI assistants provide related guidance on instruction boundaries and predictable behavior.

6. Return citations and trace data

A citation should lead the user to a source or stable document location, not merely expose an internal chunk identifier. At the same time, retain internal trace data for debugging: query text, applied filters, retrieved chunk IDs, scores, reranking order, prompt version, model version, latency, and token usage where available and appropriate.

Tools and handoffs

A maintainable implementation can be organized into five replaceable components:

• Ingestion worker: fetches documents, extracts content, records versions, and sends normalized records onward.
• Chunking and embedding job: produces chunks and vectors with explicit processing-version metadata.
• Search service: applies authorization filters, runs semantic or hybrid retrieval, and optionally reranks candidates.
• Generation service: assembles the prompt, calls the language model, validates structured output, and formats citations.
• Evaluation and monitoring layer: stores test results and operational traces without exposing sensitive content unnecessarily.

Use clear contracts between these components. For example, a chunk record might contain text, source_id, title, section, permissions, updated_at, and processing_version. A retrieval result should include the query, filters, chunk identifiers, scores, and source metadata. These contracts make it easier to replace a vector database, embedding model, or reranker without rewriting the entire application.

Keep prompt templates and retrieval settings under version control. Log which versions produced an answer so that a quality change can be connected to a specific deployment. Reliability, cost, and latency should be reviewed together; the LLM application reliability checklist covers the broader operational concerns.

Quality checks

Test retrieval separately from generation. Build a small, representative question set with expected source documents or passages. For each question, check whether the correct evidence appears in the candidate results and whether irrelevant or unauthorized content is excluded. Include questions that should produce no answer.

Then evaluate the complete response. Review factual support, citation accuracy, completeness, instruction following, formatting, and refusal behavior. A fluent answer can still be wrong if the retriever selected the wrong section or if the prompt encouraged unsupported inference.

Include adversarial and operational cases: ambiguous terms, outdated documents, conflicting versions, long questions, prompt injection inside a source document, empty search results, model timeouts, and malformed structured output. Test authorization at retrieval time and again at the source-linking layer where appropriate.

Track quality by document type and question category rather than relying only on one aggregate score. Also measure practical signals such as ingestion failures, stale records, retrieval latency, generation latency, context size, and cost per workflow. See how to benchmark LLM latency for a methodical approach to timing different AI workflows.

When to revisit

Revisit the pipeline whenever its inputs, dependencies, or user expectations change. That includes adding a new document source, changing parsing logic, updating the embedding or language model, modifying chunk size, introducing a reranker, changing permission rules, or altering the citation format. A model upgrade is not automatically an improvement for your particular retrieval set, so rerun the same evaluation questions before and after the change.

Set a review routine for document freshness and failed ingestion jobs. Investigate user reports that cite missing evidence, outdated answers, or irrelevant sources. If questions are consistently missed, inspect the failure stage: the content may not have been indexed, the chunk may be poorly formed, the query may need expansion, the filter may be too strict, or the generation prompt may be misusing correct evidence.

For the next implementation step, create a small end-to-end slice: one trusted document source, versioned ingestion, structure-aware chunks, searchable metadata, retrieval traces, citations, and a written evaluation set. Ship that slice only after testing both supported answers and explicit no-answer cases. Then expand the corpus and capabilities one handoff at a time, preserving the tests that explain whether each change makes the RAG application more useful and more reliable.