Advanced RAG Engineering

Treat RAG as a retrieval pipeline, not a prompt trick

A production RAG system separates knowledge ingestion from question-time retrieval. During ingestion, source material is parsed, divided into chunks, enriched with metadata, converted into embeddings, and written to a search store. At request time, the application converts the user's information need into one or more searches, retrieves candidates, optionally filters and reranks them, then supplies selected evidence to a foundation model. Every stage can fail independently: a correct answer cannot be grounded in a document that was never parsed, a perfect embedding cannot rescue a chunk that split the required fact in half, and a capable model cannot cite evidence that retrieval did not return.

Amazon Bedrock Knowledge Bases manages much of this flow, but the architecture choices remain visible. The source, parsing strategy, chunking strategy, embedding model, vector store, retrieval configuration, filters, reranking configuration, and generation behavior determine both quality and cost. For AIP-C01, this is the useful mental model: diagnose retrieval before changing the generator, and measure retrieval relevance separately from answer quality.

Chunking and retrieval change what the model can see

Chunk size is a precision-versus-context trade-off. Small chunks can isolate a relevant fact but may remove surrounding definitions or qualifications. Large chunks preserve local context but consume more prompt space and can dilute the relevant passage. Bedrock Knowledge Bases supports strategies including fixed-size, semantic, and hierarchical chunking where supported. Hierarchical chunking creates parent and child relationships so retrieval can find precise child segments while returning broader parent context. A no-chunking strategy can be appropriate when source units are already deliberately sized.

Vector search retrieves semantically similar content. Hybrid search combines semantic and text-oriented signals when supported by the selected store, which can improve queries containing exact names, codes, identifiers, or phrases that embeddings alone may underweight. Metadata filters should eliminate ineligible content before generation—for example tenant, document type, language, validity date, or authorization scope—rather than asking the LLM to ignore material it was never supposed to see.

• Use semantic similarity when wording varies but meaning is related; exact lexical signals can be valuable for identifiers and domain terms.
• Query decomposition can break a multi-part question into smaller retrieval requests before composing a final answer.
• Metadata is part of retrieval design: access and relevance filters should be deterministic whenever possible.

Reranking spends more intelligence on fewer candidates

Initial retrieval is optimized to find plausible candidates efficiently. A reranker can then score those candidates with a model designed to judge query-document relevance and reorder them before context is assembled. Amazon Bedrock exposes reranking both directly and in Knowledge Bases retrieval. This creates a deliberate two-stage pattern: retrieve broadly enough to preserve recall, then rerank a smaller candidate set to improve precision. Reranking adds latency and cost, so it should be justified with retrieval evaluation rather than enabled automatically.

Bedrock Knowledge Bases also exposes the maximum number of source chunks to retrieve. AWS documents that a query returns up to five results by default; `numberOfResults` changes the maximum and hierarchical chunking gives that setting child-chunk semantics. More chunks do not guarantee a better answer: each additional chunk occupies context, can introduce conflicting evidence, and increases generation cost. Tune retrieval count, filters, query decomposition, and reranking against a representative question set.

Production RAG needs freshness, provenance, and evaluation

A knowledge index is a derived representation of source data, so it needs synchronization and lifecycle controls. When documents change, ingestion must update or remove the corresponding chunks and vectors. Metadata should preserve source identity, version, timestamps, and other attributes required for filtering or citation. The source of truth and the vector index have different operational roles; do not let a stale index silently become the authoritative record.

Evaluate retrieval with known queries and expected evidence, then evaluate generation with that evidence. Track cases such as 'relevant document not retrieved', 'relevant chunk ranked too low', 'retrieved evidence was stale', and 'model ignored good evidence'. This classification turns a vague hallucination complaint into an actionable pipeline defect and maps directly to AIP-C01 testing and troubleshooting expectations.