Every Type of RAG Explained: Naive, Advanced, Modular, Graph, Agentic

Most people deploy RAG as if it's a single technique: chunk your documents, embed them, retrieve the top-k at query time, drop them in the prompt. That works well enough to ship — and then stops working the moment your questions get harder than "what does section 3.2 say?"

RAG is not one thing. It's a family of architectures that evolved because the baseline approach breaks in predictable ways, and each variant was built to fix a specific kind of breakage. Understanding the family — what each member optimizes for, where each one fails — is the only way to pick the right one for your actual problem.

Naive RAG

Naive RAG is the canonical baseline, and it's worth describing precisely before you complicate it. The pipeline is linear: split source documents into fixed-size chunks (512 tokens is a common starting point), embed each chunk with a text embedding model, store the vectors in a database like Pinecone or pgvector, and at query time retrieve the top-k chunks by cosine similarity and drop them into the LLM prompt.

For simple factual questions with answers contained in a single passage, this is fast to build, cheap to run, and easy to debug. It earns its place.

It breaks when the answer requires combining facts across multiple locations, or when the relevant chunk is buried under many superficially similar chunks, or — more often than people expect — when a chunk boundary literally cuts the answer in half. Bad chunking decisions propagate silently downstream, and Naive RAG gives you no way to know they're happening.

Advanced RAG

Advanced RAG adds intelligence at two points: before retrieval (query transformation) and after retrieval (reranking). Both matter.

On the pre-retrieval side, query rewriting rephrases the user's question before embedding it so the query better matches how answers actually appear in the corpus. HyDE — Hypothetical Document Embedding — takes this further: ask the LLM to generate a hypothetical ideal answer, then embed that answer rather than the raw question, because the hypothesis tends to be closer to stored document language than the original query. Parent-child chunking indexes small child chunks for precise retrieval but returns the larger parent chunk to the LLM for richer context — a compromise that sounds obvious in retrospect and took the field longer to adopt than it should have.

On the post-retrieval side, cross-encoder rerankers like Cohere Rerank or BGE-reranker take the top-k retrieved chunks and re-score them with full attention to both the query and the passage. The bi-encoder similarity that drove initial retrieval is fast but approximate; the cross-encoder is slower but far more precise, and running it over a candidate set of twenty chunks costs a fraction of what it would cost to run it over the full corpus.

Advanced RAG significantly improves retrieval quality for ambiguous queries, long documents, and cases where you need both precision and recall. What it doesn't change is the fundamental model: you're still doing similarity search, and the system still has no explicit representation of relationships or structure.

Modular RAG

Modular RAG treats retrieval as a system of composable components rather than a fixed pipeline. Retrieval, reranking, fusion, and generation become separate modules with clear interfaces; you can swap in BM25 sparse retrieval, dense vector search, or hybrid combinations; multiple retrievers can run in parallel with results merged via Reciprocal Rank Fusion; and routing logic decides which sub-pipeline handles each query based on its characteristics.

Think of it as the microservices architecture of retrieval — which means it inherits the same tradeoff. More flexibility, more orchestration, more surface area to monitor and debug. The engineering complexity is real, but so is the payoff when you're serving genuinely diverse query patterns (FAQ-style, analytical, code search) against heterogeneous data sources (PDFs, databases, APIs, wikis).

GraphRAG

GraphRAG, open-sourced by Microsoft Research in 2024, takes a structurally different approach: it makes the relationship structure of a corpus explicit before any query arrives.

The pipeline runs entity and relationship extraction across all source documents, constructs a knowledge graph where nodes are entities and edges are relationships, runs community detection via the Leiden algorithm to find clusters of related entities, and generates a high-level summary for each community. At query time, two modes are available: local search retrieves specific entities and their neighborhoods for precise factual questions, and global search retrieves community summaries and runs map-reduce synthesis across them for thematic questions that span the entire corpus.

GraphRAG handles questions like "what are the main themes in this corpus?" — questions that are essentially impossible for vector search to answer well — because it reasons about structure, not similarity. The performance difference on global synthesis tasks is significant enough to matter.

The cost is also significant. Indexing a large document collection requires LLM calls at every stage of extraction, and can run into hundreds of dollars before a single user query is answered. Every corpus update triggers a partial or full re-run of extraction and community summarization. GraphRAG is the right tool when the corpus is stable, the questions demand global synthesis, and you can absorb the upfront indexing cost — which, to be direct, rules out most production systems with live, frequently updated data.

Agentic RAG

Agentic RAG replaces the static pipeline with an LLM agent that decides dynamically when and how to retrieve. The agent receives the user query, decides whether retrieval is needed at all, formulates a search query, calls retrieval tools, inspects results, reformulates if context is insufficient, and can mix retrieval strategies and combine evidence across multiple passes before answering. Tool-use frameworks — function calling, ReAct — give the agent structured access to retrieval APIs.

The strength is flexibility: Agentic RAG handles complex multi-step reasoning tasks better than any fixed pipeline and can dynamically explore a knowledge space rather than relying on a single retrieval shot. The cost is latency (multiple LLM calls per query), higher cost per query, and emergent behavior that is genuinely harder to evaluate and debug than a fixed pipeline.

Agentic RAG earns its overhead when answer quality is paramount and latency is not the binding constraint — research tools, complex analytical workflows, anything where the user expects to wait.

Beyond Vectors: Hierarchical Approaches

Not every RAG architecture depends on vector similarity. PageIndex builds a hierarchical tree index from document structure itself — chapters, sections, subsections — and uses LLM reasoning to navigate that structure. Retrieval is driven by structural position and LLM judgment rather than semantic similarity, which makes it more robust for long-form, highly structured documents like technical manuals or legal texts where embeddings lose fine-grained precision.

The important architectural principle here — one worth carrying forward regardless of which system you use — is that structure embedded at index time beats structure inferred at query time. Every technique in this family is a way of recovering structure that was never encoded in the first place.

Understanding which problem you're actually solving — simple factual Q&A, multi-hop reasoning, global synthesis, or adaptive exploration — determines which architecture earns its place. There is no general-purpose answer. There is only the right tradeoff for your specific failure mode.