∇GradientSchool
Advanced
210–270 min read
#Advanced RAG#Graph RAG#Hybrid Search#Reranking#Query Routing#Agentic RAG#Multi-Agent Systems#Memory#Planning#LangGraph#Workflow Orchestration#Production AI

Advanced RAG & Agent Architectures

An advanced guide to designing high-quality retrieval and agent systems, covering hybrid and graph retrieval, reranking, query routing, agentic RAG, planning, memory, multi-agent architectures, workflow orchestration, evaluation, and production patterns.

Advanced RAG & Agent Architectures

1. Introduction#

Basic RAG follows:

Architecture & Data Flow
Question
 |
 v
Retrieve documents
 |
 v
LLM
 |
 v
Answer

This works well for many applications.

But complex enterprise and educational workloads often require:

Architecture & Data Flow
Query understanding
 |
 v
Query routing
 |
 v
Multiple retrieval strategies
 |
 v
Reranking
 |
 v
Context construction
 |
 v
Reasoning / planning
 |
 v
Tools
 |
 v
Verification
 |
 v
Final answer

This notebook explores these advanced architectures.

2. Learning Objectives

By the end of this notebook, you should understand:

  1. Limitations of basic RAG
  2. Advanced retrieval
  3. Hybrid search
  4. Sparse vs dense retrieval
  5. Reranking
  6. Query rewriting
  7. Query decomposition
  8. Multi-query retrieval
  9. Query routing
  10. Metadata-aware retrieval
  11. Parent-child retrieval
  12. Contextual retrieval
  13. Graph RAG
  14. Knowledge graphs
  15. Agentic RAG
  16. Iterative retrieval
  17. Retrieval planning
  18. Verification
  19. Agent memory
  20. Short-term memory
  21. Long-term memory
  22. Episodic memory
  23. Semantic memory
  24. Planning architectures
  25. ReAct-style agents
  26. Workflow agents
  27. Multi-agent systems
  28. Supervisor architectures
  29. Handoff architectures
  30. Parallel agents
  31. Human-in-the-loop workflows
  32. LangGraph-style state machines
  33. Evaluation
  34. Cost and latency optimization
  35. Production architecture

3. Why Basic RAG Is Not Always Enough

Consider:

text
"What were the main reasons revenue declined in Q3,
which departments were affected, and what actions were
recommended?"

This may require:

text
Multiple documents
Multiple retrieval queries
Entity relationships
Numerical reasoning
Cross-document comparison

A single top-k vector search may miss important evidence.

4. Advanced RAG Pipeline

A more sophisticated pipeline:

Architecture & Data Flow
User Query
 |
 v
Query Analysis
 |
 v
Query Routing
 |
 +----------+----------+
 | | |
 v v v
Vector Keyword Graph
Search Search Search
 | | |
 +----------+----------+
 |
 v
 Fusion / Merge
 |
 v
 Reranker
 |
 v
 Context Builder
 |
 v
 LLM
 |
 v
 Verification

5. Sparse Retrieval

Sparse retrieval represents text using sparse term-based representations.

A classic approach is:

›BM25

It is useful when exact terms matter.

Examples:

text
Invoice ID
Product code
Employee ID
Legal clause
Technical error code

6. Dense Retrieval

Dense retrieval converts content into vectors.

Architecture & Data Flow
Text
 |
 v
Embedding model
 |
 v
Vector

Semantic similarity can retrieve conceptually related text even when exact words differ.

7. Sparse vs Dense

RetrievalStrength
SparseExact terminology
DenseSemantic similarity
HybridCombines both

Hybrid retrieval is often useful for enterprise systems.

8. Hybrid Search

A hybrid retriever can combine:

text
BM25 score
+
Vector similarity

Conceptually:

Architecture & Data Flow
Query
 |
 +--> Keyword search
 |
 +--> Vector search
 |
 v
Score fusion
 |
 v
Candidate documents

9. Score Fusion

Suppose:

Mathematical Formulation
Document A
Keyword score = 0.8
Vector score = 0.6

A combined score might be:

Mathematical Formulation
final =
alpha × keyword
+
(1-alpha) × vector

The exact weighting should be evaluated empirically.

10. Reciprocal Rank Fusion

RRF combines rankings rather than raw scores.

Conceptually:

Mathematical Formulation
Score(d) =
sum 1 / (k + rank(d))

This can combine:

text
Keyword ranking
+
Vector ranking
+
Other retrieval rankings

without requiring scores to be on the same scale.

11. Reranking

Initial retrieval may produce:

›Top 50 documents

A reranker can reorder them:

Architecture & Data Flow
50 candidates
 |
 v
Reranker
 |
 v
Top 5–10 documents

This can improve relevance.

12. Retriever vs Reranker

Retriever:

text
Fast
High recall
Large candidate set

Reranker:

text
More expensive
Higher precision
Smaller candidate set

A common architecture:

Architecture & Data Flow
Vector / BM25
 |
 v
Top 50
 |
 v
Reranker
 |
 v
Top 5

13. Query Rewriting

The user's query may not be ideal for retrieval.

Example:

›User:
"How much did it go down?"

The query is ambiguous.

The system can rewrite it using conversation context:

›"What was the percentage decline in Q3 revenue?"

Then retrieve.

14. Query Rewriting Risks

An incorrect rewrite can change the user's intent.

Therefore evaluate:

text
Original intent
vs
Rewritten intent

Do not blindly trust generated queries.

15. Multi-Query Retrieval

Generate multiple search formulations.

Example:

text
Original:
"Why did revenue fall?"

Query 1:
"Q3 revenue decline causes"

Query 2:
"Factors affecting Q3 revenue"

Query 3:
"Revenue decline management analysis"

Retrieve for each query and merge results.

16. Query Decomposition

Complex questions can be split.

Example:

›"Compare the pricing and security policies
of products A and B."

Decompose:

text
1. Product A pricing
2. Product A security
3. Product B pricing
4. Product B security

Retrieve separately.

17. Parallel Retrieval

Independent subqueries can execute simultaneously.

Architecture & Data Flow
 Query
 |
 +---------+---------+
 | | |
 v v v
 Search A Search B Search C
 | | |
 +---------+---------+
 |
 v
 Merge

This can reduce latency.

18. Query Routing

Not every question needs the same retrieval strategy.

Example:

Architecture & Data Flow
"What is photosynthesis?"
 |
 v
Course-content retriever

while:

Architecture & Data Flow
"Which students completed lesson 3?"
 |
 v
Database query

The router selects the correct source.

19. Retrieval Router

Conceptually:

🐍 Python
def route(query):

 if asks_about_course_content(query):
 return "course_rag"

 if asks_about_student_data(query):
 return "database"

 if requires_current_web_data(query):
 return "web_search"

 return "general_llm"

Production routing should use explicit policies and authorization.

20. Metadata Filtering

Retrieval can filter by:

text
tenant_id
course_id
subject
grade
document_type
date
access_level

Example:

🐍 Python
filters = {
 "tenant_id": tenant_id,
 "course_id": course_id
}

Metadata filtering is both a relevance and security mechanism.

21. Parent-Child Retrieval

A useful pattern is:

Architecture & Data Flow
Large parent document
 |
 +--> Child chunk 1
 +--> Child chunk 2
 +--> Child chunk 3

Search using small child chunks.

Return the larger parent context.

This can improve:

›Retrieval precision
Context completeness

22. Contextual Chunking

Instead of blindly splitting text:

›Every 500 tokens

preserve structure:

text
Chapter
 |
Section
 |
Subsection
 |
Paragraph

Metadata can include:

text
Chapter title
Section title
Page
Document

23. Contextual Retrieval

A chunk such as:

›"The rate increased by 12%."

is ambiguous.

Contextualization can attach:

text
Document:
Annual Financial Report

Section:
Q3 Revenue

Context:
The company increased the subscription rate by 12%.

This can improve retrieval.

24. Lost-in-the-Middle Problem

When many documents are placed into a long context, the model may not use information equally well.

Conceptually:

Architecture & Data Flow
Context
|
| Relevant
|
| Noise
|
| Critical information
|
| Noise
|
| Relevant

This is one reason:

›Better retrieval

can be more valuable than:

›More context

25. Context Compression

After retrieval:

Architecture & Data Flow
10 documents
 |
 v
Extract relevant passages
 |
 v
Compact context
 |
 v
LLM

Compression can reduce:

text
Tokens
Latency
Noise
Cost

But verify that important evidence is not removed.

26. Graph RAG

Graph RAG combines retrieval with relationships.

A knowledge graph represents:

text
Entities
+
Relationships
+
Attributes

Example:

Architecture & Data Flow
Company A
 |
 | owns
 v
Product B
 |
 | uses
 v
Technology C

27. Why Graph RAG?

Vector retrieval is good for:

›Semantic similarity

Graphs are good for:

text
Relationships
Multi-hop reasoning
Entity connections
Structured dependencies

28. Graph RAG Architecture

Architecture & Data Flow
Documents
 |
 v
Entity extraction
 |
 v
Relationship extraction
 |
 v
Knowledge graph
 |
 v
Graph retrieval
 |
 v
Relevant entities + relationships
 |
 v
LLM

29. Graph Query Example

Question:

›"Which technologies are used by companies
owned by Company X?"

A graph can traverse:

Architecture & Data Flow
Company X
 |
 owns
 v
Company A
 |
 uses
 v
Technology A

Company X
 |
 owns
 v
Company B
 |
 uses
 v
Technology B

This is a multi-hop relationship query.

30. Graph + Vector Retrieval

The strongest architecture may combine:

text
Vector search
+
Keyword search
+
Graph traversal

Pipeline:

Architecture & Data Flow
Query
 |
 +--> Vector
 |
 +--> Keyword
 |
 +--> Graph
 |
 v
Fusion
 |
 v
Reranking
 |
 v
LLM

31. Temporal Retrieval

Enterprise data changes over time.

A query might ask:

›"What was the policy in 2024?"

The retriever should consider:

text
Document version
Effective date
Expiration date

Do not return only the newest document.

32. Event-Aware Retrieval

Some applications need event relationships:

Architecture & Data Flow
Event A
 |
 v
Event B
 |
 v
Event C

Examples:

text
Incident timelines
Project milestones
Financial events
Learning progress

Graph or temporal models can help.

33. Agentic RAG

Traditional RAG:

Architecture & Data Flow
Retrieve once
 |
 v
Answer

Agentic RAG:

Architecture & Data Flow
Question
 |
 v
Plan
 |
 v
Retrieve
 |
 v
Evaluate evidence
 |
 +--> insufficient --> Retrieve again
 |
 v
Reason
 |
 v
Verify
 |
 v
Answer

The system dynamically decides whether additional retrieval is needed.

34. Agentic RAG Loop

Architecture & Data Flow
START
 |
 v
Understand question
 |
 v
Retrieve
 |
 v
Assess evidence
 |
 +---- weak ----> Rewrite query
 | |
 | v
 | Retrieve
 |
 +---- strong
 |
 v
 Answer

Set limits on:

text
Iterations
Tokens
Latency
Cost

35. Retrieval Verification

After retrieving documents, ask:

›Does this evidence actually support the question?

Possible approaches:

text
Reranker
LLM evaluator
Rule-based checks
Citation matching

36. Citation Verification

If an answer says:

›"Revenue declined 12%."

the system should identify evidence:

text
Document X
Page 14
"Revenue declined 12%."

Citation verification helps detect unsupported claims.

37. Corrective RAG

Corrective RAG can detect poor retrieval.

Conceptually:

Architecture & Data Flow
Retrieve
 |
 v
Grade documents
 |
 +--> Relevant -> Generate
 |
 +--> Irrelevant -> Rewrite / alternate search

This adds a retrieval-quality control loop.

38. Self-Query Retrieval

A natural-language query can be converted into:

text
Semantic query
+
Metadata filters

Example:

›"Show me physics lessons for grade 8
published after January 2026."

Possible structured representation:

json
{
 "semantic_query": "physics lessons",
 "filters": {
 "grade": 8,
 "published_after": "2026-01-01"
 }
}

The application must validate generated filters.

39. Memory

Agents may need memory beyond the current request.

A useful distinction:

›Short-term memory
Long-term memory

40. Short-Term Memory

Short-term memory contains the current conversation state.

Architecture & Data Flow
User
 |
 +--> Question
 +--> Follow-up
 +--> Clarification

This is usually session-scoped.

41. Long-Term Memory

Long-term memory may contain information that persists across sessions.

Examples:

text
User preferences
Past tasks
Important facts
Learning progress

Store only information that is appropriate and useful.

42. Semantic Memory

Semantic memory stores facts.

Example:

›Student prefers explanations with examples.

The system can retrieve this later.

43. Episodic Memory

Episodic memory stores events.

Example:

›Student struggled with quadratic equations
during the previous session.

This can support personalized learning.

44. Memory Architecture

Architecture & Data Flow
Conversation
 |
 +--> Short-term state
 |
 +--> Memory extraction
 |
 v
 Long-term store
 |
 v
 Future retrieval

Do not automatically save every conversation.

45. Memory Security

Memory may contain sensitive information.

Use:

text
Authorization
Tenant isolation
Encryption
Retention policies
Deletion

Users should have appropriate control over persistent information.

46. Memory Retrieval

At the beginning of a request:

Architecture & Data Flow
User
 |
 v
Current request
 |
 +--> Retrieve relevant memory
 |
 v
Context builder
 |
 v
LLM

Only relevant memories should enter the context.

47. Memory Conflicts

Suppose memory says:

›Student prefers advanced explanations.

Current request:

›"Explain this like I'm a beginner."

Current explicit instruction should take priority.

Memory should be treated as context, not absolute instruction.

48. Planning

Complex agents may need explicit planning.

Example:

text
Goal:
Create a lesson.

Plan:
1. Analyze topic
2. Identify learning objectives
3. Generate explanation
4. Create examples
5. Generate quiz
6. Validate

Planning can improve complex workflows.

49. ReAct-Style Agents

A ReAct-style agent alternates between:

Architecture & Data Flow
Reason / decide
 |
 v
Act / tool
 |
 v
Observe result
 |
 v
Reason / decide

Conceptually:

Architecture & Data Flow
Thought
 |
Action
 |
Observation
 |
Thought
 |
Action

In production systems, internal reasoning should not automatically be exposed to users.

50. Workflow Agents

Not every agent needs open-ended reasoning.

A workflow can define:

Architecture & Data Flow
Node A
 |
 v
Node B
 |
 +--> condition
 |
 +--> Node C

This is often more predictable.

51. Deterministic vs Agentic

Use deterministic workflows when:

text
Steps are known
Rules are strict
Risk is high

Use agentic workflows when:

text
Path is uncertain
Tool selection varies
Problem is exploratory

Hybrid systems are often strongest.

52. Hybrid Agent Architecture

Architecture & Data Flow
Fixed workflow
 |
 v
Agent decision
 |
 v
Tool selection
 |
 v
Fixed validation
 |
 v
Next workflow step

This combines flexibility and control.

53. Multi-Agent Systems

A multi-agent system uses multiple specialized agents.

Example:

Architecture & Data Flow
Supervisor
 |
 +--> Research Agent
 |
 +--> Analyst Agent
 |
 +--> Writer Agent
 |
 +--> Reviewer Agent

Each agent has a specific responsibility.

54. Why Multiple Agents?

Specialization can help:

text
Research
Analysis
Coding
Review
Planning

But multi-agent systems also add:

text
Latency
Cost
Complexity
Coordination failures

Do not use multiple agents unless they provide a real benefit.

55. Supervisor Pattern

A supervisor decides which agent should act.

Architecture & Data Flow
 Supervisor
 / | \
 / | \
 v v v
 Research Analysis Writer

The supervisor maintains overall state.

56. Handoff Pattern

One agent transfers control to another.

Architecture & Data Flow
Agent A
 |
 | handoff
 v
Agent B
 |
 | handoff
 v
Agent C

Useful when responsibility changes.

57. Parallel Multi-Agent Pattern

Independent tasks can execute simultaneously.

Architecture & Data Flow
 Supervisor
 |
 +----------+----------+
 | | |
 v v v
 Research A Research B Research C
 | | |
 +----------+----------+
 |
 v
 Synthesis

This can reduce latency.

58. Multi-Agent Educational Example

Question:

›"Create a Grade 8 lesson on climate change."

Agents:

text
Curriculum Agent
Content Agent
Quiz Agent
Safety Reviewer

Workflow:

Architecture & Data Flow
Curriculum
 |
 v
Content
 |
 +--> Quiz
 |
 v
Reviewer
 |
 v
Teacher approval

59. Multi-Agent Failure Modes

Potential failures:

text
Agent disagreement
Repeated handoffs
Duplicate work
Incorrect delegation
Cost explosion
Conflicting outputs

Use:

text
Maximum steps
Timeouts
Budgets
Explicit state
Validation

60. State Machines

Complex workflows benefit from explicit state.

Example:

🐍 Python
state = {
 "query": "...",
 "documents": [],
 "evidence": [],
 "draft": None,
 "approved": False
}

Nodes modify state.

61. LangGraph-Style Architecture

Conceptually:

Architecture & Data Flow
START
 |
 v
Query Analyzer
 |
 v
Retriever
 |
 v
Evidence Grader
 |
 +---- weak ----> Query Rewriter
 | |
 | v
 | Retriever
 |
 +---- strong
 |
 v
 Writer
 |
 v
 Reviewer
 |
 v
 END

This is a graph rather than a simple chain.

62. Conditional Routing

A graph can route based on state.

🐍 Python
if state["evidence_quality"] < 0.7:
 return "retrieve_again"

return "generate"

The threshold should be evaluated experimentally.

63. Retry Policies

Not every failure should restart the entire workflow.

Example:

Architecture & Data Flow
Retriever fails
 |
 v
Retry retriever

while:

Architecture & Data Flow
Authorization fails
 |
 v
Stop

Use error-specific policies.

64. Human-in-the-Loop Graph

Architecture & Data Flow
Generate
 |
 v
Review required?
 |
 +--+--+
 | |
No Yes
 | |
 v v
End Human
 |
 v
 Approve
 |
 v
 End

Useful for high-impact outputs.

65. Agent Guardrails

Define:

text
Maximum steps
Maximum tokens
Maximum cost
Maximum tool calls
Allowed tools
Timeout

These are essential production controls.

66. Agent Memory + RAG

Memory and RAG solve different problems.

text
Memory:
What should the system remember about the user?

RAG:
What external knowledge should the system retrieve?

They can work together:

Architecture & Data Flow
Current request
 |
 +--> Memory
 |
 +--> RAG
 |
 v
Context
 |
 v
Agent

67. Agentic RAG + Tools

A complex assistant may use:

text
RAG
+
Search
+
Database
+
Calculator

Example:

Architecture & Data Flow
Question
 |
 v
Plan
 |
 +--> Retrieve course material
 |
 +--> Query database
 |
 +--> Calculate
 |
 v
Synthesize

68. Educational Agent Example

Student asks:

›"How am I doing in mathematics and
what should I study next?"

The agent may:

text
1. Read learning progress
2. Retrieve recent lesson content
3. Identify weak topics
4. Generate recommendations
5. Build a study plan

Deterministic scheduling logic should validate the final plan.

69. Retrieval Quality Evaluation

Useful metrics include:

text
Precision@K
Recall@K
MRR
NDCG

These measure retrieval quality rather than final answer quality.

70. Answer Quality Evaluation

Evaluate:

text
Correctness
Relevance
Groundedness
Completeness
Citation correctness

71. Agent Evaluation

Evaluate:

text
Tool selection
Tool arguments
Task completion
Trajectory length
Efficiency
Failure recovery
Safety

72. End-to-End Evaluation

A strong evaluation pipeline:

Architecture & Data Flow
User query
 |
 v
Agent
 |
 +--> Retrieval
 |
 +--> Tools
 |
 v
Answer
 |
 v
Evaluate:
 - retrieval
 - tool use
 - correctness
 - groundedness
 - safety
 - latency
 - cost

73. Cost Optimization

Advanced RAG and agents can become expensive.

Control:

text
Retrieval count
Reranking candidates
Context size
Agent steps
Tool calls
Model selection

74. Latency Optimization

Strategies:

text
Parallel retrieval
Parallel agents
Caching
Smaller models
Streaming
Fewer agent steps
Precomputed embeddings

75. Agent Budget

Define a budget:

Mathematical Formulation
Maximum tokens = 20,000
Maximum tool calls = 10
Maximum runtime = 60 seconds
Maximum cost = $0.20

If the budget is exceeded:

text
Stop
Fallback
Escalate

76. Advanced Production Architecture

Architecture & Data Flow
 CLIENT
 |
 v
 API Gateway
 |
 v
 Authentication
 |
 v
 AI Gateway
 |
 +------------+------------+
 | | |
 v v v
 Cache Router Policy
 |
 v
 Agent Graph
 |
 +-----------------+-----------------+
 | | |
 v v v
 Hybrid RAG Knowledge Graph Tools
 | | |
 +-----------------+-----------------+
 |
 v
 Reranker
 |
 v
 LLM
 |
 v
 Validator
 |
 v
 Response
 |
 v
 Observability / Eval

77. Educational Platform Architecture

Architecture & Data Flow
 EDUCATIONAL PLATFORM
 |
 +----------------+----------------+
 | | |
 v v v
 Student Teacher Admin
 | | |
 +----------------+----------------+
 |
 v
 AI Gateway
 |
 +--------------------+--------------------+
 | | |
 v v v
 Tutor Content Analytics
 | Generation |
 | | |
 +--------------------+--------------------+
 |
 v
 Agent Graph
 |
 +-----------------------+-----------------------+
 | | |
 v v v
 Course RAG Student DB Tools
 | | |
 +-----------------------+-----------------------+
 |
 v
 Validation / Safety
 |
 v
 Student Response

78. Advanced Educational Tutor Flow

Architecture & Data Flow
Student question
 |
 v
Intent classification
 |
 +--> Concept question
 | |
 | v
 | Course RAG
 |
 +--> Progress question
 | |
 | v
 | Student DB
 |
 +--> Practice request
 |
 v
 Question generator
 |
 v
 Validation
 |
 v
 Tutor

79. Personalization Flow

Architecture & Data Flow
Student
 |
 v
Current question
 |
 +--> Course context
 |
 +--> Learning history
 |
 +--> Relevant memory
 |
 +--> Current lesson
 |
 v
Personalized context
 |
 v
Tutor

Keep the context minimal and authorized.

80. Advanced Project 1: Hybrid RAG

Build:

text
BM25
+
Vector search
+
RRF
+
Reranker

Compare against:

›Vector-only RAG

Measure:

text
Recall
Precision
Answer quality
Latency

81. Advanced Project 2: Graph RAG

Build a small knowledge graph for:

text
Courses
Topics
Lessons
Prerequisites
Concept relationships

Query:

›"What should a student learn before
quadratic equations?"

Use graph traversal to identify prerequisites.

82. Advanced Project 3: Agentic RAG

Build:

Architecture & Data Flow
Retrieve
 |
 v
Grade evidence
 |
 +--> weak -> rewrite query
 |
 v
Retrieve again
 |
 v
Generate
 |
 v
Verify

Set:

Mathematical Formulation
Maximum retrieval loops = 3

83. Advanced Project 4: Multi-Agent Content Generator

Build:

Architecture & Data Flow
Curriculum Agent
 |
 v
Content Agent
 |
 +--> Quiz Agent
 |
 v
Reviewer Agent
 |
 v
Teacher approval

Evaluate each stage independently.

84. Advanced Project 5: Personalized Study Agent

Build an agent that:

Architecture & Data Flow
Reads student progress
 |
 v
Identifies weak topics
 |
 v
Retrieves lessons
 |
 v
Generates practice
 |
 v
Creates recommendation
 |
 v
Validates against scheduling rules

85. Advanced Project 6: Research Agent

Build:

Architecture & Data Flow
Question
 |
 v
Planner
 |
 +--> Search
 +--> RAG
 +--> Calculator
 |
 v
Evidence
 |
 v
Writer
 |
 v
Reviewer

Include citations.

86. Advanced Project 7: Multi-Modal Agent

Build an agent that accepts:

text
Text
Image
Audio

and chooses appropriate tools/models.

Example:

Architecture & Data Flow
Image question
 -> Vision model

Audio question
 -> Speech model

Document question
 -> RAG

87. Advanced Project 8: Educational Graph

Create a graph:

Architecture & Data Flow
Algebra
 |
 +--> Variables
 |
 +--> Equations
 |
 +--> Linear equations
 |
 +--> Quadratic equations

Use it for:

text
Prerequisite recommendations
Learning paths
Question generation
Personalized revision

88. Common Mistakes

Mistake 1: Making every application agentic#

Use agents only when dynamic decisions are useful.

Mistake 2: Retrieving too much context#

More context can increase noise and cost.

Mistake 3: Skipping reranking#

Initial retrieval may not produce the best ordering.

Mistake 4: Ignoring metadata#

Metadata improves relevance and security.

Mistake 5: No loop limits#

Agents can become expensive or unstable.

Mistake 6: Using memory as truth#

Memory can be stale or incorrect.

Mistake 7: Too many agents#

Multi-agent systems increase complexity.

Mistake 8: No retrieval evaluation#

A fluent answer can hide poor retrieval.

89. Final Mental Model

Advanced RAG:

Architecture & Data Flow
Retrieve better
 |
 v
Rerank better
 |
 v
Construct better context
 |
 v
Generate better answers

Advanced agents:

Architecture & Data Flow
Understand
 |
 v
Plan
 |
 v
Act
 |
 v
Observe
 |
 v
Verify
 |
 v
Complete

Advanced production systems combine both:

Architecture & Data Flow
 AI SYSTEM
 |
 +------------+------------+
 | |
 v v
 Advanced RAG Agent Workflow
 | |
 +------------+------------+
 |
 v
 LLM / Tools
 |
 v
 Validation
 |
 v
 Evaluation / Ops

90. Key Takeaways

  1. Basic vector RAG is only one retrieval strategy.
  2. Sparse retrieval is useful for exact terms.
  3. Dense retrieval is useful for semantic similarity.
  4. Hybrid retrieval combines complementary signals.
  5. Reranking can improve precision after broad retrieval.
  6. Query rewriting can improve ambiguous searches.
  7. Multi-query retrieval increases retrieval coverage.
  8. Query decomposition helps answer complex questions.
  9. Query routing selects the appropriate data source.
  10. Metadata filtering improves both relevance and security.
  11. Parent-child retrieval balances precision and context.
  12. Contextual chunking preserves document structure.
  13. Context compression can reduce noise and cost.
  14. Graph RAG is useful for relationship-heavy questions.
  15. Graph and vector retrieval can be combined.
  16. Temporal retrieval matters when documents change over time.
  17. Agentic RAG can retrieve iteratively based on evidence quality.
  18. Retrieval loops require strict budgets and limits.
  19. Memory and RAG solve different problems.
  20. Short-term and long-term memory should be separated conceptually.
  21. Memory must be secured and governed.
  22. Current explicit instructions should override stale memory.
  23. Deterministic workflows are preferable when steps and rules are known.
  24. Agents are useful when decisions and paths are uncertain.
  25. Hybrid deterministic-agentic systems often provide a strong balance.
  26. Multi-agent systems can provide specialization but increase complexity.
  27. Supervisor, handoff, and parallel-agent patterns solve different coordination problems.
  28. Agent state makes complex workflows easier to control.
  29. Tool permissions must remain outside the model's authority.
  30. Retrieval quality and answer quality should be evaluated separately.
  31. Agent evaluation should include tool selection and trajectory quality.
  32. Cost and latency must be controlled explicitly.
  33. Educational agents can combine course RAG, student data, memory, and tools.
  34. Personalized educational AI should minimize and authorize student context.
  35. Advanced RAG and agent architectures should be introduced only when they solve a real problem.

91. Knowledge Check

Question 1#

Why might hybrid search outperform vector-only search?

Question 2#

What is the purpose of a reranker?

Question 3#

When is query decomposition useful?

Question 4#

What is query routing?

Question 5#

Why is metadata filtering important for multi-tenant RAG?

Question 6#

What is parent-child retrieval?

Question 7#

What problem does Graph RAG address?

Question 8#

What is agentic RAG?

Question 9#

Why should retrieval loops have maximum iteration limits?

Question 10#

What is the difference between short-term and long-term memory?

Question 11#

Why should memory not be treated as absolute truth?

Question 12#

When should a deterministic workflow be preferred over an agent?

Question 13#

What is the supervisor multi-agent pattern?

Question 14#

Why can multi-agent systems become expensive?

Question 15#

How would you combine RAG, memory, and student progress data in an educational AI tutor?

92. Course Progression

The Generative AI track now progresses through:

Architecture & Data Flow
Generative AI Foundations
 |
 v
Transformers & LLM Architecture
 |
 v
RAG, Embeddings & Vector Databases
 |
 v
LangChain, LangGraph & Agents
 |
 v
LLM Evaluation, Safety & Guardrails
 |
 v
Multimodal Generative AI
 |
 v
Fine-Tuning, LoRA, QLoRA & PEFT
 |
 v
Open-Source, Open-Weight & Sovereign AI
 |
 v
LLMOps, Inference Optimization & Production
 |
 v
End-to-End GenAI Application Projects
 |
 v
Security, Privacy, Governance & Responsible AI
 |
 v
Advanced RAG & Agent Architectures

The next stage should focus on building and operating an AI platform itself, including AI gateways, model routing, evaluation pipelines, prompt management, model registries, vector infrastructure, feature/data pipelines, deployment automation, and a complete educational AI platform architecture.

Knowledge Checkpoint

Advanced RAG & Retrieval Systems Checkpoint

Q1.What is Hypothetical Document Embeddings (HyDE)?
AA technique where an LLM first generates a hypothetical answer to the user query, and that generated answer's embedding is used to search the vector database.
BA method for encrypting documents.
CA compression algorithm for embedding models.
DA synthetic document benchmark.
Q2.What is Multi-Query Expansion in advanced retrieval?
AUsing an LLM to rephrase a single user prompt into 3-5 diverse query formulations to retrieve and union documents across different semantic perspectives.
BRunning 5 database servers simultaneously.
CSubmitting user passwords to multiple databases.
DSplitting queries across multiple CPU threads.
Q3.How does Self-RAG enhance retrieval-augmented generation?
AThe model outputs reflection tokens to dynamically decide when retrieval is needed, evaluate whether retrieved chunks are relevant, and verify if its own response is supported by evidence.
BIt retrieves documents without network connections.
CIt fine-tunes the database index on each query.
DIt bypasses LLM inference.
Track Your Learning

Finished studying this notebook?

Mark this guide as completed to update your course progress roadmap.

Previous Lesson
Retrieval-Augmented Generation: Embeddings, Vector Databases & Semantic Search
Next Lesson
LangChain, LangGraph & Agentic AI Systems