Dayward AI

RAG in 14 Days: From Retrieval to Trustworthy Answers

Taught in Chinese

Week 1 · Building a Usable RAG Service From Scratch

D1

Why Retrieve at All: Hallucination, Knowledge Cutoffs, and the Cost of Long Context; a Minimal Keyword-Only RAG

First get clear on the three specific problems Retrieval-Augmented Generation actually solves for a model, weigh it against fine-tuning and very long context on cost and fit, then write your first working question-answering system with BM25 scoring, without touching a single vector.

D2

Embeddings and Vector Search: Similarity, Dimensionality, and Model Choice; Storing Text in pgvector

Understand what an embedding actually encodes through the coordinate-system analogy, get clear on the difference between cosine similarity and dot product and normalization's precondition, weigh model choice and dimensionality against a leaderboard, then write a corpus into pgvector and query its nearest neighbors.

D3

Getting Documents In: Parsing PDF and HTML, Tables and Scans, Cleaning Rules, and Metadata You Must Keep

Retrieval quality's ceiling is parsing quality; today go all the way through the parsing pitfalls of PDF, HTML, and Markdown sources, handle tables and scanned pages, settle on a set of cleaning rules, and lock down the metadata — heading level, page number, source address — you'll need later for citations and filtering.

D4

Chunking Strategies: Five Approaches — Fixed, Recursive, Structure-Based, Parent-Child, and Semantic — and Choosing by Evaluation, Not Intuition

Chunking is the step in the whole pipeline most often decided by gut feel, yet it affects results the most. Today implement five mainstream chunking approaches one by one, explain where each fits and fails, cover what overlap and contextual chunk headers are for, then hold all five up against the same ruler with a fixed set of questions.

D5

Vector Indexes and Store Selection: HNSW vs. Inverted File, Quantization to Save Memory, Filtered Queries and Multi-Tenant Isolation

Push vector search from working to holding up under load: understand how the two index structures are built and how their parameters affect recall and latency, compress memory with half-precision and binary quantization, solve the trickiest problem — filtered queries — then give selection criteria between pgvector and a dedicated vector store.

D6

The Generation Side: Ordering Context, Labeling Citations, When You Must Refuse to Answer, and Streaming Responses

Once retrieval brings back a pile of chunks, what actually decides the user experience is how you assemble them, how you get the model to cite its sources, and how it stays quiet when the evidence is thin. Today write a question-answering endpoint with verifiable citations and a refusal policy, and stream the answer.

D7

Week One Capstone: Assembling Six Days of Parts Into a One-Command Question-Answering Service, and a Retrospective

Assemble the previous six days — parsing, chunking, indexing, retrieval, assembly, generation — into a real service: one ingestion command, one question-answering endpoint, one configuration guide, all started with a single Docker command, then look back at the reasoning behind each decision this week and the technical debt left behind.

Week 2 · Making It Trustworthy, Evaluable, and Shippable

D8

Evaluation First: Building a Golden Set, Computing Recall and Ranking Metrics, Using a Model as Judge for Faithfulness

Every optimization in week two needs to be backed by data, so build evaluation first. Today construct a golden question-answer set from scratch, implement three retrieval metrics — recall, Mean Reciprocal Rank, and Normalized Discounted Cumulative Gain — use a model as judge for faithfulness and answer relevance, then run a baseline report on week one's system.

D9

Hybrid Search and Reranking: Two-Path Retrieval, Reciprocal Rank Fusion, Then Re-Ranking the Top Results With a Cross-Encoder

Keyword search and vector search each have their own blind spots; combining both paths with Reciprocal Rank Fusion is often an instant win. Then use a cross-encoder to precisely re-rank the top few dozen results, and use day eight's evaluation to prove how much each step actually contributed.

D10

Query-Side Optimization: Rewriting, Hypothetical Document Embeddings, Multi-Query, Step-Back Prompting, and Intent Routing

A user's question is often short and vague, and a retrieval miss isn't always the index's fault. Today put the effort into what happens after the query comes in and before retrieval happens: rewriting, generating a hypothetical answer and retrieving on that, splitting into multiple sub-queries, stepping back to ask a more general question, and judging intent first to decide which path to take.

D11

Advanced Indexing: Parent-Child Documents, Summary Indexes, Contextual Retrieval, and the Trade-Offs of Tree Aggregation vs. Graph Retrieval

The same set of documents can support several index structures. Today implement parent-child and summary indexes, land contextual retrieval — a technique with a very good cost-to-benefit ratio — then explain exactly what problems tree-based recursive aggregation and graph retrieval each solve, what they cost, and when not to use them.

D12

Agentic RAG: Turning Retrieval Into a Tool So the Model Decides Whether to Search, How Many Times, and Whether to Start Over

A single fixed retrieval pass can't handle multi-hop questions or retrieval failures. Today wrap retrieval as a tool inside an agent loop, implement self-reflection and result correction, handle multi-hop queries, and set a call-count and budget cap on the loop so it doesn't keep searching forever.

D13

Going to Production: Incremental Sync and Deduplication, Permission-Based Filtering, Cache Layering, Tracing, and the Cost-Latency Ledger

One stretch remains before handing the system to real users: documents change, people have different permissions, repeated questions shouldn't be recomputed every time, and when something breaks you need to find which step was slow and expensive. Today finish all four in one pass.

D14

Capstone Project and Retrospective: A Multi-Tenant Enterprise Knowledge-Base Q&A, a RAG Decision Map, and an Interview Deep Dive

Turn thirteen days of work into a portfolio-ready project: a multi-tenant, cited, evaluation-dashboarded enterprise knowledge-base Q&A system. Then compress the whole course into one decision map that answers the questions interviewers most often follow up on.