MODULE m1 · 8.0 HOURS

Understanding RAG Architecture

0% complete

LEARNING OBJECTIVES

Module objectives

  1. Understand the core components of the RAG (Retrieval-Augmented Generation) architecture.
  2. Identify the knowledge limitations of Large Language Models (LLMs) and the necessity of retrieval-based augmentation.
  3. Explain the structural flow of the retrieval-generation pipeline.

RAG (Retrieval-Augmented Generation) Architecture Overview

While state-of-the-art Natural Language Processing (NLP) and deep learning models show excellent performance by learning from vast amounts of text data, they have limitations such as exhibiting hallucinations or lacking knowledge regarding the latest information not included at the time of model training, or private data within specific domains [S1].

Overcoming LLM Limitations through Retrieval

RAG is a method that, instead of forcing the model to memorize all knowledge within its parameters, retrieves reliable external documents related to the question at the ‘just-in-time’ moment and provides them as input for the generation stage [S2].

Core Components

  1. Retriever: Receives the user’s query and identifies highly relevant document chunks from sources such as vector databases.
  2. Augmentation: Combines the retrieved documents with the original question to construct the prompt to be delivered to the LLM.
  3. Generator: Generates a fact-based response based on the augmented information.

This structure contributes to securing reliability by keeping the model’s knowledge up-to-date and making the generated answers’ evidence traceable.

WORKED EXAMPLES

Worked examples

  1. Case 1: Traditional LLM approach - When asked 'Tell me today's news', there is a risk of generating incorrect information because it is unaware of events after the training data cutoff.
  2. Case 2: RAG approach - When asked 'Tell me today's news', 1) the Retriever collects relevant articles through an external search engine or real-time news API, and 2) generates an accurate, up-to-date response by including these as context and passing them to the LLM.

LAB PROTOCOL

RAG Architecture Flow Visualization and Analysis

  1. 1

    Open Jupyter Notebook and diagram the structure of the RAG base pipeline's 3 stages (input, retrieval, generation).

  2. 2

    Extract 5 short texts from an open-license document corpus to create a dataset sample.

  3. 3

    Implement a simple keyword-matching Retriever function to return documents matching the question.

  4. 4

    Write code for the augmentation stage that injects the retrieved documents into a prompt template.

Safety check
  • Never use actual personal information or confidential documents as corpus data.
  • Check the Rate Limit when using APIs and set a seed value in the test code to ensure reproducibility.

Lab deliverables

  • RAG architecture diagram (included within a Notebook cell)
  • Simple keyword-based retriever implementation code
  • Document-injected prompt generation output

ASSIGNMENT

RAG-based Information Retrieval Pipeline Analysis Report

Deliverables

Rubric

  • Are the 3 stages of RAG (retrieval, augmentation, generation) explained and distinguished accurately?
  • Is the analysis of the possibility of retrieving irrelevant documents during the retrieval stage valid?
  • Was the implementation done in compliance with private data security guidelines?

KNOWLEDGE CHECK

Knowledge check

1What is the main advantage of RAG compared to LLM training methods?
2What is the correct role of the Retriever?

FIELD CHECK

Completion criteria

  • Can explain the components of the RAG architecture.
  • Confirm that the practiced RAG pipeline code works properly to retrieve and augment relevant documents.
  • Describe the limitations of the RAG pipeline and directions for improvement in an analysis report.

MODULE SOURCES

Module sources

  1. Stanford CS 224N | Natural Language Processing with Deep Learningweb.stanford.edu · university
  2. Natural Language Processing with Deep Learning CS224N/Ling284web.stanford.edu · university