Through visual notes, we explore various training techniques and their underlying principles for taming Large Language Models (LLMs) to perform as we desire.
Imagine you have just hired a bright college graduate who lacks social experience. This new employee has vast knowledge, but is not yet familiar with your company’s way of working or the tone their boss expects. How would you teach them ‘how to work’ to ensure they deliver top performance?
Recently, the world of Large Language Models (LLMs) has been grappling with the exact same challenge. Technologies to ‘tame’ AI—which may have plenty of knowledge but need guidance to be truly useful—are advancing daily, and visual learning materials that make these concepts easy for anyone to understand are currently a hot topic Source: Show HN: Interactive notes on RLHF, attention, LoRA and LLM evals.
Why is this important?
In the past, to adapt AI for specific tasks, you had to retrain the entire model. This was astronomically expensive and required immense computing resources Source: Advanced Fine-Tuning Techniques: LoRA, QLoRA, PEFT, and RLHF. However, as companies increasingly try to integrate AI into real-world business, the core question has shifted to: ‘How can we teach it intelligently at a low cost?’ Source: Lessons from the Trenches: Building LLM Evals That Work…. As these technologies mature, your phone’s AI assistant will better understand your preferences, and building customized AI for your company will become much easier.
Understanding made easy: AI training techniques
Let’s explain these training techniques using the ‘new employee training’ analogy mentioned earlier.
1. LoRA (Low-Rank Adaptation) — Teaching just the essentials
Retraining a giant model from start to finish is highly inefficient. ‘LoRA’ is a method where you do not modify the billions of parameters of the model (the adjustable numerical values that AI learns), but instead adjust only the necessary, critical portions Source: Fine-Tuning LLMs: LoRA, QLoRA, RLHF, and Instruction Tuning. Simply put, instead of making the new employee reread thick college textbooks from cover to cover, you teach them the ‘company’s key takeaway notes’ so they can be immediately productive Source: Fine-tuning LLM v 2026: guide to LoRA, QLoRA and full fine-tuning. Once training is complete, these notes can be lightly added to the original model, making it highly versatile Source: LoRA (Low-Rank Adaptation) · Hugging Face.
2. RLHF (Reinforcement Learning from Human Feedback) — Refining with a boss’s advice
Judging whether an answer produced by AI is appropriate is difficult. This is where ‘humans’ step in. It is a process where, after the AI provides an answer, a person gives feedback such as, “This is a bit too formal,” or “This answer is kinder and better” Source: Fine-Tuning LLMs: LoRA, QLoRA, RLHF, and Instruction Tuning. The AI incorporates this feedback to correct itself and produce better answers. Think of it as a boss reviewing a new employee’s report and advising, “How about modifying it like this instead of that expression?” However, some research suggests that in this process, the AI may try to give only ‘safe’ answers, which can somewhat reduce the diversity of its outputs Source: Understanding the Effects of RLHF on LLM Generalisation and Diversity.
Current status: How far have we come?
Current technology is split into two major currents. Training techniques like PEFT (Parameter-Efficient Fine-Tuning) that efficiently adjust model weights continue to advance, and an environment has already been created where many developers can fine-tune specific models for their own purposes on consumer hardware Source: GitHub - jackaduma/ChatGLM-LoRA-RLHF-PyTorch.
Additionally, the field of ‘LLM Evals’ (evaluating AI properly) has become very important Source: Lessons from the Trenches: Building LLM Evals That Work…. However, there is still debate within the industry regarding what ‘LLM evaluation’ specifically means Source: Lessons from the Trenches: Building LLM Evals That Work….
What will happen in the future?
Moving forward, methods that greatly simplify the complex training processes of today will gain attention. For instance, alternatives such as DPO (Direct Preference Optimization), a simpler approach than RLHF, are being actively researched Source: Fine-Tuning LLMs: LoRA, QLoRA, RLHF, and Instruction Tuning.
As technology advances, we will move toward an era where we no longer need to worry about the ‘internal mechanics’ of AI, but can instead build our own customized AI as if assembling Lego blocks. It will be just like how the era of everyone easily building apps came about, even without knowing complex computer programming.
AI perspective: MindTickleBytes’ AI Reporter view
“AI training technology is shifting from simply ‘how to increase performance’ to ‘how to make it my own assistant easier and faster.’ The more materials that help with intuitive understanding—like the visual guides introduced here—are available rather than complex equations, the lower the barrier to AI technology will become.”
References
- Show HN: Interactive notes on RLHF, attention, LoRA and LLM evals
- Fine-Tuning LLMs: LoRA, QLoRA, RLHF, and Instruction Tuning
- Understanding the Effects of RLHF on LLM Generalisation and Diversity
- Lessons from the Trenches: Building LLM Evals That Work… - YouTube
- Fine-tuning LLM v 2026: guide to LoRA, QLoRA and full fine-tuning
- LoRA (Low-Rank Adaptation) · Hugging Face
- GitHub - jackaduma/ChatGLM-LoRA-RLHF-PyTorch
- Advanced Fine-Tuning Techniques: LoRA, QLoRA, PEFT, and RLHF
- Full Fine-Tuning
- LoRA
- RLHF
- RAG
- RLHF
- Attention
- Reduced diversity
- Slower speed
- Increased cost