NariQ is a quantization technique that drastically reduces the weight of AI models, making it the first method to deliver performance equivalent to the original model even in long-horizon agent tasks exceeding 100K tokens.
Imagine you give a complex errand to an Artificial Intelligence (AI): “Analyze our company’s sales data for the past 3 years, review reports for each department, fix any errors, and email me the final conclusion.” The AI opens multiple tools on its own, reads tens of thousands of lines of documents, and meticulously carries out tasks step-by-step over dozens of stages.
This process where an AI goes beyond simply answering questions to make its own judgments amid long conversations and documents, combining various tools to achieve long-term goals, is called long-horizon agentic tasks. Metaphorically speaking, rather than solving a multiple-choice quiz, it resembles an expert consultant independently planning and completing a major project spanning several months. Such tasks typically require an enormously long memory space (context) of over 100,000 (100K) tokens [Source Title].
However, there is a practical problem. High-performing Large Language Models (LLMs, large-scale artificial intelligence models that understand and generate human language) are so bulky that running them on regular computers or smart devices is as difficult as plucking a star from the sky. The technique of dieting down this massive model is called quantization (a technology that reduces model capacity by lowering the numerical precision inside AI models). Until now, quantization techniques have had a fatal flaw. While they survived relatively well on short texts, their performance would be halved when facing complex and long agent tasks [Source Title].
Recently, researchers have brilliantly broken through this limitation with a new quantization technique called NariQ. Let’s examine step-by-step what the secret is that shrinks the model size to the eye of a needle while fully preserving its genius agent capabilities.
Why Is This Important?
As artificial intelligence technology advances, model sizes are becoming increasingly massive. Running a model with hundreds of billions of parameters (adjustable numerical values inside an AI model that act like human synapses) in its original state (BF16, 16-bit floating-point precision) requires astronomical server costs and massive power. For ordinary developers or companies, using high-performance AI freely in local environments or cost-effective infrastructure is a pie in the sky.
To solve this, low-bit quantization—which lowers the number of bits occupied by each weight (weight values representing the importance of neural network connections)—has been widely used. For example, compressing a model to under 3 bits dramatically reduces storage space and memory usage.
However, a major wall was hit here. Many existing low-bit quantization methods were calibrated based on short web texts on the internet, and performance evaluations were also centered around short-form tests or language understanding (Perplexity) [Source Title].
The cost of this mismatch between theory and actual environment was high. Simply put, it was like practicing reading thick novels or encyclopedias only to lose one’s way when assigned practical work at a company. In fact, the Qwen3.8-27B model compressed using conventional methods showed performance comparable to the original (BF16) on relatively short tests like AIME, but achieved only up to 72% of its original performance on real-world long-horizon agent benchmarks like DeepSWE [Source Title]. In other words, it was a half-dieted model that passed tests well but easily tired or lost its way when deployed to actual work.
NariQ directly targets this exact point. It emerged as the first quantization technique that achieves performance identical to the original in complex practical work and tool-call-heavy long-horizon agent tasks while extremely reducing the model size to under 3 bits [Source Title].
Understanding It Easily
To easily understand the principles of quantization and NariQ, let’s use an analogy of chef training.
Stage 1: A Chef Who Studied Only Textbooks vs. A Chef Trained in a Real Kitchen
Conventional quantization methods calibrated models using only well-structured, short texts floating on the internet (cooking textbook theory) when compressing AI. As a result, the AI solved virtual theory tests well, but began to panic when placed in urgent situations in real kitchens where orders piled up and customers kept coming (long-horizon agent sessions).
On the other hand, NariQ (Trellis-coded post-training quantization method, a technology that compresses and calibrates models while minimizing information loss using trellis coding structures) took a completely different approach. NariQ directly utilized vivid tokens generated and interacted with by the full-precision (BF16) model as it called tools and held conversations within actual agent sessions as training and calibration data [Source Title].
Simply put, instead of making a chef memorize only a recipe book, training was conducted by observing the actual movement paths and know-how of a head chef in a busy restaurant kitchen. Thanks to this, even when the model’s size was reduced, it could adapt perfectly without wandering in actual agent tasks.
Stage 2: Limits of Compression, How Far Can We Reduce?
The researchers thoroughly verified NariQ’s performance across various bit levels using the Qwen3.8-27B model [Source Title].
- 3.0 bits per weight (using 3.0 bits per parameter)
- 2.5 bits per weight (using 2.5 bits per parameter)
- 2.0 bits per weight (using 2.0 bits per parameter)
Metaphorically speaking, this is like compressing an immensely thick encyclopedia into concise summary notes of 30%, 25%, and 20% thickness, respectively. Surprisingly, models compressed to 3.0 bits and 2.5 bits showed near-flawless performance with almost unnoticeable performance drops compared to the original (BF16) model [Source Title]. However, checkpoints further reduced to the 2.0-bit level exhibited slight performance limitations (trailing) [Source Title].
Current Status
NariQ’s performance was rigorously proven through several recognized benchmark (standard test sets for performance evaluation) tests. The researchers compressed the Qwen3.8-27B model with NariQ and put it to the test against demanding evaluation metrics such as [Source Title]:
- DeepSWE: A representative long-horizon agent benchmark handling complex software engineering problems. While existing low-bit quantization models stayed at 72% of original performance, NariQ (3.0 and 2.5 bits) fully maintained performance equivalent to the original (BF16) [Source Title].
- Terminal Bench: An agent evaluation environment performing long-term tasks by combining multiple commands in a terminal (command-line interface) environment. This also recorded high accuracy at the original level [Source Title].
- AIME and RealWorldQA: In benchmarks measuring mathematical reasoning and real-world visual/logical understanding, NariQ compressed models performed side-by-side without losing any of the original performance [Source Title].
Long-horizon agent tasks easily accumulate errors because tools must be invoked across numerous steps within a long context exceeding 100K tokens [Source Title]. This is the industry’s first instance where an ultra-low-bit quantization model under 3 bits successfully completed this long-distance race identically to the original [Source Title].
Currently, the researchers have made the fruits of this technology publicly available as open source so anyone can use them. Related files are distributed via Qwen3.8-27B-NariQ-GGUF (https://huggingface.co/nari-labs/Qwen3.8-27B-NariQ-GGUF), and inference tasks can be easily used through a dedicated forked llama.cpp (https://github.com/nari-labs/nari-llama.cpp) [Source Title].
What Lies Ahead?
The emergence of NariQ heralds practical changes in the AI ecosystem. Hitherto, running high-performance AI agents on local computers or cost-limited environments was blocked by massive hardware walls. However, if the core intelligence of agents is maintained even at ultra-low bits below 3 bits, a path opens up to run powerful autonomous AI agents effortlessly on personal computers or edge devices (devices around the user rather than servers).
Going forward, research optimizing compression algorithms tailored to “what AI is actually used for (agent workflows)” beyond simply reducing model size will become the mainstream trend. The day we meet smart AI assistants on our computers that code, analyze documents, and freely handle tools over long breaths of 100K+ tokens is approaching fast.
AI’s Perspective
NariQ demonstrates well that in the ‘agent era’, where AI goes beyond simple knowledge memorization to use tools by itself and work in long breaths, compression techniques must also resemble practical work patterns. This achievement of preserving practical skills while breaking down the capacity wall will serve as a powerful priming water for the popularization of on-device agents.
References
- NariQ: Sub-3-Bit Quantization for Long-Horizon Agentic Tasks - https://arxiv.org/abs/2611.26157
- Calibration with short web texts
- Calibration using tokens generated by the model during agent sessions
- Improving image generation speed
- 1.0 bit
- 3.0 and 2.5 bits
- 8.0 bits
- Tool calls across multiple steps in a context of over 100K tokens
- Single-sentence translation
- Simple image classification