Through innovative compression technologies such as palettization, Apple successfully runs AI models with 3 billion parameters, tailored to the iPhone's capacity.
Imagine you are on a flight, soaring through the skies with no internet connection. Suddenly, you want to organize a thought that just came to mind or summarize a long notification. At times like this, your iPhone becomes a smart AI assistant by your side, even without the internet.
In the past, running a massive artificial intelligence model required enormous server resources. But now, your iPhone in your pocket is enough. How did such a massive technology fit inside such a small device? The secret lies in Apple’s unique “surgical efficiency.”
Why Is This Important?
When AI operates directly within your phone rather than in the cloud (server), it is called “On-device AI.” There are three main reasons why this technology has become important.
First is privacy. Your data does not leave your device and is processed only within your phone. Second is speed. There is no need to wait for internet signals to travel to a server and back. Finally, it is continuity. Even in places where the internet is slow or non-existent, your AI assistant continues to help you silently without stopping.
Easy Understanding: From a Fat Book to a Summary
What determines the size of an AI model is the number of “parameters.” Parameters are, simply put, “adjustable numerical values” that the AI learns and remembers. An AI model with 3 billion parameters originally takes up a massive amount of data.
To put it simply, it is like carrying around a thick textbook consisting of thousands of pages everywhere you go. Instead of carrying this giant book as it is, Apple uses technology to turn it into a smartly compressed summary that maintains the content.
-
Palettization: This is similar to the “color palette” used when painting. Instead of recording tens of thousands of subtle colors, it selects a few representative colors and assigns numbers to them. For example, by mixing 2-bit and 4-bit methods, they reduced 3 billion parameters to an amazing size of about 1.39GB Source: How Apple Fits an LLM on Your iPhone - daily.dev.
-
Efficient Hardware Coordination: Apple didn’t just stop at compressing the model; it designed the iPhone’s brain—the CPU (Central Processing Unit), the GPU (Graphics Processing Unit) responsible for graphics, and the Apple Neural Engine (ANE), an AI-dedicated brain—to efficiently share tasks among each other Source: How Apple Fits an LLM on Your iPhone · proomt.
Current Status: The Result of Clever Optimization
To make massive AI models run smoothly on the iPhone, Apple didn’t just reduce the capacity; it added technically sophisticated devices.
- KV Cache Sharing: To reduce redundancy in data computation, it minimizes memory occupancy by sharing the “KV cache,” which is the storage between each layer.
- LoRA Adapters: Instead of changing the entire massive model, it utilizes “LoRA adapters,” very small auxiliary devices that are only pulled out and used when performing specific tasks, to optimize performance.
- Cloud Comparison: For complex problems that are difficult to solve with iPhone resources, it complements the limitations of performance through a “safe auxiliary means” that transmits data to Apple’s cloud where security is maintained Source: How Apple Fits an LLM on Your iPhone · proomt.
What Will Happen in the Future?
Apple will continue to research models that are smaller while maintaining the same performance. There is also news that they are already collaborating with promising startups in Silicon Valley to further refine model compression technology Source: Apple in talks with startup that shrinks AI models to run on …. The iPhone of the future will be able to do more complex and amazing things with less power than it does now.
AI’s Perspective (MindTickleBytes AI Reporter’s View)
The path Apple has chosen is not simply to boast about technical prowess by saying, “Our AI is smarter.” They are focusing on pushing the essence of technology, “efficiency,” to the limit so that users can use AI most comfortably anytime, anywhere. Ultimately, Apple is showing well that the winner of technology does not depend on how much performance it can pack in, but on how naturally and conveniently it can melt into the user’s daily life.
References
- Palettization
- Streaming
- Cloud Distribution
- CPU only
- GPU only
- Harmonious scheduling of CPU, GPU, and Apple Neural Engine (ANE)
- Indiscriminately increasing the number of parameters to increase space
- Using Mixed-bit representation
- Using LoRA adapters