My Browser is Running AI Locally? The Future of Web AI with Three-LLM

AI Summary

Thanks to Three-LLM and WebLLM technology, an era is dawning where AI runs directly inside the user's PC browser without a server connection.

Imagine this: You open your laptop in a café without an internet connection and ask an AI to summarize a long meeting document. In the past, you would have had to wait while watching a spinning loading icon as the AI connected to a cloud server (a remote computer connected to the internet). Now, the answer appears instantly, as if by magic. This is because your laptop itself has acquired a small “AI brain.” Recently emerging technologies like “Three-LLM” and “WebLLM” are making this magic possible.

Why It Matters

Until now, most AI we use has operated by receiving results processed by supercomputers in massive server rooms. However, this creates several problems.

First, maintaining servers costs a fortune. Second, the further away the server is, the slower the response time. Third, the user’s sensitive data must travel over the network to the server, raising privacy concerns. It is similar to having to travel to a very distant restaurant every time you want to eat a delicious meal.

These new web technologies are changing the game entirely. When a web browser runs AI directly, server costs are eliminated, and since all calculations are completed within your computer, concerns about information leaking externally are reduced. Additionally, instant reactions become possible without network loading times, enabling a much smoother AI experience. Reference 5

The Explainer

How can such smart AI run in a web browser? The key is a technology called “WebGPU.”

Simply put, a traditional web browser was an “office clerk” capable of only very simple calculations. However, WebGPU is like handing the browser a powerful “graphics-dedicated calculator.” This calculator is specialized for rendering complex graphics or processing the complex mathematical calculations of AI in parallel (doing many things at once).

Three-LLM goes a step further by converting the model’s mathematical structure (inference graph) into “shaders” (programs dedicated to the GPU) that Three.js can understand. Reference 8 To use an analogy, it translates the mathematical language the AI understands into the language computer graphics understands, allowing it to run directly.

WebLLM, on the other hand, is a full framework implemented in JavaScript (the standard language that makes web pages move). Reference 4 It is like having an independent “AI operating system” planted inside the browser, so if AI calculations become too heavy, it smartly manages them by delegating them to a separate “Web Worker” so that the browser screen does not freeze. Reference 6

Where We Stand

Currently, these technologies are developing rapidly. Three-LLM has already succeeded in running language models like GPT-2, SmolLM2, Qwen, and Phi directly in a web browser environment. Reference 8 Also, WebLLM is an open-source project that provides tools in the exact same format (API) as OpenAI, allowing any developer to easily add AI features to their websites. Reference 2, Reference 9

However, it is currently difficult for the browser to run the massive, hundred-billion-parameter (a metric of AI intelligence) models we use on smartphones. Currently, lightweight yet efficient AI optimized for browser environments is primarily being utilized. It is like using a fast and agile motorcycle instead of a heavy cargo truck.

What’s Next

In the future, AI will be “embedded” in every website we visit. While we currently have to open a browser and access AI services separately, websites will soon possess intelligence themselves. Functions like telling a website to “adjust the brightness of this photo” and having the browser correct it immediately without asking a server, or having the browser summarize a long article we are reading, will become standard. As web technology advances, the web browser we know will become a massive artificial intelligence toolbox. Reference 9, Reference 10

MindTickleBytes AI Reporter Opinion

Bringing AI out of the server and into the browser in the palm of our hands is the beginning of technical independence. Developers now face an era where they can provide powerful AI experiences to users without worrying about massive cloud costs. Just like solving all your worries in your own living room, AI has moved one step closer to us.

References

Share this article:

Test Your Understanding

Q1. What is the core technology Three-LLM uses to run models?

Python scripts
Three.js TSL compute shaders
Cloud APIs

Three-LLM converts a model's inference graph into Three.js TSL (Three.js Shading Language) compute shaders to run on WebGPU.

Q2. What language is WebLLM implemented in?

C++
Python
JavaScript

Unlike most inference engines implemented in C++ or Python, WebLLM is an open-source framework implemented in JavaScript.

Q3. What is the primary advantage of running AI within a web browser?

Always works without an internet connection
Eliminates server processing and reduces network latency
Unlimited model size

Running AI in a local browser eliminates the need for server processing and removes network round-trip time, reducing latency.