In an AI world dominated by transformer architectures, a project attempting to build a unique non-transformer AI model, 'PSSA', using only the Rust language—without established tools like PyTorch or TensorFlow—is garnering attention.
Imagine a chef who creates elaborate dishes using only their hands and a single knife, without the common, complex modular kitchen appliances we are used to. Because they aren’t using off-the-shelf molds, the chef’s skill is laid bare, but they have complete control over the entire cooking process. This is exactly what is happening in the AI industry right now.
Why does this matter?
For the past several years, the AI ecosystem has seen the “Transformer”—the core AI structure that understands context by identifying relationships between words in a sentence—become the de facto standard for virtually all language models. Recently, however, a project called “PSSA” has emerged, causing waves in this rigid order. GitHub - Sparticle62ops/pssa Escaping the authority of Transformers, this project is a non-transformer language model built from scratch using only “Rust,” a systems programming language. PSSA: A non-transformer language model While it may seem like a purely technical difference to the average person, it suggests revolutionary potential that the “way of making AI” can be fundamentally different.
In simple terms: Abandoning the “off-the-shelf mold” for AI
Currently, most AI models are developed on top of vast toolboxes like PyTorch or TensorFlow. It’s a method of taking proven existing parts and arranging them, much like assembling LEGO bricks. PSSA, however, rejects these existing machine learning frameworks entirely. GitHub - Sparticle62ops/pssa
To use an analogy, if a Transformer model is a machine assembled from parts produced by a standardized process, PSSA is the result of craftsmanship, where the raw materials are refined and the screws are filed by hand. Instead of reading the entire text at once, this model processes tokens (word fragments) sequentially and manages weights (knowledge values acquired through learning) on its own. GitHub - Sparticle62ops/pssa
Current status: How far has it come?
Of course, PSSA is not yet at a level to replace the massive AI models we use today. The AI industry is currently advancing rapidly thanks to the excellent efficiency and versatility of the mainstream Transformer architecture. 【AI】打破Transformer的霸权？
Nevertheless, the attempt to rebuild AI from the ground up using a language like Rust, which excels in direct hardware control, is very encouraging. Rust has already proven its performance in the AI field, being actively used for inference engines and vector database management. Rust Ecosystem for AI & LLMs The emergence of PSSA means that the Rust ecosystem has reached a stage where it can now design the very core “brain structure” of AI itself.
What will happen in the future?
PSSA’s experiment poses a fundamental question: “Must we strictly insist on the existing Transformer mold?” 【AI】打破Transformer的霸权？ If such non-transformer models built with Rust prove higher efficiency, they could become a new standard for devices that require lighter, faster response speeds, such as smartphones or IoT appliances.
Building large-scale language models from scratch, of course, is a task that requires immense cost and engineering time. Training a Language Model End-to-End in Rust But even if it is not a finished product right now, this kind of challenge to design the very bottom of technology is a valuable asset that will eventually make the AI ecosystem more diverse and healthy.
MindTickleBytes AI Reporter’s Perspective
Challenging mainstream architectures is the engine of AI advancement. Some might ask, “Why go through the trouble?” but the act of “building AI from the ground up using only Rust” is a clear indicator of how deeply we can understand and control technology. Watching what kind of massive ripples the small pebble thrown by PSSA will create in the AI industry will certainly be an interesting spectacle.
References
- GitHub - Sparticle62ops/pssa
- PSSA: A non-transformer language model written from scratch in Rust
- 【AI】Breaking the Hegemony of Transformers? Chatting about PSSA: Building a non-Transformer language model from scratch in Rust
- Rust Ecosystem for AI & LLMs - HackMD
- Training a Language Model End-to-End in Rust: An Experience Report
- A complete clone of the GPT-4 model
- Built from scratch using only the Rust language without frameworks
- Optimized based on PyTorch
- Follows the Transformer architecture as-is
- It is a non-transformer model
- It is an image generation-only model
- Processes the entire sentence at once
- Processes sequentially token by token
- Converts image data into tokens