56,000 Words per Second? The Magic of AI 'GateGPT' Achieved at the Speed of an Old Clock
The secret behind GateGPT, which generates 56,000 tokens per second on an 80MHz chip slower than a smartphone. The principles of Transformers, KV cache, and FPGAs are explained easily for everyday readers.