A 14MB AI agent hidden in my smartphone? 'Needle2' is coming

Imagine this: You wake up in the morning and say to your smartwatch, “Adjust the house temperature to 22 degrees according to today’s schedule.” Your smartwatch understands and executes this request instantly without needing a server. This is because a lightweight AI, breathing easily on your wrist, is running instead of a massive, heavy AI.

Needle2, recently released by Cactus Compute, is a technology bringing this future forward. An astonishingly small 14MB AI model is poised to breathe life into the devices around us.

Why does this matter?

Until now, AI technology has only raced toward “bigger and more massive.” However, running Large Language Models (LLMs)—AI trained on vast data to write like humans—requires enormous server capacity and power. Consequently, running large AI directly on everyday devices like smartphones or smartwatches has been virtually impossible.

Ultra-lightweight models like Needle2 show us the potential for “On-device AI” (AI that runs on the device itself without an external server connection). It means you can enjoy instant AI services on smartphones, wearable devices, robots, and even mini-computers like the ESP32-S3 (microcontrollers). Since data does not leave for a server, it is advantageous for privacy protection, and AI agent functions (AI that performs commands on the user’s behalf) can be used even in environments with unstable internet connections.

Simple understanding: An ‘assistant’ instead of a ‘professor’

It’s easy to think of it this way: If existing Large Language Models are “knowledgeable professors” who carry all the world’s knowledge in their heads like an encyclopedia, Needle2 is a small, agile “skilled assistant.”

A knowledgeable professor may be good at conversation but can be clumsy at tasks like actually operating office equipment or running apps like an assistant. On the other hand, Needle2 has focused all its abilities on Tool calling (the ability for AI to directly control external apps or devices) and structured data extraction rather than engaging in casual chatter. This model, which has 26 million parameters (controllable numerical values where AI stores knowledge), is fast enough to process 1,000–6,000 tokens (units of words recognized by AI) per second on mobile devices.

Simply put, Needle2 is a “practical assistant” that is small and fast but capable of executing the tasks you assign accurately. It is also notable that this model has been trained to recognize when it is wrong and request assistance when necessary.

Current status

Currently, Needle2 is ready to operate in the following environments:

• Ultra-compact capacity: Consists of a binary file of just 14MB, requiring only about 28MB of RAM to run.
• Various platforms: Can be embedded in various devices, including smartphones, wearables, robots, smart homes, and automobiles.
• Technical characteristics: Released under the open-source Apache 2.0 license, allowing anyone to download model weights from Hugging Face and use them.
• Cloud integration: While it runs on the device by default, it also features a cloud fallback function if needed.

However, because it is not a general conversational AI, it is not suitable for the purpose of chatting with friends. It is a model specialized solely for agent tasks such as device control.

What will happen in the future?

Technology like Needle2 will fundamentally change how we use our devices. We may no longer need to find and click through complicated app menus one by one. Smartphone screens will now turn into places where AI executes commands on our behalf, not just spaces for searching.

In the future, even smaller models than 14MB may emerge, and the day will come when this model combines with more diverse devices to quietly help our lives. AI is no longer a massive entity existing inside a server; it will stay by your side in your pocket and on your wrist in a smaller, more practical form.

---

MindTickleBytes’ AI Reporter Perspective

If a massive model is the “peak of intelligence,” Needle2 is the “democratization of intelligence.” As technology becomes lighter, our lives become freer. Next time you look at your smartwatch, imagine a future where that small device becomes your assistant.

References

1. GitHub - cactus-compute/needle: 14MB foundation model for tiny devices; phones, wearables, smart home, and robots.
2. Cactus - On-device AI for Smartphones, Laptops & Edge

3. [Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

   Hacker News](https://news.ycombinator.com/item?id=48111896)

4. GitHub - jmccardle/cactus-needle: Cactus foundation model for tiny devices; 14mb, 26m params, 1-6k toks/sec on mobiles, wearables smart home and robots.

5. [Needle - Tiny LLM for Edge Devices

   EveryDev.ai](https://www.everydev.ai/tools/needle-cactus-compute)

6. Needle, a lightweight version of Gemini’s tool invocation functionality designed to run on smartphones, has been released, with developers touting its usefulness in building AI agents for mobile devices. - GIGAZINE

7. [Needle2- The14MBAgenticLLMforTiny Devices

   Cactus](https://cactuscompute.com/needle)

8. ShowHN:Needle2:14MBagenticLLMforphones,wearables,smarthomeandrobots.
9. Needle2:14MBagenticLLMtargetsphones,wearables, and robots
10. AgenticAIPhonesand the Future of Indian Banking

11. [Cactus NeedleAgenticLLMfortiny devices

    Vuink.com](https://vuink.com/post/pnpghfpbzchgr-d-dpbz/needle)