/pocket-ai

AI which can run offline on some small devices.

MIT LicenseMIT

Pocket AI

AI models can run offline on some small devices. You can take your LLM with you. It helps you stay up with some new AI technology when no internet connection.

Prediction

In the near future, all our smartphones will have AI chips with LLM models. Big companies like Google, Microsoft, and Apple are working on this, and we will pay a subscription to use these models. The presentation of the new smartphone will not be about new processors, memory, and camera but about new AI chips and new AI models. With technology like stable diffusion, you can add something to your photo, delete or even replace a person on a building. The new era will be the era of AI. Our life will change completely.

But now, we can use small devices like Raspberry PI, Rock PI, or NVIDIA Jetson Xavier NX to implement some of these concepts.

Models

  • MBZUAI/LaMini-Flan-T5-783M
  • MBZUAI/LaMini-T5-61M

Tesing

Rock PI 3A - 8Gb

NVIDIA Jetson Xavier NX - 6Gb

Orange Pi 5 Plus - 16Gb

soon ...