/LocalAI

:robot: Self-hosted, community-driven, local OpenAI compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU required. Runs ggml, gguf, GPTQ, onnx, TF compatible models: llama, llama2, gpt4all, rwkv, whisper, vicuna, koala, cerebras, falcon, dolly, starcoder, and many others

Primary LanguageGoMIT LicenseMIT



LocalAI

LocalAI forks LocalAI stars LocalAI pull-requests

πŸ’‘ Get help - ❓FAQ πŸ’­Discussions πŸ’¬ Discord πŸ“– Documentation website

πŸ’» Quickstart πŸ“£ News πŸ›« Examples πŸ–ΌοΈ Models

testsBuild and Releasebuild container imagesBump dependenciesArtifact Hub

LocalAI is a drop-in replacement REST API that's compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families that are compatible with the ggml format. Does not require GPU.

Follow LocalAI

Follow LocalAI_API Join LocalAI Discord Community

Connect with the Creator

Follow mudler_it Follow on Github

Share LocalAI Repository

Follow _LocalAI Share on Telegram Share on Reddit Buy Me A Coffee


In a nutshell:

  • Local, OpenAI drop-in alternative REST API. You own your data.
  • NO GPU required. NO Internet access is required either
    • Optional, GPU Acceleration is available in llama.cpp-compatible LLMs. See also the build section.
  • Supports multiple models
  • πŸƒ Once loaded the first time, it keep models loaded in memory for faster inference
  • ⚑ Doesn't shell-out, but uses C++ bindings for a faster inference and better performance.

LocalAI was created by Ettore Di Giacinto and is a community-driven project, focused on making the AI accessible to anyone. Any contribution, feedback and PR is welcome!

Note that this started just as a fun weekend project in order to try to create the necessary pieces for a full AI assistant like ChatGPT: the community is growing fast and we are working hard to make it better and more stable. If you want to help, please consider contributing (see below)!

πŸ”₯πŸ”₯ Hot topics / Roadmap

πŸš€ Features

πŸ“– πŸŽ₯ Media, Blogs, Social

πŸ’» Usage

Check out the Getting started section in our documentation.

πŸ’‘ Example: Use GPT4ALL-J model

See the documentation

πŸ”— Resources

❀️ Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project:

Spectro Cloud logo_600x600px_transparent bg
Spectro Cloud
Spectro Cloud kindly supports LocalAI by providing GPU and computing resources to run tests on lamdalabs!

🌟 Star history

LocalAI Star history Chart

πŸ“– License

LocalAI is a community-driven project created by Ettore Di Giacinto.

MIT - Author Ettore Di Giacinto

πŸ™‡ Acknowledgements

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

πŸ€— Contributors

This is a community project, a special thanks to our contributors! πŸ€—