LocalAI

💡 Get help - ❓FAQ 💭Discussions 💬 Discord 📖 Documentation website

💻 Quickstart 📣 News 🛫 Examples 🖼️ Models 🚀 Roadmap

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU.

🔥🔥 Hot topics / Roadmap

Roadmap

Upload file API: mudler#1703
Tools support: mudler#1715
LLaVa 1.6: mudler#1714
ROCm container images: mudler#1595
Intel GPU support (sycl): mudler#1653
Deprecation of old backends: mudler#1651
Mamba support: mudler#1589
Start and share models with config file: mudler#1522
🐸 Coqui: mudler#1489
Img2vid mudler#1442

Hot topics (looking for contributors):

Backends v2: mudler#1126
Improving UX v2: mudler#1373

If you want to help and contribute, issues up for grabs: https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3A%22up+for+grabs%22

💻 Getting started

For a detailed step-by-step introduction, refer to the Getting Started guide. For those in a hurry, here's a straightforward one-liner to launch a LocalAI instance with phi-2 using docker:

docker run -ti -p 8080:8080 localai/localai:v2.7.0-ffmpeg-core phi-2

🚀 Features

📖 Text generation with GPTs (llama.cpp, gpt4all.cpp, ... 📖 and more)
🗣 Text to Audio
🔈 Audio to Text (Audio transcription with whisper.cpp)
🎨 Image generation with stable diffusion
🔥 OpenAI functions 🆕
🧠 Embeddings generation for vector databases
✍️ Constrained grammars
🖼️ Download Models directly from Huggingface
🆕 Vision API

💻 Usage

Check out the Getting started section in our documentation.

🔗 Community and integrations

Build and deploy custom containers:

https://github.com/sozercan/aikit

WebUIs:

Model galleries

https://github.com/go-skynet/model-gallery

Auto Docker / Model setup

Other:

Helm chart https://github.com/go-skynet/helm-charts
VSCode extension https://github.com/badgooooor/localai-vscode-plugin
Local Smart assistant https://github.com/mudler/LocalAGI
Home Assistant https://github.com/sammcj/homeassistant-localai / https://github.com/drndos/hass-openai-custom-conversation
Discord bot https://github.com/mudler/LocalAGI/tree/main/examples/discord
Slack bot https://github.com/mudler/LocalAGI/tree/main/examples/slack
Telegram bot https://github.com/mudler/LocalAI/tree/master/examples/telegram-bot
Examples: https://github.com/mudler/LocalAI/tree/master/examples/

🔗 Resources

🆕 New! LLM finetuning guide
How to build locally
How to install in Kubernetes
Projects integrating LocalAI
How tos section (curated by our community)

📖 🎥 Media, Blogs, Social

Citation

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: The free, Open source OpenAI alternative},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/go-skynet/LocalAI}},

❤️ Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project:


Spectro Cloud
Spectro Cloud kindly supports LocalAI by providing GPU and computing resources to run tests on lamdalabs!

And a huge shout-out to individuals sponsoring the project by donating hardware or backing the project.

Sponsor list
JDAM00 (donating HW for the CI)

🌟 Star history

📖 License

LocalAI is a community-driven project created by Ettore Di Giacinto.

MIT - Author Ettore Di Giacinto

🙇 Acknowledgements

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

🤗 Contributors

This is a community project, a special thanks to our contributors! 🤗

kawais/LocalAI