/local.ai

🎒 local.ai - Run AI locally on your PC!

Primary LanguageTypeScriptGNU General Public License v3.0GPL-3.0

🎒 local.ai

A desktop app for local, private, secured AI experimentation. Included out-of-the box are:

  • A known-good model API and a model downloader, with descriptions such as recommended hardware specs, model license, blake3/sha256 hashes etc...
  • A simple note-taking app, with inference config PER note. The note and its config are output into plain text .mdx format
  • A model inference streaming server (/completion endpoint, similar to OpenAI)

It's made to be used alongside https://github.com/alexanderatallah/window.ai/ as a simple way to have a local inference server up and running in no time. window.ai + local.ai enable every web app to utilize AI without incurring any cost from either the developer or the user!

Right now, local.ai uses the https://github.com/rustformers/llm rust crate at its core. Check them out, they are super cool!

🚀 Install

Go to the site at https://www.localai.app/ and click the button for your machine's architecture. You can also find the build manually in the GitHub release page.

Windows and MacOS binaries are signed under Plasmo Corp. - a company owned by the author of this project (@louisgv).

You may also build from source!

📺 Demo

local.ai.demo.v0.2.x.mp4

🧵 Development

Here's how to run the project locally:

Prerequisites

  1. node >= 18
  2. rust >= 1.69
  3. pnpm >= 8

Workflow

git submodule update --init --recursive
pnpm i
pnpm dev

🪪 License

🤔 Trivia

Why the backpack?

Ties into the bring your own model concept -- Alex from window.ai

Why GPLv3?

Anything AI-related including their derivatives should be open-source for all to inspect. GPLv3 enforces this chain of open-source.

Is there a community?

Where should I ask question?

I made something with local.ai, where should I post it?

I have some nice things to say about local.ai, where should I post it?

  • Here
  • Also, consider giving the repo a Star ⭐️

The naming seems close to LocalAI?

  • When I first started the project and got the domain localai.app, I had no idea LocalAI was a thing. A friend of mine forwarded me a link to that project mid May, and I was like dang it, let's just add a dot and call it a day (for now...) - local "dot" ai vs LocalAI lol
  • We might rename the project.

Do you accept contribution/PR?

Absolutely - Please note that any contribution toward this repo shall be relicensed under GPLv3. There are many ways to contribute, such as: