/webgpu-llm-loader

A loader that lets you try running LLMs built for WebGPU.

Primary LanguageTypeScript

WebGPU LLM Loader

Getting Started

First, run the development server:

npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev

Open http://localhost:3000 with your browser to see the result.

You can choose to load a pre-built Llama 2 Model(7B, 13B, 70B) or a Custom Model that you build yourself.

If you select a Custom LLM Model, three inputs are required.

  • Model Local Id: The local_id from your mlc-chat-config.json.
  • Model URL: The URL of the Model you uploaded to huggingface.
  • WASM File: The .wasm file built for your WebGPU.

Learn More

To learn more about Next.js, take a look at the following resources:

You can check out GitHub repository - your feedback and contributions are welcome!