cpu-inference

There are 26 repositories under cpu-inference topic.

  • kennethleungty/Llama-2-Open-Source-LLM-CPU-Inference

    Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

    Language:Python9671324211

  • CoderLSF/fast-llama

    Runs LLaMA with Extremely HIGH speed

    Language:C++928710

  • rbitr/llm.f90

    LLM inference in Fortran

    Language:Fortran61558
  • homelab

    jozsefszalma/homelab

    The bare metal in my basement

    21102

  • lucienhuangfu/eLLM

    eLLM Infers LLM on CPUs in Real Time

    Language:Rust10

  • yybit/pllm

    Portable LLM - A rust library for LLM inference

    Language:Rust9101

  • laelhalawani/gguf_llama

    Wrapper for simplified use of Llama2 GGUF quantized models.

    Language:Python7121

  • codito/arey

    Simple large language model playground app

    Language:Rust6400

  • JohnClaw/chatllm.v

    V-lang api wrapper for llm-inference chatllm.cpp

    Language:C6100

  • lahcenkh/rag-network-docs

    Privacy-focused RAG chatbot for network documentation. Chat with your PDFs locally using Ollama, Chroma & LangChain. CPU-only, fully offline.

    Language:Python6

  • JohnClaw/chatllm.cs

    C# api wrapper for llm-inference chatllm.cpp

    Language:C#4100

  • JohnClaw/chatllm.vb

    VB.NET api wrapper for llm-inference chatllm.cpp

    Language:Visual Basic .NET4100

  • BjornMelin/local-llm-workbench

    🧠 A comprehensive toolkit for benchmarking, optimizing, and deploying local Large Language Models. Includes performance testing tools, optimized configurations for CPU/GPU/hybrid setups, and detailed guides to maximize LLM performance on your hardware.

    Language:Shell310

  • JohnClaw/chatllm.nim

    Nim api-wrapper for llm-inference chatllm.cpp

    Language:C3100

  • Nishant1998/PlantAi

    PlantAi is a ResNet-based CNN model trained on the PlantVillage dataset to classify plant leaf images as healthy or diseased. This repository includes PyTorch training code, tools to convert the model to TensorFlow Lite (TFLite) for deployment, and an Android app integrating the model for real-time leaf disease detection from camera images.

    Language:Java30

  • chinese-soup/cbot-telegram-whisper

    Simple bot that transcribes Telegram voice messages. Powered by go-telegram-bot-api & whisper.cpp Go bindings.

    Language:Go2100

  • JohnClaw/chatllm.rs

    rust api wrapper for llm-inference chatllm.cpp

    Language:Rust2100

  • JohnClaw/chatllm.d

    D-lang api wrapper for llm-inference chatllm.cpp

    Language:D1100

  • JohnClaw/chatllm.kt

    kotlin api wrapper for llm-inference chatllm.cpp

    Language:C1100

  • JohnClaw/chatllm.lua

    lua api wrapper for llm-inference chatllm.cpp

    Language:Lua1100

  • JohnClaw/gemma-2-2b-it.cs

    gemma-2-2b-it int8 cpu inference in one file of pure C#

    Language:C#10

  • JohnClaw/llama-3.2-1b.vb

    llama 3.2 1b fp16 cpu inference in one file of pure VB.NET

    Language:Visual Basic .NET1100

  • JohnClaw/qwen3.java

    Java-port of qwen3.c

    Language:Java110

  • SIYAKS-ARES/survival-with-llms

    The Ark Project: Selecting the perfect AI model to reboot civilization from a 64GB USB drive. Comprehensive analysis of open-source LLMs under extreme constraints, with final recommendation: Meta Llama 3.1 70B Instruct (Q6_K GGUF). Includes interactive tools, detailed comparisons, and complete implementation guide for offline deployment.

    Language:HTML1

  • bhimanbaghel/llama-streamlit-app

    🤖 AI Text Completion App built with Streamlit and Llama-3.2-1B. Generate creative text completions with an intuitive web interface. GPU & CPU optimized, easy to deploy, perfect for content creation and AI experimentation.

    Language:Python

  • ukkit/llama-chat

    Lightweight web UI for llama.cpp with dynamic model switching, chat history & markdown support. No GPU required. Perfect for local AI development.

    Language:Shell