defenseunicorns/leapfrogai

Production-ready Generative AI for local, cloud native, airgap, and edge deployments.

Table of Contents

• Table of Contents
• Overview
• Why Host Your Own LLM?
• Components
  • Getting Started
  • API
  • Backends
  • Image Hardening
  • SDK
  • User Interface
• Community

Overview

LeapfrogAI is a self-hosted AI platform designed to be deployed in air-gapped environments. This project aims to bring sophisticated AI solutions to air-gapped resource-constrained environments, by enabling the hosting all requisite components of an AI stack.

Our services include vector databases, model backends, API, and UI. These capabilities can be easily accessed and integrated with your existing infrastructure, ensuring the power of AI can be harnessed irrespective of your environment's limitations.

Why Host Your Own LLM?

Large Language Models (LLMs) are a powerful resource for AI-driven decision making, content generation, and more. How can LeapfrogAI bring AI to your mission?

• Data Independence: Sending sensitive information to a third-party service may not be suitable or permissible for all types of data or organizations. By hosting your own LLM, you retain full control over your data.

• Scalability: Pay-as-you-go AI services can become expensive, especially when large volumes of data are involved and require constant connectivity. Running your own LLM can often be a more cost-effective solution for missions of all sizes.

• Mission Integration: By hosting your own LLM, you have the ability to customize the model's parameters, training data, and more, tailoring the AI to your specific needs.

Components

Getting Started

GitHub Repo:

• Tadpole

Tadpole is a simple way to get LeapfrogAI up and running locally. While not intended for production, it helps the user to understand the various components of LeapfrogAI and how they interact.

API

GitHub Repo:

• leapfrog-api

LeapfrogAI provides an API that closely matches that of OpenAI's. This feature allows tools that have been built with OpenAI/ChatGPT to function seamlessly with a LeapfrogAI backend.

Backends

GitHub Repos:

Repo	AMD64 Support	ARM64 Support	Cuda Support	Docker Ready	K8s Ready	Zarf Ready
llama-cpp-python	✅	🚧	🚧	✅	✅	✅
whisper	✅	🚧	🚧	✅	✅	✅
ctransformers	✅	🚧	🚧	✅	✅	✅
instructor-xl	✅	🚧	🚧	🚧	🚧	🚧
vllm	❌	❌	🚧	🚧	🚧	🚧

LeapfrogAI provides several backends for a variety of use cases.

Image Hardening

GitHub Repo:

• leapfrogai-images

LeapfrogAI leverages Chainguard's apko to harden base python images - pinning Python versions to the latest supported version by the other components of the LeapfrogAI stack.

SDK

GitHub Repo: leapfrogai-sdk

The LeapfrogAI SDK provides a standard set of protobuff and python utilities for implementing backends and gRPC.

User Interface

GitHub Repo:

• leapfrog-ui

LeapfrogAI provides some options of UI to get started with common use-cases such as chat, summarization, and transcription.

Advanced Deployments

GitHub Repo:

• leapfrogai-deployment

These instructions will assist advanced users in standing up the latest production version of LeapfrogAI on Kubernetes.

Community

LeapfrogAI is supported by a community of users and contributors, including:

• Defense Unicorns
• Beast Code
• Chainguard
• Exovera
• Hypergiant
• Pulze
• SOSi
• United States Navy
• United States Air Force
• United States Space Force

Want to add your organization or logo to this list? Open a PR!