🐸 FrogBase

Create navigable knowledge from multi-media content

FrogBase (previously whisper-ui) simplifies the download-transcribe-embed-index workflow for multi-media content. It does so by linking content from various platforms (yt_dlp) with speech-to-text models (OpenAI's Whisper), image & text encoders (SentenceTransformers), and embedding stores (hnswlib).

from frogbase import FrogBase
fb = FrogBase()
fb.demo()
fb.search("What is the name of the squeaky frog?")

Full Documentation (WIP).

FrogBase also comes with a ready-to-use UI for non-technical users!

whisper-ui-update-demo.mp4

Features

FrogBase currently provides functionality to:

Download media files from a wide range of platforms (YouTube, TikTok, Vimeo, etc.) using yt_dlp
Transcribe audio streams for downloaded & local files using OpenAI's Whisper
Embed transcribed text from corresponding video segments using Sentence Transformers
Index & search the embedded content using hnswlib

FrogBase also includes a Streamlit UI to provide a simple GUI for the above functionality enabling a locally hosted, interactive experience.

Software Developers

This section is for software developers who want to use FrogBase as a python package.

Install ffmpeg and FrogBase

sudo apt install ffmpeg
pip install frogbase

Import FrogBase and use it as follows -

from frogbase import FrogBase

fb = FrogBase()

sources = [
   "https://www.youtube.com/watch?v=HBxn56l9WcU",
   "https://www.youtube.com/@hayabhay"
]

fb.add(sources)\
   .transcribe()\
   .embed()\
   .index()

fb.search("What is the name of the squeaky frog?")

Non-technical Users

This section is for non-technical users who want to use FrogBase primarily through the accompanying Streamlit UI.

Download the latest release of FrogBase from here and unzip it. Or, you can also clone the repository console git clone https://github.com/hayabhay/frogbase.git
Install FrogBase dependencies manually and run the UI.

Note: This also requires ffmpeg to be installed on your system. You can install it using sudo apt install ffmpeg on Ubuntu.
1. Using pip
```
pip install frogbase streamlit
streamlit run ui/01_🏠_Home.py
```

[Coming soon] Instructions, environment for installation using Docker & Anaconda

DeeNihl/frogbase

🐸 FrogBase

Features

Quickstart

Software Developers

Non-technical Users

Links