`blockoli` 🥦🔎

Blockoli is a high-performance tool for code indexing, embedding generation and semantic search tool for use with LLMs. blockoli is built in Rust and uses the ASTerisk crate for semantic code parsing. blockoli allows you to efficiently index, store, and search code blocks and their embeddings using vector similarity.

Features

Index code blocks from a codebase 📂🔍
Generate vector embeddings for code blocks using a pre-trained model 🤖🧠
Store code blocks and their embeddings in a SQLite database (Support for Qdrant soon!) 💾🗄️
Perform efficient similarity search on code blocks using vector embeddings (k-d tree algorithm) 🔎⚡
REST API for easy integration with other tools and platforms 🌐🔗
Fast and memory-efficient implementation using Rust ⚡💻

Installation (from source)

Ensure you have Rust installed on your system. You can install it from the official Rust website: https://www.rust-lang.org/tools/install
Clone the blockoli repository:

git clone https://github.com/stitionai/blockoli.git
cd blockoli

Download tree-sitter grammar files

mkdir grammars
chmod +x get-grammar.sh
./get-grammar.sh

Build the project:

cargo build --release

Run the server:

./target/release/blockoli <port>

Replace <port> with the desired port number for the server.

Usage

Blockoli provides a REST API for indexing and searching code blocks. Here are some example API endpoints:

POST /project: Create a new project
GET /project/{project_name}: Get information about a project
DELETE /project/{project_name}: Delete a project
POST /project/generate: Generate embeddings for code blocks in a project
POST /search/{code_block}: Search for similar code blocks in a project
POST /get_blocks/{project_name}: Get all function blocks in a project
POST /search_blocks/{function_block}: Search for function blocks in a project
POST /search_by_function/{function_name}: Search for blocks by function name in a project

Refer to the routes.rs file for detailed information about each API endpoint and its parameters.

Configuration

ASTerisk uses a configuration file named asterisk.toml for specifying indexing options. Modify this file to customize the behavior of the indexer according to your needs.

Contribution Guidelines

Contributions to Blockoli are welcome! If you find a bug, have a feature request, or want to contribute code improvements, please open an issue or submit a pull request on the GitHub repository.

When contributing code, please ensure that your changes are well-tested and follow the Rust coding conventions and style guidelines.

Contribution

Ways to contribute:

Suggest a feature
Report a bug
Fix something and open a pull request
Help document the code
Spread the word

License

Licensed under the MIT License, see LICENSE for more information.

Liked the project?

Support the project by starring the repository. ⭐

getAsterisk/blockoli

blockoli 🥦🔎