semanticrouter-go

Go Semantic Router is a superfast decision-making layer for your LLMs and agents written in pure Go .

Rather than waiting for slow LLM generations to make tool-use decisions, use the magic of semantic vector space to make those decisions — routing requests using configurable semantic meaning.

A pure-go package for abstractly computing similarity scores between a query vector embedding and a set of vector embeddings.

Installation

go get github.com/conneroisu/semanticrouter-go

Conversational Agents Example

// Package main shows how to use the semantic router to find the best route for a given utterance
// in the context of a veterinarian appointment.
package main

import (
	"context"
	"fmt"
	"os"

	"github.com/conneroisu/semanticrouter-go"
	"github.com/conneroisu/semanticrouter-go/encoders/ollama"
	"github.com/conneroisu/semanticrouter-go/stores/memory"
	"github.com/ollama/ollama/api"
)

// NoteworthyRoutes represents a set of routes that are noteworthy.
// noteworthy here means that the routes are likely to be relevant to a noteworthy conversation in a veterinarian appointment.
var NoteworthyRoutes = semanticrouter.Route{
	Name: "noteworthy",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is the best way to treat a dog with a cold?"},
		{Utterance: "my cat has been limping, what should I do?"},
	},
}

// ChitchatRoutes represents a set of routes that are chitchat.
// chitchat here means that the routes are likely to be relevant to a chitchat conversation in a veterinarian appointment.
var ChitchatRoutes = semanticrouter.Route{
	Name: "chitchat",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is your favorite color?"},
		{Utterance: "what is your favorite animal?"},
	},
}

// main runs the example.
func main() {
	if err := run(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

// run runs the example.
func run() error {
	ctx := context.Background()
	cli, err := api.ClientFromEnvironment()
	if err != nil {
		return fmt.Errorf("error creating client: %w", err)
	}
	router, err := semanticrouter.NewRouter(
		[]semanticrouter.Route{NoteworthyRoutes, ChitchatRoutes},
		&ollama.Encoder{
			Client: cli,
			Model:  "mxbai-embed-large",
		},
		memory.NewStore(),
	)
	if err != nil {
		return fmt.Errorf("error creating router: %w", err)
	}
	finding, p, err := router.Match(ctx, "how's the weather today?")
	if err != nil {
		fmt.Println("Error:", err)
	}
	fmt.Println("Found:", finding)
	fmt.Println("p:", p)
	return nil
}

Output:

Found: chitchat

Veterinarian Example

The following example shows how to use the semantic router to find the best route for a given utterance in the context of a veterinarian appointment.

The goal of the example is to decide whether spoken utterances are relevant to a noteworthy conversation or a chitchat conversation.

The Code Example:

// Package main shows how to use the semantic router to find the best route for a given utterance
// in the context of a veterinarian appointment.
package main

import (
	"context"
	"fmt"
	"os"

	"github.com/conneroisu/semanticrouter-go"
	"github.com/conneroisu/semanticrouter-go/encoders/ollama"
	"github.com/conneroisu/semanticrouter-go/stores/memory"
	"github.com/ollama/ollama/api"
)

// NoteworthyRoutes represents a set of routes that are noteworthy.
// noteworthy here means that the routes are likely to be relevant to a noteworthy conversation in a veterinarian appointment.
var NoteworthyRoutes = semanticrouter.Route{
	Name: "noteworthy",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is the best way to treat a dog with a cold?"},
		{Utterance: "my cat has been limping, what should I do?"},
	},
}

// ChitchatRoutes represents a set of routes that are chitchat.
// chitchat here means that the routes are likely to be relevant to a chitchat conversation in a veterinarian appointment.
var ChitchatRoutes = semanticrouter.Route{
	Name: "chitchat",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is your favorite color?"},
		{Utterance: "what is your favorite animal?"},
	},
}

// main runs the example.
func main() {
	if err := run(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

// run runs the example.
func run() error {
	ctx := context.Background()
	cli, err := api.ClientFromEnvironment()
	if err != nil {
		return fmt.Errorf("error creating client: %w", err)
	}
	router, err := semanticrouter.NewRouter(
		[]semanticrouter.Route{NoteworthyRoutes, ChitchatRoutes},
		&ollama.Encoder{
			Client: cli,
			Model:  "mxbai-embed-large",
		},
		memory.NewStore(),
	)
	if err != nil {
		return fmt.Errorf("error creating router: %w", err)
	}
	finding, p, err := router.Match(ctx, "how's the weather today?")
	if err != nil {
		fmt.Println("Error:", err)
	}
	fmt.Println("Found:", finding)
	fmt.Println("p:", p)
	return nil
}

The Output

The output of the veterinarian example is:

Found: chitchat

Development

Testing

To run the tests, run the following command:

make test

Making a new Store Implementation

Implement the Store interface in the stores package.

The store interface is defined as follows:

type Store interface { // size=16 (0x10)
	Storer
	Getter
	io.Closer
}

Store is an interface that defines a method, Store, which takes a []float64 and stores it in a some sort of data store, and a method, Get, which takes a string and returns a []float64 from the data store.

func (io.Closer) Close() error
func (Getter) Get(ctx context.Context, key string) ([]float64, error)
func (Storer) Store(ctx context.Context, keyValPair Utterance) error

semanticrouter.Store on pkg.go.dev

Code Generated Documentation

semanticrouter

import "github.com/conneroisu/semanticrouter-go"

Package semanticrouter is a package for abstractly computing similarity scores between a query vector embedding and a set of vector embeddings.

It provides a simple key-value store for embeddings and a router that can be used to find the best route for a given utterance.

The router uses a similarity score to determine the best route for a given utterance.

The semantic router is designed to be used in conjunction with LLMs and agents to provide a superfast decision-making layer.

Index

func NormalizeScores(sim []float64) []float64
type Encoder
type ErrEncoding
- func (e ErrEncoding) Error() string
type ErrGetEmbedding
- func (e ErrGetEmbedding) Error() string
type ErrNoRouteFound
- func (e ErrNoRouteFound) Error() string
type Getter
type Option
type Route
type Router
- func NewRouter(routes []Route, encoder Encoder, store Store, opts ...Option) (router *Router, err error)
- func (r *Router) Match(ctx context.Context, utterance string) (bestRoute *Route, bestScore float64, err error)
type Setter
type Store
type Utterance

func NormalizeScores

func NormalizeScores(sim []float64) []float64

NormalizeScores normalizes the similarity scores to a 0-1 range. The function takes a slice of float64 values representing the similarity scores.

The function takes a slice of float64 values representing the similarity scores and returns a slice of float64 values representing the normalized similarity scores.

type Encoder

Encoder represents a encoding driver in the semantic router.

It is an interface that defines a single method, Encode, which takes a string and returns a []float64 representing the embedding of the string.

type Encoder interface {
    Encode(ctx context.Context, utterance string) ([]float64, error)
}

type ErrEncoding

ErrEncoding is an error that is returned when an error occurs during encoding.

type ErrEncoding struct {
    Message string
}

func (ErrEncoding) Error

func (e ErrEncoding) Error() string

Error returns the error message.

type ErrGetEmbedding

ErrGetEmbedding is an error that is returned when an error occurs during getting an embedding.

type ErrGetEmbedding struct {
    Message string
    Storage Store
}

func (ErrGetEmbedding) Error

func (e ErrGetEmbedding) Error() string

Error returns the error message.

type ErrNoRouteFound

ErrNoRouteFound is an error that is returned when no route is found.

type ErrNoRouteFound struct {
    Message   string
    Utterance string
}

func (ErrNoRouteFound) Error

func (e ErrNoRouteFound) Error() string

Error returns the error message.

type Getter

Getter is an interface that defines a method, Get, which takes a string and returns a []float64 from the data store.

If the key does not exist, it returns an error.

type Getter interface {
    Get(ctx context.Context, key string) ([]float64, error)
}

type Option

Option is a function that configures a Router.

type Option func(*Router)

func WithEuclideanDistance

func WithEuclideanDistance(coefficient float64) Option

WithEuclideanDistance sets the EuclideanDistance function with a coefficient.

$$d(x, y) = \sqrt{\sum_{i=1}^{n}(x_i - y_i)^2}$$

func WithJaccardSimilarity

func WithJaccardSimilarity(coefficient float64) Option

WithJaccardSimilarity sets the JaccardSimilarity function with a coefficient.

$$J(A, B)=\frac{|A \cap B|}{|A \cup B|}$$

It adds the jaccard similarity to the comparision functions with the given coefficient.

func WithManhattanDistance

func WithManhattanDistance(coefficient float64) Option

WithManhattanDistance sets the ManhattanDistance function with a coefficient.

$$d(x, y) = |x_1 - y_1| + |x_2 - y_2| + ... + |x_n - y_n|$$

It adds the manhatten distance to the comparision functions with the given coefficient.

func WithPearsonCorrelation

func WithPearsonCorrelation(coefficient float64) Option

WithPearsonCorrelation sets the PearsonCorrelation function with a coefficient.

$$r=\frac{\sum\left(x_{i}-\bar{x}\right)\left(y_{i}-\bar{y}\right)}{\sqrt{\sum\left(x_{i}-\bar{x}\right)^{2} \sum\left(y_{i}-\bar{y}\right)^{2}}}$$

It adds the pearson correlation to the comparision functions with the given coefficient.

func WithSimilarityDotMatrix

func WithSimilarityDotMatrix(coefficient float64) Option

WithSimilarityDotMatrix sets the similarity function to use with a coefficient.

$$a \cdot b=\sum_{i=1}^{n} a_{i} b_{i}$$

It adds the similarity dot matrix to the comparision functions with the given coefficient.

func WithWorkers

func WithWorkers(workers int) Option

WithWorkers sets the number of workers to use for computing similarity scores.

type Route

Route represents a route in the semantic router.

It is a struct that contains a name and a slice of Utterances.

type Route struct {
    Name       string      // Name is the name of the route.
    Utterances []Utterance // Utterances is a slice of Utterances.
}

type Router

Router represents a semantic router.

Router is a struct that contains a slice of Routes and an Encoder.

Match can be called on a Router to find the best route for a given utterance.

type Router struct {
    Routes  []Route // Routes is a slice of Routes.
    Encoder Encoder // Encoder is an Encoder that encodes utterances into vectors.
    Storage Store   // Storage is a Store that stores the utterances.
    // contains filtered or unexported fields
}

func NewRouter

func NewRouter(routes []Route, encoder Encoder, store Store, opts ...Option) (router *Router, err error)

NewRouter creates a new semantic router.

func (*Router) Match

func (r *Router) Match(ctx context.Context, utterance string) (bestRoute *Route, bestScore float64, err error)

Match returns the route that matches the given utterance.

The score is the similarity score between the query vector and the index vector.

If the given context is canceled, the context's error is returned if it is non-nil.

type Setter

Setter is an interface that defines a method, Store, which takes a []float64 and stores it in a some sort of data store.

type Setter interface {
    Set(ctx context.Context, keyValPair Utterance) error
}

type Store

type Store interface {
    Setter
    Getter
    io.Closer
}

type Utterance

Utterance represents a utterance in the semantic router.

type Utterance struct {
    // ID is the ID of the utterance.
    ID  int
    // Utterance is the text of the utterance.
    Utterance string
    // Embed is the embedding of the utterance. It is a vector of floats.
    Embed embedding
}

Generated by gomarkdoc