⍼ Resin Search Engine

Resin is a remote HTTP search engine and an embedded library

Resin is a vector space index based search engine that's available as a HTTP service or as an embedded library.

How to use

Write a document remotely

HTTP POST [host]/write?collection=[collection]
(e.g. http://localhost/write?collection=mycollection)
Content-Type: application/json

[
	{
		"field1": "value1",
		"field2": "value2"
	}
]

Write a document locally

using (var database = new DocumentDatabase<string>(_directory, collectionId, model, strategy))
{
    foreach (var document in documents)
    {
        database.Write(document);
    }

    database.Commit();
}

Query

GET query

HTTP GET [host]/query/?collection=mycollection&q=[my_query]&field=field1&field=field2&select=field1&skip=0&take=10
(e.g. http://localhost/write?collection=mycollection&q=value1&field=field1&field=field2&select=field1&skip=0&take=10)
Accept: application/json

POST query

HTTP POST [host]/query/?select=field1&skip=0&take=10
Content-Type: application/json
Accept: application/json

{
	"and":
	{
		"collection": "film,music",
		"title": "rocky eye of the tiger",
		"or":
		{
			"title": "rambo",
			"or": 
			{
				"title": "cobra"
				"or":
				{
					"cast": "antonio banderas"
				}			
			}	
		},
		"and":
		{
			"year": 1980,
			"operator": "gt"
		},
		"not":
		{
			"title": "first blood"
		}
	}
}

Local query

using (var database = new DocumentDatabase<string>(_directory, collectionId, model, strategy))
{
    var queryParser = database.CreateQueryParser();
    var query = queryParser.Parse(collectionId, word, "title", "title", and:true, or:false, label:true);
    var result = database.Read(query, skip: 0, take: 1);
}

Document database

Resin stores data as document collections. It applies your prefered IModeland indexing strategy onto your data while you write and query it. The write pipeline produces a set of indices (graphs), one for each document field, that you may interact with by using the Resin read/write JSON HTTP API or programmatically.

Vector-based indices

Resin indices are binary search trees that create clusters of vectors that are similar to each other, as you populate them with your data. When a node is added to the graph its cosine angle, i.e. its similarity to other nodes, determine its position (path) within the graph.

Performance

Currently, Wikipedia size data sets produce indices capable of sub-second phrase searching.

You may also

build, validate and optimize indices using the command-line tool Sir.Cmd
read efficiently by specifying which fields to return in the JSON result
implement messaging formats such as XML (or any other, really) if JSON is not suitable for your use case
construct queries that join between fields and even between collections, that you may post as JSON to the read endpoint or create programatically.
construct any type of indexing scheme that produces any type of embeddings with virtually any dimensionality using either sparse or dense vectors.

kreeben/resin