Yorkie

Yorkie is an open source document store for building collaborative editing applications. Yorkie uses JSON-like documents(CRDT) with optional types.

Yorkie consists of three main components: Client, Document and Agent.

 Client "A" (Go)                 Agent                        Mongo DB
┌───────────────────┐           ┌────────────────────────┐   ┌───────────┐
│  Document "D-1"   │◄─Changes─►│  Collection "C-1"      │   │ Changes   │
│  { a: 1, b: {} }  │           │ ┌───────────────────┐  │◄─►│ Snapshots │
└───────────────────┘           │ │  Document "D-1"   │  │   └───────────┘
 Client "B" (JS)                │ │  { a: 2, b: {} }  │  │
┌───────────────────┐           │ │                   │  │
│  Document "D-1"   │◄─Changes─►│ │  Document "D-2"   │  │
│  { a: 2, b: {} }  │           │ │  { a: 3, b: {} }  │  │
└───────────────────┘           │ └───────────────────┘  │
 Client "C" (Admin)             │                        │
┌────────────────────┐          └────────────────────────┘
│  Query "Q-1"       │               ▲
│ db[c-1].find({a:2})├─MongoDB Query─┘
└────────────────────┘

Clients can have a replica of the document representing an application model locally on several devices.
Each client can independently update the document on their local device, even while offline.
When a network connection is available, the client figures out which changes need to be synced from one device to another, and brings them into the same state.
If the document was changed concurrently on different devices, Yorkie automatically syncs the changes, so that every replica ends up in the same state with resolving conflict.

Agent and SDKs

Agent: https://github.com/yorkie-team/yorkie
JS SDK: https://github.com/yorkie-team/yorkie-js-sdk
Go Client: https://github.com/yorkie-team/yorkie/tree/main/client

Quick Start

The easiest way to install yorkie is from pre-built binaries:

Download the compressed archive file for your platform from Releases.
Unpack the archive file. This results in a directory containing the binaries.
Add the executable binaries to your path. For example, rename and/or move the binaries to a directory in your path (like /usr/local/bin), or add the directory created by the previous step to your path.
From a shell, test that yorkie is in your path:

$ yorkie --version
Yorkie: 0.1.8
...

Yorkie uses MongoDB to store its data. To start MongoDB, type docker-compose -f docker/docker-compose.yml up -d.

Next, let's start a Yorkie agent. Agent runs until they're told to quit and handle the communication of maintenance tasks of Agent. and start the agent:

$ yorkie agent

Use the -c option to change settings such as the MongoDB connectionURI.

$ yorkie agent -c yorkie.yml

The configuration file with default values is shown below.

https://github.com/yorkie-team/yorkie/blob/main/yorkie/config.sample.yml

Documentation

Full, comprehensive documentation is viewable on the Yorkie website:

https://yorkie.dev/docs/main/

Developing Yorkie

For building Yorkie, You'll first need Go installed (version 1.17+ is required). Make sure you have Go properly installed, including setting up your GOPATH. Then download a pre-built binary from release page and install the protobuf compiler (version 3.4.0+ is required).

We need to install Golang packages to build Yorkie locally. You can run make tools to install the required packages.

Next, clone this repository into some local directory and then just type make build. In a few moments, you'll have a working yorkie executable:

$ make build
...
$ bin/yorkie

Tests can be run by typing make test.

NOTE: make test includes integration tests that require local applications such as MongoDB, etcd. To start them, type docker-compose -f docker/docker-compose-ci.yml up --build -d.

If you make any changes to the code, run make fmt in order to automatically format the code according to Go standards.

Contributing

See CONTRIBUTING for details on submitting patches and the contribution workflow.

Contributors ✨

Thanks goes to these incredible people:

wonjerry/yorkie