/holochain

Holographic storage for distributed applications -- a validating monotonic DHT "backed" by authoritative hashchains for data provenance (a Ceptr sub-project)

Primary LanguageGoGNU General Public License v3.0GPL-3.0

Holochain

Code Status Travis Codecov Go Report Card Gitter In Progress License: GPL v3 Twitter Follow

Holographic storage for distributed applications. A holochain is a monotonic distributed hash table (DHT) where every node enforces validation rules on data before publishing that data against the signed chains where the data originated.

In other words, a holochain functions very much like a blockchain without bottlenecks when it comes to enforcing validation rules, but is designed to be fully distributed with each node only needing to hold a small portion of the data instead of everything needing a full copy of a global ledger. This makes it feasible to run blockchain-like applications on devices as lightweight as mobile phones.

Code Status: Alpha. Not for production use. The code has not yet undergone a security audit. We expect to destructively restructure code APIs and data chains until Beta. Proof-of-concept was unveiled at our first hackathon (March 2017). Alpha 0 was released (October 2017).

Holochain Links: FAQ Developer Wiki White Paper GoDocs

Table of Contents

Installation

Developers Only: At this stage, holochain is only for use by developers (either developers of applications to run on holochains, or developers of the holochain software itself). App developers should bundle their app in an installer as either approach below is not for non-technical folks.

There are two approaches to installing holochain:

  1. as a standard Go language application for direct execution on your machine
  2. using docker for execution in a container.

Which you choose depends on your preference and your purpose. If you intend to develop holochain applications, then you should almost certainly use the docker approach as we provide a testing harness for running multiple holochain instances in a docker cluster. If you will be developing in Go on holochain itself then you will probably end up doing both.

Go Based Install

  1. Download Go. Download the "Archive" or "Installer" for version 1.8 or later for your CPU and OS. The "Source" download does not contain an executable and step 3 will fail.
  2. Install Go on your system. See platform specific instructions and hints below for making this work.
  3. Install the command line tool suite with:
$ go get -d -v github.com/metacurrency/holochain
$ cd $GOPATH/src/github.com/metacurrency/holochain
$ make
  1. Test that it works (should look something like this):
$ hcadmin -v
hcadmin version 0.0.x (holochain y)

Unix

(Unix includes macOS and Linux.) You'll need to have a working environment set up for Go version 1.8 or later. See the installation instructions for Go.

Most importantly you'll need to: (Almost all installation problems that have been reported stem from skipping one of these steps.)

  1. Export the $GOPATH variable in your shell profile.
  2. Add $GOPATH/bin to your $PATH in your shell profile.

For example, add the following to the end of your shell profile (usually ~/.bashrc or ~/.bash_profile):

export GOPATH="$HOME/go"
export PATH="$GOPATH/bin:$PATH"

Windows

First you'll need to install some necessary programs if you don't already have them.

  • Download Go. Download the "Archive" or "Installer" for version 1.8 or later for Windows and your CPU. The "Source" download does not contain an executable.
  • Install Windows git. Be sure to select the appropriate options so that git is accessible from the Windows command line.
  • Optional: Install GnuWin32 make.

Next, in your Control Panel, select System>Advanced system settings>Environment Variables... and under System Variables do the following:

  1. Add a new entry with the name GOPATH and the value %USERPROFILE%\go (Or your Go workspace folder).
  2. Double-click Path, and in the window that pops up add the following entries:
    • %GOPATH%\bin
    • C:\Go\bin (Or wherever you installed Go to+\bin).
    • C:\Program Files (x86)\GnuWin32\bin (Or wherever you installed GnuWin32 make to+\bin).

Docker Based Install

Using docker, you don't have to install Go first. Our docker scripts manage installation of Go, holochain dependencies and holochain. The docker installation can run alongside Local ("Go") installation of holochain, sharing config directories. See docker usage on our wiki for more on how this works.

  1. Install the latest version of Docker on your machine
    1. Docker Installation. The Community edition; stable is sufficient.
    2. See Docker Getting Started for help.
    3. It is recommended to add your user to the docker group as in: Post Installation Steps, rather than use sudo before all script commands. Holochain Apps cannot exploit the kinds of security concerns mentioned in the Post Installation Steps document.  
  2. Confirm that docker installation and permissions are working by running:
$ docker info
  1. Get our holochain repository from github:
$ git clone https://github.com/metacurrency/holochain.git holochain
$ cd holochain
  1. Build the holochain core with all dependencies
$ docker/build
  • the first time build is run on a machine, it will download Alpine unix, and install all holochain dependencies.
  • During development cycles, docker/build will just compile changes made to the holochain go code, and run tests
  1. To run holochain in your new environment, suitable to continue the walkthrough below in usage
$ docker/run
  1. This will put you into an new command shell that may behave differently than what you're used to. To exit this holochain (Alpine) shell, press Ctrl-D or type exit

Usage

These instructions are for using the holochain command line tool suite: hcadmin, hcdev and hcd. They should work equally well for Go based or docker based installation.

(Note that since Holochain is intended to be used behind distributed applications, end users should not have to do much through the command or may not have it installed at all, as the application will probably have wrapped up the holochain library internally.)

Each of the tools includes a help command, e.g., run hcadmin help or for sub-commands run hcadmin <COMMAND> help. For more detailed information, see the wiki page

The tool suite include these commands:

  • hcadmin for administering your installed holochain applications
  • hcd for running and serving a holochain application
  • hcdev for developing and testing holochain applications

Getting Started

The instructions below walk you through the basic steps necessary to run a holochain application.

Initializing the Holochain environment

$ hcadmin init 'your@emailaddress.here'

This command creates a ~/.holochain directory for storing all chain data, along with initial public/private key pairs based on the identity string provided as the second argument.

Joining a Holochain

You can use the hcadmin tool to join a pre-existing Holochain application by running the following command (replacing SOURCE_PATH with a path to an application's DNA and CHAIN_NAME with the name you'd like it to be stored as).

For example: hcadmin join ./examples/chat chat

Note: this command will be replaced by a package management command still in development.

Running a Holochain

Holochains run and serve their UI via local web sockets. This let's interface developers have a lot of freedom to build HTML/JavaScript files and drop them in that chain's UI directory. You start a holochain and activate it's UI with the hcd command:

$ hcd <CHAIN_NAME> [PORT]

Developing a Holochain

The hcdev tool allows you to:

  1. generate new holochain application source files by cloning from an existing application, from the a package file, or a simple empty template.
  2. run stand-alone or multi-node scenario tests
  3. run a holochain and serve it's UI for testing purposes
  4. dump out chain and dht data for inspection

Please see the wiki for more detailed documentation.

Note that the hcdev command creates a separate ~/.holochaindev directory for serving and managing chains, so your dev work won't interfere with any running holochain apps you may be using.

Test-driven Application Development

We have designed Holochain around test-driven development, so the DNA should contain tests to confirm that the rest of the DNA is functional. Our testing harness includes two types of testing, stand-alone and multi-instance scenarios. Stand-alone tests allow you to tests the functions you create in your application. However, testing a distributed application requires being able to spin up many instances of it and have them interact. Our docker cluster testing harness automates that process, and enables app developers to specify scenarios and roles and test instructions to run on multiple docker containers.

Please see the App-Testing documentation for details.

File Locations

By default holochain data and configuration files are assumed to be stored in the ~/.holochain directory. You can override this with the -path flag or by setting the HOLOPATH environment variable, e.g.:

$ hcadmin -path ~/mychains init '<my@other.identity>'
$ HOLOPATH=~/mychains hcadmin

You can use the form: hcadmin -path=/your/path/here but you must use the absolute path, as shell substitutions will not happen.

Logging

All the commands take a --debug flag which will turn on a number of different kinds of debugging. For running chains, you can also control exactly which of these logging types you wish to see in the chain's config.json file. You can also set the DEBUG environment variable to 0 or 1 to temporarily override your settings to turn everything on or off.

Architecture Overview and Documentation

Architecture information and application developer documentation is in our Holochain Wiki.

You can also look through auto-generated reference API on GoDocs

Holochain Core Development

We accept Pull Requests and welcome your participation.

Some helpful links: In Progress

If you'd like to get involved you can:

  • Contact us on Gitter to set up a pair coding session with one of our developers to learn the lay of the land.
  • join our dev documentation calls twice weekly on Tuesdays and Fridays.

Current Throughput graph:

Throughput Graph

Contribute

Contributors to this project are expected to follow our development protocols & practices.

Dependencies

This project depends on various parts of libp2p, which uses the gx package manager. All of which will be automatically installed by make by following the setup instructions above.

Tests

To compile and run all the tests:

$ cd $GOPATH/github.com/metacurrency/holochain
$ make test

If you want to use go test instead of make test, you'll need to do a couple extra things because of this project's dependency on gx:

  • Before running go test you need to run make work to configure the imports properly.
  • If you do this, before commiting you must also run make pub to revert the changes it makes.

The docker setup runs tests automatically during builds.

License

License: GPL v3

Copyright (C) 2017, The MetaCurrency Project (Eric Harris-Braun, Arthur Brock, et. al.)

This program is free software: you can redistribute it and/or modify it under the terms of the license provided in the LICENSE file (GPLv3). This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Note: We are considering other 'looser' licensing options (like MIT license) but at this stage are using GPL while we're getting the matter sorted out.

Acknowledgements

  • MetaCurrency & Ceptr: Holochains are a sub-project of Ceptr which is a semantic, distributed computing platform under development by the MetaCurrency Project.  
  • Ian Grigg: Some of our initial plans for this architecture were inspired in 2006 by his paper about Triple Entry Accounting and his work on Ricardian Contracts.  
  • Juan Benet & the IPFS team: For all their work on IPFS, libp2p, and various cool tools like multihash, multiaddress, etc. We use libP2P library for our transport layer and kademlia dht.  
  • Crypto Pioneers And of course the people who paved the road before us by writing good crypto libraries and preaching the blockchain gospel. Back in 2008, nobody understood what we were talking about when we started sharing our designs. The main reason people want it now, is because blockchains have opened their eyes to the power of decentralized architectures.