Textile's Threads protocol & database (js-threads)

A protocol & event-sourced database for decentralized user-siloed data written in Typescript

Using Threads

To get started using Threads, check out the docs site and API documentation.

Getting help

The Textile/Threads developers/community are active on Slack and Twitter (@textileio), join us there for news, discussions, questions, and status updates. Also, check out our blog for the latest posts and announcements.

If you think you've found a bug in Threads, please file a Github issue. Take a look at our comprehensive contributor guide for details on how to get started.

Textile's Threads protocol & database (js-threads)
- Using Threads
- Getting help
- Background
  - Overview
- Install
- Usage
- Authentication
- Developing
- Releasing
- API
- Maintainers
- Contributing
- License

Background

Textile's Threads Protocol and Database provides an alternative architecture for data on the web. Threads aims to help power a new generation of web technologies by combining a novel use of event sourcing, Interplanetary Linked Data (IPLD), and access control to provide a distributed, scalable, and flexible database solution for decentralized applications. Threads is backed by a great deal of research and experience, and provides protocols for securely storing and sharing content-addressable data (on IPFS), with tooling to help standardize data creation and dissemination.

Want something specific? Take a look at our contributor guide for details on how to ask for features, or better yet, submit a PR yourself 😉!

Overview

Install

js-threads is implemented as a series of sub-packages. For general use, you can get most of what you need by simply installing and using the @textile/threads library.

npm install --save @textile/threads

If you are running js-threads in an environment that does not support WebSockets by default (such as Node), be sure to include it explicitly by installing the ws module:

npm i ws

Usage

For an introduction to use the Database class, read the Threads Introduction here. Additionally, there are a growing list of examples available.

Authentication

Textile provides remote Threads APIs you can use when developing your app. See @textile/hub (or docs.textile.io) for details on authenticating with these APIs, and how to set up your own local development peer.

Developing

Underlying the Threads Database are a number of ideas and technologies, which are outlined in detail in the Threads whitepaper. These components are all housed within this mono-repo, and include a set of core modules for creating Thread identities and keys (@textile/threads-core), as well as tooling for data encryption and encoding (@textile/threads-encoding), networking (with support for local (@textile/threads-network) and remote (@textile/threads-network-client) key management), and a local-first, event-sourced storage layer (@textile/threads-store). There are also two entry points for running Threads DB, including a local-first Javascript module (@textile/threads-database), and a remote-only, Javascript client (@textile/threads-client).

This mono-repo is all managed by lerna. You shouldn't have to do anything special to get started, however, here are a few steps that will make it easier to develop new functionality locally.

git clone git@github.com:textileio/js-threads.git
cd js-threads
npm install
npm run bootstrap

Then you are pretty much ready to go. You can run tests for all sub-packages from the root directory:

npm test
# Or run the node- or browser-specific tests separately
npm run test:node
npm run test:browser

Similarly, you can compile the Typescript-based sub-packages to Javascript all at once.

npm run build

Browsers

If you'd like to build browser bundles (say, using webpack), you may want to use something like the following in your webpack.config.js. Note however, that if you are working with a framework such as React or Angular, you might be better off letting that bundler handle the tree-shaking etc for you!

const path = require('path')

module.exports = {
  entry: './src/index.ts',
  devtool: 'inline-source-map',
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/,
      },
    ],
  },
  resolveLoader: {
    modules: ['../../node_modules'],
  },
  resolve: {
    modules: ['./node_modules'],
    ['.tsx', '.ts', '.js', 'json'],
    symlinks: false,
  output: {
    filename: './[name].js',
    path: path.resolve(process.cwd(), 'dist'),
    library: 'threads',
    libraryTarget: 'var',
  },
  optimization: {
    splitChunks: {
      chunks: 'all',
    },
    minimize: true,
  },
}

The the bundle(s) can be built with:

cd packages/threads
webpack --mode production --config webpack.config.js

Typescript

This project also uses incremental Typescript builds, so to take advantage of that (rather than building from scratch each time) use compile. You should notice significant speed-ups in your build process:

npm run compile

Releasing

This project utilises lerna's 'independent' release mode. So each time you run lerna publish, it will check for local updates and determine which packages have changed and need publishing. To do this locally, follow these steps:

Ensure you are on a clean master branch (we only want to release from master). If you have to, run lerna clean && lerna bootstrap to get to a clean state. If this produces new changes, then make sure you commit these (or reject them) before proceeding.
Ensure that webpack and webpack-cli are available. Ideally, install these globally. It is also a good idea to have typescript available globally. npm i -g webpack webpack-cli typescript
Ensure your git config does not specify signed tags. The lerna cli tool does not seem to be able to handle these at this time.
Ensure you are logged in to an npmjs.org account that has permission to publish to the @textile org/scope.
Run lerna publish, and sit back and wait. Ideally, this will go off without a hitch.

If it fails after creating tags and/or pushing the version updates to git, but before the actual npm publish completes, you should be able to 'recover' by running lerna publish from-package, which will attempt to publish new releases based on the versions available on npmjs.org. If it failed before or during tag creation, you might have to start over by dropping the version updates (git checkout -- .) and trying again. If lerna managed to commit the changes, you can use something like git reset --hard HEAD~1 to reset the last commit, followed by git push --force --follow-tags (if it was pushed to remote already and you are allowed to do pushes to master), but check that the right commit was reverted first.

See the lerna docs for other things you can do to make working across multiple packages easier. Note: If you are using lerna commands directly (as opposed to the ones wrapped in the package.json scripts), you'll want to install lerna globally to make your life easier: npm i -g lerna.

API

See https://textileio.github.io/js-threads, which includes the technical API docs for all subpackages.

Maintainers

Carson Farmer

Contributing

PRs gratefully accepted! Please see the contributing guide for details on getting started.

Small note: If editing the README, please conform to the standard-readme specification.

sunminghong/js-threads