Dat

Dat is a decentralized data tool for distributing data small and large.

Key features:

Live sync folders by sharing files as they are added or changed.
Distribute large files without copying data to a central server by connecting directly to peers.
Intelligently sync by deduplicating data between versions.
Verify data integrity using strong cryptographic hashes.
Work everywhere, including on the command line, in the browser, and on the desktop.

Documentation | Video Demo | Ecosystem

Dat Command Line Tool

This guide will help you get started with the Dat command line tool. We are also developing web and desktop applications for Dat.

Getting Started

The Dat command line tool can be used to share, download, and sync files across many computers via the command line.

Windows	Mac/Linux	Version

Installation

npm install -g dat

You should be able to run the dat command now. If not, see the installation troubleshooting for tips.

Demo

We have Dat installed, now let's use it! In this quick demo we will download our documentation files using Dat.

You tell Dat what files to download by giving it a Dat link. Dat links are 64 characters long, for example acd8aa85a22bc65e3933bcbd785639d6382bcd3b78faff1d22a482c5f9e5a39d.

Along with the link, you tell Dat where to download the files. All together, you can download files by typing dat <dat-link> <download-directory>.

We have our Dat documentation folders being shared by Dat (at the key above). For this example, we can download those files to your computer. In your console, run:

dat acd8aa85a22bc65e3933bcbd785639d6382bcd3b78faff1d22a482c5f9e5a39d dat_docs

You should see the output below. Once the download is finished, the files will be available on your computer in the dat_docs folder!

CLI Development Status

This is the Dat CLI 1.0 release candidate (RC2). We are actively seeking feedback & developing this release candidate. Follow this issue for the Dat CLI road map discussion and see known RC2 issues.

Please note that previous versions of Dat (alpha, beta) are incompatible with the 1.0 release candidate.

Using Dat

There are two commands in Dat:

Share data: dat <directory> will share a directory on your computer.
Download data: dat <dat-link> <download-directory> will download files from the Dat link to a directory on your computer.

Running dat in the console, with no arguments, will show you the usage guide. You can always use this as a reference for all the commands:

dat <directory>

  Share the given directory and create a link which then can be used to
  download the data. The process will stay open and watch for filesystem changes.

  --watchFiles          watch files for changes (beta: still in development)
  --snapshot            create a snapshot of directory

dat <link> <directory>

  Download a dat-link into directory. Continues to listen for changes and
  download them as they are added by the remote peer.

  --exit                exit process after download finishes

General options

  --version, -v         get installed dat version
  --temp                use in-memory database instead of .dat folder
  --webrtc              connect to webrtc peers via electron-webrtc
  --doctor              run dat doctor
  --quiet, -q           output only dat-link, no progress information
  --debug               show debugging output
  --ignore-hidden       ignore hidden files (true by default)
  --port, -p            set a specific inbound tcp port

Sharing Files

Share a directory by typing dat <directory>:

$ dat my_data/
Sharing /Users/joe/my_data/

Share Link: d6e1875598fae25165eff440ffd01513197ad0db9dbb9898f2a141288b9322c6
The Share Link is secret and only those you share it with will be able to get the files

[==============>] Added 2 files (1.44 kB/1.44 kB)

Connected to 1 peers. Uploading 288.2 B/s. Watching for updates...

You are now publishing that data from your computer. It will be publicly accessible as long as your terminal is open and the process is still running. The hash is a secret hash, your data is visible to anyone you send the hash to.

Updating Shared Files

Dat makes it easy to share a folder and send files as they are added to the folder. By default, when you share using dat my_data/ you will be in live sync mode. Anyone connected to you will receive new files.

Creating a snapshot

A snapshot reads the files and creates a unique link that will always be the same for that set of files (if they remain unchanged). To create a snapshot use the snapshot option: dat my_data/ --snapshot. Snapshots are automatically created for you in live mode as files update.

Sharing Options

dat <directory> --snapshot

Share a snapshot of the current files.

dat <directory> --port=1234

Set your inbound TCP port. This is useful for debugging or on restrictive networks.

Downloading Files

Download files from a Dat link by typing: dat <dat-link> <download-directory>:

$ dat d6e1875598fae25165eff440ffd01513197ad0db9dbb9898f2a141288b9322c6 download_dir
Downloading in /Users/joe/download_dir

Share Link: d6e1875598fae25165eff440ffd01513197ad0db9dbb9898f2a141288b9322c6
The Share Link is secret and only those you share it with will be able to get the files

[==============>] Downloaded 3 files (1.44 kB/1.44 kB)

Connected to 1 peers. Downloading 1.44 kB/s. Watching for updates...

Dat will start downloading the data into the download_dir folder. Once the download is finished (a message will print and the bar will turn green), you can safely exit the process with Ctrl-C (Cmd-C on Mac).

While downloading, you may be connected to more than 1 peer. Anyone who has the Dat link will be able to download and re-host a copy of the data. So you may be downloading from (and sharing to) other people that are also downloading that data! You only need one block of data to start helping as a host. It's distributed mad science!

Updating the Downloaded Files

What happens if the files get updated? Dat auto-syncs new files if it is still running. If you exited the process, you can run the same command you ran before (with the same link and directory) and the files will update!

Download Options

dat <dat-link> <directory> --exit

After files are done downloading, exit the process. If you are connected to a live Dat you will not get new files unless you run the command again.

dat <dat-link> <directory> --port=1234

Set your inbound TCP port. This is useful for debugging or on restrictive networks.

Live Sync & Snapshots

Dat makes it easy to share a folder and send files as they are changed or added. By default, when you share using Dat you will be in live sync mode. Anyone connected to you will receive file changes as you make them.

When downloading a Dat, you do not have to worry about live mode. It will automatically start in the right mode based on the link.

To create a snapshot when sharing files use the snapshot option: dat my_data/ --snapshot. A snapshot reads the files and creates a specific link that will never change (as long as the files don't change).

Dat Metadata Storage

When you run a command, Dat creates a hidden folder, .dat, in the directory specified. Similar to git, this folder stores information about your Dat. File metadata and the Dat link are stored to make it easy to continue sharing or downloading the same directory.

Temporary Database

Use the --temp option to keep the metadata in memory instead of the .dat folder.

Sharing or Download files with dat.land

You can use the --webrtc option to share files to dat.land. You'll need to install electron-webrtc in order for this to work.

Troubleshooting

We've provided some troubleshooting tips based on issues users have seen. Please open an issue or ask us in our chat room if you need help troubleshooting and it is not covered here.

If you have trouble sharing/downloading in a directory with a .dat folder, try deleting it and running the command again.

Check Your Dat Version

Knowing the version is really helpful if you run into any bugs, and will help us troubleshoot your issue.

Check your Dat version:

dat -v

You should see the Dat semantic version printed, e.g. 11.1.2.

Installation Issues

Node & npm

To use the Dat command line tool you will need to have node and npm installed. Make sure those are installed correctly before installing Dat. You can check the version of each:

node -v
npm -v

Global Install

The -g option installs Dat globally allowing you to run it as a command. Make sure you installed with that option.

If you receive an EACCES error, read this guide on fixing npm permissions.
If you receive an EACCES error, you may also install dat with sudo: sudo npm install -g dat.
Have other installation issues? Let us know, you can open an issue or ask us in our chat room.

Networking Issues

Networking capabilities vary widely with each computer, network, and configuration. Whenever you run a Dat there are several steps to share or download files with peers:

Discovering Peers
Connecting to Peers
Sending & Receiving Data

With successful use, Dat will show Connected to 1 peer after connection. If you never see a peer connected your network may be restricting discovery or connection. Please try using the dat --doctor command (see below) between the two computers not connecting. This will help troubleshoot the networks.

Dat may have issues connecting if you are using iptables.

Dat Doctor

We've included a tool to identify network issues with Dat, the Dat doctor. You will need to run the command on both the computers you are trying to share data between. On the first computer, run:

dat --doctor

The doctor will print out a command to run on the other computer, dat --doctor=<64-character-string>. The doctor will run through the key steps in the process of sharing data between computers to help identify the issue.

For Developers

Please see guidelines on contributing before submitting an issue or PR.

Installing from source

Clone this repository and in a terminal inside of the folder you cloned run this command:

npm link

This should add a dat command line command to your PATH. Now you can run the dat command to try it out.

The contribution guide also has more tips on our development workflow.

Internal API

Note: we are in the process of moving the js library to a separate module, joehand/dat-js.

dat.download(cb)

download dat.key to dat.dir

dat.share(cb)

share directory specified in opts.dir

Swarm is automatically joined for key when it is available for share & download (dat.joinSwarm()).

Events

Initialization

dat.on('ready'): db created/read & hyperdrive archive created.
dat.on('error'): init/database error

Swarm

Swarm events and stats are available from dat.swarm.

dat.on('connecting'): looking for peers
dat.on('swarm-update'): peer number changed

dat.on('key'): key is available (this is at archive-finalized for snapshots)
dat.on('append-ready'): file count available (dat.appendStats), about to start appending to hyperdrive
dat.on('file-added'): file added to archive
dat.on('upload', data): piece of data uploaded
dat.on('archive-finalized'): archive finalized, all files appended
dat.on('archive-updated'): live archive changed

Download

dat.on('key'): key is available
dat.on('file-downloaded', file): file downloaded
dat.on('download', data): piece of data downloaded
dat.on('upload', data): piece of data uploaded
dat.on('download-finished'): archive download finished

Other API

dat.key: key
dat.dir: directory
dat.datPath: path to .dat folder
dat.db: database instance
dat.swarm: hyperdrive-archive-swarm instance
dat.archive: hyperdrive archive
dat.snapshot (boolean): sharing snapshot archive

Internal Stats

dat.stats = {
    filesTotal: 0, // Latest archive size
    bytesTotal: 0,
    bytesUp: 0,
    bytesDown: 0,
    rateUp: speedometer(),
    rateDown: speedometer()
}

// Calculated on share before append starts. Used for append progress.
// Not updated for live.
dat.appendStats = {
    files: 0,
    bytes: 0,
    dirs: 0
}

License

BSD-3-Clause

cnhup/dat

Dat

Key features:

Documentation | Video Demo | Ecosystem

Dat Command Line Tool

Table of Contents

Getting Started

Installation

Demo

CLI Development Status

Using Dat

Sharing Files

Updating Shared Files

Creating a snapshot

Sharing Options

Downloading Files

Updating the Downloaded Files

Download Options

Live Sync & Snapshots

Dat Metadata Storage

Temporary Database

Sharing or Download files with dat.land

Troubleshooting

Check Your Dat Version

Installation Issues

Node & npm

Global Install

Networking Issues

Dat Doctor

For Developers

Installing from source

Internal API

dat.download(cb)

dat.share(cb)

Events

Initialization

Swarm

Share

Download

Other API

Internal Stats

License