/ginkgo

Golang version gingko p2p

Primary LanguageGoApache License 2.0Apache-2.0

gink-go

Ginkgo is a p2sp file transfer tool designed for ML dataset.

Build

go build cmd/ginkgo.go

Usage

On server side, just start server with the following command:

ginkgo

On client side, just run ginkgo as a scp command, both directory and file are supported.

ginkgo SrcHost:/path/to/src /path/to/dest
  • The /path/to/src can be either absolute or relative.
  • The src and dest path handling behavior is just compatible with GNU scp command

How

  • Client will get other clients list from server side and broadcast itself to them.
  • Client chooses the block to download according to consistent hashing result of its serving host:port.
  • Other clients also choose which peer to download the block from by the hash.
  • The seed.Seed structure records the blocks and files metadata.

Golang version of gingko

ginkgo is an alternative form of gingko. Gink-go is the golang version Gingko.

Todo

  • HTTP range downloader
  • HTTP file server
  • Directory support
  • Consistent hashing locator
  • Client rate limit
  • Server rate limit