/DataTransferELab

Large-scale data transfer solution of ELab

Primary LanguagePython

Data Tranfer for ELab

Features ☕

  • Transfer
  • Scan
  • Log

Usage:

1. Directory structure:



| ELab5T
    | Folder 1
        | File 1
        | File 2
        ......
        | File N
    | Folder 2
    | Folder 3
    ......
    | Folder N

2. Parameters:

--num_workers: The number of processors
--shard: The specified shard
--total: The total number of shards
--num_chunks: The number of chunks in one shard