/supload

Bash script for upload file to cloud storage based on OpenStack Swift API.

Primary LanguageShellGNU General Public License v3.0GPL-3.0

What is supload?

It's simple script for upload file to cloud storage based on OpenStack Swift API.

Feature

  • recursive upload
  • check files by MD5 hash
  • upload only modified files
  • expiring files
  • exclude files matches pattern
  • find and upload only newest files
  • limit upload speed
  • auto unpacks archive on uploading (if storage supports)

Restrictions

  • support only less than 5G file to upload

Installation

For use script need installed utilities curl and file. Install for debian/ubuntu:

$ apt-get install curl file

Get script:

$ wget https://raw.github.com/selectel/supload/master/supload.sh && chmod +x supload.sh
# or
$ curl -Ok https://raw.github.com/selectel/supload/master/supload.sh && chmod +x supload.sh

You may move script to /usr/bin or /usr/local/bin if want:

$ mv supload.sh /usr/local/bin/supload

Usage

By default the script work with Selectel Cloud Storage.

Get help:

$ ./supload.sh

Upload one file, file will be uploaded if not existence in storage or local file is different from file in storage (md5 check):

$ ./supload.sh -u USERNAME -k USERKEY storage_dir local_file.path

Upload all files in directory recursive (option -r):

$ ./supload.sh -u USRNAME -k USERKEY -r storage_dir local_dir/

Force upload without checking existing files in storage by md5 (option -M):

$ ./supload.sh -u USERNAME -k USERKEY -r -M storage_dir local_dir/

If the Storage supports expiring object you can set time when uploaded file must be deleted (option -d). Uploaded file will be deleted after 7 days:

$ ./supload.sh -u USERNAME -k USERKEY -d 7d storage_dir file.doc

Exclude some files for upload:

$ ./supload.sh -u USRNAME -k USERKEY -r -e ".git/*" -e "file_in_root_dir" -e "dir1/*/*.cache" -e "*~" -e "*.bak" storage_dir local_dir/

Enable detect mime-type (by utility file) and set Content-Type for uploading file (option -c):

$ ./supload.sh -u USERNAME -k USERKEY -c storage_dir localfile

Upload only files modified in the last 7 days (option -m):

$ ./supload.sh -u USERNAME -k USERKEY -r -m -7 storage_dir local_dir/

Limit upload speed (option -s):

$ ./supload.sh -u USERNAME -k USERKEY -r -s 2M storage_dir local_dir/

Upload and extract archive (option -z):

$ ./supload.sh -u USERNAME -k USERKEY -z tar.bz2 storage_dir archive.tar.bz2

Changes

3.0:

  • unpack archive
  • improve handle error and debug output
  • add functional tests

2.7:

  • limit upload speed (-s)

2.6:

  • added new option for find&filter files by modified time p(-m)
  • fixed parser for etag header
  • added handle for 401 Unauthorized

2.5:

  • disabled detect mime-type and set content-type by default
  • add option for enable detect mime-type (-c)

2.4:

  • fixed: reauth when access denied

2.3:

  • hide password key in cmdline

2.2:

  • add option for exclude files (-e)

2.1:

  • add support for expiring files (-d)
  • util file not necessarily now

2.0:

  • ignore case for auth headers
  • support MacOsX
  • small fixes