Watches a directory and uploads new files to specified s3 bucket
To run the server, run:
python3 watch_for_changes.py config.ini
Warning: All the parameters are required. Sample below
[Configuration]
# Directory to watch
input: /root/lpTest/test
# Ignore Files with extenstions (semicolon separated)
extensions: *.log;*.py;*.ini;*.swp;*.swx
# tls usually set to True
tls : True
# S3 Endpoint
endpoint: s3-eu-west-1.amazonaws.com
# S3 Access Key
access_key: xxxxxxx
# S3 Security Key
security_key: xxxxxxxxx
# S3 Bucket name
bucket_name: xxxxxxx
# Results from the upload are logged to this file
logFile : ./results.log
<filename> <response code> <epoch timestamp>
- only files dumped to the input dir are uploaded (no directories inside folder yet!)
- no failure recovery and retries (upload fails, network fail etc.,) only logs
- no delete feature (only uploads new files)
- multiple files can’t be dumped at once (e.g 10000 files, susceptible to failures due to resource limit)
- there should be no errors in config file
- Delete and Modify files
- More threaded support with Thread pools