Cannot upload backup to google cloud storage

Question

Cannot upload backup to google cloud storage

umsalotagi opened this issue 8 years ago · 2 comments

........................................................config

  upload:
  #  method: none
     method: gs
  #  remove_uploaded: [true|false] (default: false)
  #  retries: [1+]           (default: 5)
  #  threads: [1+]           (default: 4)
     gs:
       project_id: linen-radio-180413
       access_key: XXXXXXXXXXXXXXXXX
       secret_key: fgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6Q+
       bucket_name: xxxxxxx-database-bckup
       bucket_prefix:
  #  rsync:

also tried with config default bucket_prefix:

I got following error

[2017-11-11 15:36:18,957] [INFO] [MainProcess] [Stage:init:31] Notify stage disabled, skipping
[2017-11-11 15:36:21,238] [INFO] [MainProcess] [State:init:145] Initializing root state directory /var/lib/mongodb-consistent-backup/default
[2017-11-11 15:36:21,239] [INFO] [MainProcess] [State:load_backups:162] Found 2 existing completed backups for set
[2017-11-11 15:36:21,240] [INFO] [MainProcess] [State:init:131] Initializing backup state directory: /var/lib/mongodb-consistent-backup/default/20171111_1536
[2017-11-11 15:36:21,241] [ERROR] [MainProcess] [Main:exception:209] Problem starting uploader! Error: 'bucket'
Traceback (most recent call last):
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Main.py", line 251, in run
    self.backup_directory
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Upload.py", line 11, in __init__
    self.init()
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Pipeline/Stage.py", line 45, in init
    **self.args
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Gs/Gs.py", line 34, in __init__
    self.bucket          = self.config.upload.gs.bucket
  File "/var/lib/mongodb-consistent-backup/.pex/install/yconf-0.3.4-py2-none-any.whl.4c32acf64a3f2408d71fe4642e2daf2f77807381/yconf-0.3.4-py2-none-any.whl/yconf/util.py", line 52, in __getattr__
    raise AttributeError(e)
AttributeError: 'bucket'
[2017-11-11 15:36:21,242] [INFO] [MainProcess] [Main:cleanup_and_exit:163] Starting cleanup procedure! Stopping running threads
[2017-11-11 15:36:21,244] [INFO] [MainProcess] [Main:cleanup_and_exit:193] Cleanup complete, exiting
[2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:rotate:81] Running rotation of log files
[2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:compress:60] Compressing log file: /var/log/mongodb-consistent-backup/backup.default.20171111_1529.log

It says AttributeError: 'bucket' , there is not such attribute named bucket

Answer 1 · 2017-11-11T15:48:11.000Z

This looks like a versioning issue , do you have a GS Python module installed on you systems python?

…

Sent from my iPhone

On Nov 11, 2017, at 15:43, umsalotagi ***@***.***> wrote: ........................................................config upload: # method: none method: gs # remove_uploaded: [true|false] (default: false) # retries: [1+] (default: 5) # threads: [1+] (default: 4) gs: project_id: linen-radio-180413 access_key: XXXXXXXXXXXXXXXXX secret_key: fgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6Q+ bucket_name: xxxxxxx-database-bckup bucket_prefix: # rsync: also tried with config default bucket_prefix: I got following error [2017-11-11 15:36:18,957] [INFO] [MainProcess] [Stage:init:31] Notify stage disabled, skipping [2017-11-11 15:36:21,238] [INFO] [MainProcess] [State:init:145] Initializing root state directory /var/lib/mongodb-consistent-backup/default [2017-11-11 15:36:21,239] [INFO] [MainProcess] [State:load_backups:162] Found 2 existing completed backups for set [2017-11-11 15:36:21,240] [INFO] [MainProcess] [State:init:131] Initializing backup state directory: /var/lib/mongodb-consistent-backup/default/20171111_1536 [2017-11-11 15:36:21,241] [ERROR] [MainProcess] [Main:exception:209] Problem starting uploader! Error: 'bucket' Traceback (most recent call last): File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Main.py", line 251, in run self.backup_directory File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Upload.py", line 11, in __init__ self.init() File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Pipeline/Stage.py", line 45, in init **self.args File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Gs/Gs.py", line 34, in __init__ self.bucket = self.config.upload.gs.bucket File "/var/lib/mongodb-consistent-backup/.pex/install/yconf-0.3.4-py2-none-any.whl.4c32acf64a3f2408d71fe4642e2daf2f77807381/yconf-0.3.4-py2-none-any.whl/yconf/util.py", line 52, in __getattr__ raise AttributeError(e) AttributeError: 'bucket' [2017-11-11 15:36:21,242] [INFO] [MainProcess] [Main:cleanup_and_exit:163] Starting cleanup procedure! Stopping running threads [2017-11-11 15:36:21,244] [INFO] [MainProcess] [Main:cleanup_and_exit:193] Cleanup complete, exiting [2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:rotate:81] Running rotation of log files [2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:compress:60] Compressing log file: /var/log/mongodb-consistent-backup/backup.default.20171111_1529.log It says AttributeError: 'bucket' , there is not such attribute named bucket — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

Answer 2 · 2017-11-11T16:21:03.000Z

I have run this script through docker by following command

docker run -it --net=host -v "/data/mongobackup/conf:/conf:Z" -v "/data/mongobackup/data:/var/lib/mongodb-consistent-backup:Z" -v "/data/mongobackup/logs:/var/log/mongodb-consistent-backup:Z" timvaillancourt/mongodb_consistent_backup:latest --config=/conf/mongodb-consistent-backup.conf