quentin/Duplicity-progress

Duplicity-progress doesn't update

Opened this issue · 3 comments

When trying to call duplicity-progress from inside a shell script with a restore command, like so:

/opt/Duplicity-progress/bin/duplicity-progress restore cf+http://$cf_container $dump_restore_dir

the output seems to be stuck at the following:

updates:0/0 transfered:0%

Sniffing the HTTPS traffic confirms that duplicity is running, but for some reason there isn't any corresponding console update.

Hi, I've never tried a restore :)

duplicity-progress first executes a dry-run of duplicity with full debug output in order to know how many operations should be expected during the actual run.
Currently duplicity-progress is only taking into account a limited range of the full debug output generated by duplicity.
It is highly probable that the "restore" command emit output that is currently being discarded.

Another possible explanation is that the dry-run is taking a lot of time and communications. Does the command complete properly?

Could you post the output (or at least a representative part) of the following command:

  duplicity -vdebug --dry-run restore cf+http://$cf_container $dump_restore_dir

Thanks!

Sure, here is a dump of the output for the command, with some of the repetitive parts snipped:

Using archive dir: /var/lib/postgresql/.cache/duplicity/d9bd0b2a11c5330cf44766f721590934
Using backup name: d9bd0b2a11c5330cf44766f721590934
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.u1backend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.giobackend Failed: No module named gio
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.sshbackend Failed: No module named paramiko
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Main action: restore
================================================================================
duplicity 0.6.18 (February 29, 2012)
Args: /usr/bin/duplicity -vdebug --dry-run restore cf+http://$cf_container $dump_restore_dir
Linux srv1 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64
/usr/bin/python 2.7.3 (default, Aug  1 2012, 05:14:39) 
[GCC 4.6.3]
================================================================================
Using temporary directory /tmp/duplicity-iEtvqJ-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-iEtvqJ-tempdir/mkstemp-e7Z_q7-1
Temp has 68337672192 available, backup will use approx 34078720.
Listing '$cf_container'
Sync would copy the following from remote to local:
duplicity-full-signatures.20130517T000459Z.sigtar.gpg
duplicity-full.20130517T000459Z.manifest.gpg
Sync would remove the following spurious local files:
duplicity-full.20130512T000442Z.manifest
Listing '$cf_container'
600 files exist on backend
314 files exist in cache
Extracting backup chains from list of files: ['duplicity-full-signatures.20120716T232629Z.sigtar.gpg', ..., 'duplicity-new-signatures.20130104T173355Z.to.20130106T000013Z.sigtar.gpg']
File duplicity-full-signatures.20120716T232629Z.sigtar.gpg is not part of a known set; creating new set
Ignoring file (rejected by backup set) 'duplicity-full-signatures.20120716T232629Z.sigtar.gpg'
...
File duplicity-full-signatures.20130517T000459Z.sigtar.gpg is not part of a known set; creating new set
Ignoring file (rejected by backup set) 'duplicity-full-signatures.20130517T000459Z.sigtar.gpg'
File duplicity-full.20130513T000544Z.manifest.gpg is not part of a known set; creating new set
File duplicity-full.20130513T000544Z.vol1.difftar.gpg is part of known set
...
File duplicity-full.20130517T000459Z.vol9.difftar.gpg is part of known set
File duplicity-new-signatures.20130104T173355Z.to.20130106T000013Z.sigtar.gpg is not part of a known set; creating new set
Ignoring file (rejected by backup set) 'duplicity-new-signatures.20130104T173355Z.to.20130106T000013Z.sigtar.gpg'
Found backup chain [Mon May 13 00:05:44 2013]-[Mon May 13 00:05:44 2013]
Found backup chain [Wed May 15 00:04:55 2013]-[Wed May 15 00:04:55 2013]
Found backup chain [Thu May 16 00:05:09 2013]-[Thu May 16 00:05:09 2013]
Found backup chain [Fri May 17 00:04:59 2013]-[Fri May 17 00:04:59 2013]
Last full backup date: Fri May 17 00:04:59 2013
Collection Status
-----------------
Connecting with backend: CloudFilesBackend
Archive dir: /var/lib/postgresql/.cache/duplicity/d9bd0b2a11c5330cf44766f721590934

Found 3 secondary backup chains.
Secondary chain 1 of 3:
-------------------------
Chain start time: Mon May 13 00:05:44 2013
Chain end time: Mon May 13 00:05:44 2013
Number of contained backup sets: 1
Total number of contained volumes: 80
 Type of backup set:                            Time:      Num volumes:
                Full         Mon May 13 00:05:44 2013                80
-------------------------

Secondary chain 2 of 3:
-------------------------
Chain start time: Wed May 15 00:04:55 2013
Chain end time: Wed May 15 00:04:55 2013
Number of contained backup sets: 1
Total number of contained volumes: 68
 Type of backup set:                            Time:      Num volumes:
                Full         Wed May 15 00:04:55 2013                68
-------------------------

Secondary chain 3 of 3:
-------------------------
Chain start time: Thu May 16 00:05:09 2013
Chain end time: Thu May 16 00:05:09 2013
Number of contained backup sets: 1
Total number of contained volumes: 68
 Type of backup set:                            Time:      Num volumes:
                Full         Thu May 16 00:05:09 2013                68
-------------------------


Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Fri May 17 00:04:59 2013
Chain end time: Fri May 17 00:04:59 2013
Number of contained backup sets: 1
Total number of contained volumes: 69
 Type of backup set:                            Time:      Num volumes:
                Full         Fri May 17 00:04:59 2013                69
-------------------------
No orphaned or incomplete backup sets found.
Removing still remembered temporary file /tmp/duplicity-iEtvqJ-tempdir/mkstemp-e7Z_q7-1

The dry-run in this instance took real 0m2.647s

I red the documentation again, we should get the debug output printed in a file with the following option:

--log-file filename

Could you re-run the previous command with that option and post the output ?
Thanks.