YunoHost-Apps/archivist_ynh

Rsync SSH Backup doesn’t work for some reason.

Opened this issue · 11 comments

Mrs-Feathers commented

Not sure why it doesn't transfer backup to SSH server via rsync.. here's my config from /opt/yunohost/archivist/Backup_list.conf

## OPTIONS

# Backup directory.
backup_dir=/home/yunohost.app/archivist/backup

# Encrypted backup directory.
# Usually just next to the backup directory
enc_backup_dir=/home/yunohost.app/archivist/encrypted_backup

# Encryption
# true/false
encrypt=true
# Password for encryption
# This file should be set at chmod 400
cryptpass=/opt/yunohost/archivist/passkey

# --------------------------------------------------------------------------------------
## YUNOHOST BACKUPS

# YunoHost backups
# Specific backup made by 'yunohost backup' command
# ynh_core_backup=true/false: Make a backup of the core of YunoHost, without any apps.
ynh_core_backup=true
ynh_core_pre_backup=
ynh_core_post_backup=

# ynh_app_backup=APP: Make a backup of an app.
# You can add as many lines as you want to save multiple apps.
# List all apps with sudo yunohost app list --installed | grep "id: "
# ynh_app_backup=wordpress
# If you don't want an app with a lot of data to backup those big files, please add 'do_not_backup_data' after the app name.
# ynh_app_backup=transmission do_not_backup_data
ynh_app_pre_backup=
ynh_app_post_backup=
ynh_app_backup=archivist
ynh_app_backup=portainer
ynh_app_backup=

# --------------------------------------------------------------------------------------
## FILES BACKUPS

# Max size for each backup in Mb.
# Be carreful, it's only a soft limit, that means the script will try to limit each backup.
# Because, it will never made more than one backup for a single directory (Even if it's a real big directory without subdirectories). And the same if there's files next to subdirectories, only one backup will be made for all this files.
max_size=500

# Backup of files and directories
# Add as many lines as you want for each directory or file you want to be backuped.
# Exclude a directory, a file, or multiples files with a regex with the exclude_backup instruction.
# file_to_backup="/directory/to backup"
# file_to_backup=/file/to backup
# exclude_backup="/directory/to exclude"
# exclude_backup=/file/to exclude
# exclude_backup="/exclude/all/tarball/in_this_directory/.tar.gz$"
files_pre_backup=
files_post_backup=
file_to_backup=
exclude_backup=

# --------------------------------------------------------------------------------------
## BACKUPS RECIPIENTS

# All options following a recipient name will be enabled for this recipient only.
# > recipient name=Name of this recipient.
# type=Choose between one of the scripts in the "senders" directory.
# destination directory=Directory where to put the backup in the other side.
# encrypt=Override main option for encryption. (true/false) (Optionnal option)
# specific option for type=See the chosen script to know what's the options.
# exclude backup=Exclude a backup file or a directory from the backup. This option can be duplicated. (Optionnal option)
# include backup=Choose the only file or directory which be send. This option can be duplicated. (Optionnal option)

# > recipient name=local example
# type=local
# destination directory=/my/local/backup
# encrypt=false
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

 > recipient name=backup_server
 type=rsync_ssh
 destination directory=backup
# encrypt=false
 ssh_host=192.168.0.4
 ssh_user=backupuser
 ssh_port=22
# ssh_key=/home/user/.ssh/id_rsa
 ssh_pwd=iwefulhdskn!
# ssh_options=
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

# > recipient name=b2 example
# type=b2
# destination directory=backup
# encrypt=false
# b2_bucket=my-bucket
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

any suggestions or tips? after changing this i've also rebooted...

Mrs-Feathers commented

output with that config with rsync enabled:

root@yunohost:/opt/yunohost/archivist# /opt/yunohost/archivist/archivist.sh
Tue Dec 10 20:31:10 EST 2019
> Build list of files to backup
> Compress backups

> Backup YunoHost core and apps
>> Make a temporary backup for ynh_core_backup
>>> This backup is the same than the previous one
>> Make a temporary backup for archivist_backup
>>> This backup is the same than the previous one
>> Make a temporary backup for portainer_backup
>>> This backup is different than the previous one
>>>> Make a real backup for portainer_backup
16K /home/yunohost.app/archivist/backup/ynh_backup/portainer_backup.tar.gz
> Clean old backup files
> Encrypt backups
Tue Dec 10 20:31:16 EST 2019

as you can see it doesn't transfer anything across rsync.. and the other server shows nothing has been uploaded or anything.

maniackcrudelis commented

Archivist does not expect to read options with space at the beginning of the line.
It could be modified, but for now avoid any space before option names.

Mrs-Feathers commented

oh ok. sorry i didn't think of that. i'll try it.

Mrs-Feathers commented

looks like that worked! but sadly now i have some other issue...

-> Build the list of files for the recipient backup_server
> Copy backups files in backup.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(235) [sender=3.1.2]
Wed Dec 11 06:44:33 EST 2019
maniackcrudelis commented

What a cryptic error...
First if I were you I would try with a password without special character. Special characters are often messy...

Otherwise, have a look to the logs of the other server.

Mrs-Feathers commented

i both created the backup folder in the remote user's thing and change the password. so whatever it was is solved! it works perfect!!! one single last question before i go on to setting this up on my production server since now its working on my dev servers.

how do i restore this back up... from like.. a brand new installation of YunoHost... like, lets say my server was just completely destroyed but i have backups.. so i set up a new server with a fresh YunoHost on it. then.. what? do i drop them somewhere specific and run a script? or just put them in the root and unzip them?

maniackcrudelis commented

It depends whether you did encrypt the backup or not.
If it's encrypted, first you have to use the script https://github.com/maniackcrudelis/archivist/blob/master/decrypt_backups.sh to decrypt your backups.

Then, for YunoHost backups (both core and apps), you have to put the compressed backup files into /home/yunohost.backup/archives/.
I'm not sure you can restore a backup from wherever else than this directory, unless it has been fixed.

Then, either you go to your admin panel to restore the backup, but not sure you could without a post install or you restore from the YunoHost command line.

Mrs-Feathers commented

thank you so much for detailing that out! and then obviously any other "directories" i add to the config file to archive i'll just have to copy those back manually after the fact.

thanks! this is so much easier to use than Borg. have been trying to set that up for ages then stumbled on this. i really just need super simple yet all-encompassing backups. and seems here that as long as i make sure all the apps correctly back up, that's what i'm getting. its on the same network literally just on a raspberry pi with a few 1TB drives plugged in. so i'm not too concerned about encryption more than i'm conserned about being able to restore a fully-functioning server back to normal after a fresh install.

thanks again!

Mrs-Feathers commented

and i love how your scripts put themselves in cron. its perfectly exactly what i need. as you can see with a portainer installation, i've even cleaned house of random scripts or anything that isn't where an app should be so everything gets backed up perfectly.. and any random scripts or non-yunohost app servers i need can just be run in their own docker containers... and thus backed up in-full due to portainer being backed up!

maniackcrudelis commented

If you do have a backup on the same network, I recommend you to not use encryption. Encfs is efficient but not really convenient.
If you need, for whatever reason to have a look to it, it would be easier if it's not encrypted.

But, I would also recommend, depending of the importance of your data, to have a backup elsewhere than your home. I do personally use a VPS to store my backups because I have all my personal and important documents in it. Encrypted of course !

Archivist allow you to have multiple rsync recipients, encrypted or not, by playing with the encrypt option of the recipient. (The global option should stay to true to have the ability to encrypt for any recipient)

Never tried borg, but I always think it would be a pain to use. Looks like it is indeed.

Mrs-Feathers commented