Restauring from FULL curl: option --data-binary: out of memory

[arekstuka]

Hi Guys,

I have a small problem :) When i try to restore mailbox from my full backup (~3GB ) i receiving this error:

2014-02-23 21:45:55 DEBUG: user@yourdomain.com - Restauring from 23/02/2014 21:08:03 FULL
curl: option --data-binary: out of memory
curl: try 'curl --help' or 'curl --manual' for more information

enybody has sollution for this?

p.s. when i try to make restore from another user (mailbox 200mb ) its working without any problems

[arekstuka]

problem still exist.....any idea ?

[liandros]

curl cannot get sufficient memory for build the request.
Try save memory skipping the web service. Use zimbra user, in your zimbra server, and import the
backup as follow :
$ /opt/zimbra/bin/zmmailbox -z -m UserEmailAddress postRestURL "//?fmt=tgz&resolve=reset" backup.tgz
replace "UserEmailAddress", and "backup.tgz" with the correct values.

You may get the following error:
ERROR: zclient.IO_ERROR (Broken pipe) (cause: java.net.SocketException Broken pipe)

In this case, try the follow:
Agree http://www.zimbra.com/forums/administrators/47949-recovery-tgz-mail-file-fails.html
, you can try to import each message using
$ zmmailbox -z -m @<yourmailserver.com> addMessage /Inbox *.eml

Note1: zmmailbox can use a directory argument, but it not examines recursivly, and not filters eml files.
Note2: Each directory must exist on the destination mailbox, otherwise you get an error.
See below for a script for filter eml files, and create each directory
in destination mailbox.
untar your backup.tgz file, and use zimbra user for execute the scritp as follow:

 $ import_backup.sh  account@yourmailserver.com your_decompress_backup_dir

and patience .....

############ import_backup.sh ####################################
#!/bin/bash
set -e
email=$1
bkdir="$2"
zmmailbox_bin=/opt/zimbra/bin/zmmailbox

if [ ! -x $zmmailbox_bin ] ;then echo "$zmmailbox_bin does not exists" ; exit ; fi

mb_folders=/tmp/mbox_folders

cd "$bkdir" 

debug(){ echo "$(date ): $@" ; }
create_folder(){
  debug "Creating folder \"$1\" "
  $zmmailbox_bin -v -z -m $email createFolder "$1"
}

#Get actual folders
while read Id View Unread Msg_Count Path ;do
  [ -z "$Path" ] || echo $Path
done<<EOF |grep '^/.' > $mb_folders
   $($zmmailbox_bin -v -z -m  $email getAllFolders)
EOF

#Import messages
while read dir;do
  #Folder exists?
  egrep -q "^${dir#.}$" $mb_folders || create_folder "${dir#.}"

  #Import messages in dir
  find "$dir" -mindepth 1 -maxdepth 1 -type f -regex '.*\.eml' -print0 |\
     xargs -L 1 -0 -r $zmmailbox_bin -v -z -m $email addMessage "${dir#.}"
done<<EOF
   $(find . -mindepth 1 -type d )
EOF

[dusty73]

Hi,
just tried the script of the last comment but there's something wrong with it near line 23-24, could you re-post it?

[liandros]

Sorry, I modified the previous post . This is a display problem because GitHub use Markdown parse for text styles, and ate some characters. Try now please.

[dusty73]

Hi,
i tried your script but is extremely slow, I checked it and found the problem (and a solution).
The message import is done with xargs, but zmmailbox is called for every message:

#Import messages in dir
find "$dir" -mindepth 1 -maxdepth 1 -type f -regex '.*\.eml' -print0 |\
  xargs -L 1 -0 -r $zmmailbox_bin -v -z -m $email addMessage "${dir#.}"

I swapped these previous line with:

$zmmailbox_bin -v -z -m $email addMessage "${dir#.}" "$dir"/*eml

In this way all the messages in a folder are imported in a single run of zmmailbox.

[jrios83]

Hi, I'm trying the scripts but I get this error for some messages:
Exception in thread "main" java.io.FileNotFoundException: ./Inbox!49/0000220914-CD SF LURIN_ ACTA DE REUNI??N 39.eml (No such file or directory)
at java.base/java.io.FileInputStream.open0(Native Method)
at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
at java.base/java.io.FileInputStream.(FileInputStream.java:157)
at com.zimbra.cs.zclient.ZMailboxUtil.addMessage(ZMailboxUtil.java:1936)
at com.zimbra.cs.zclient.ZMailboxUtil.doAddMessage(ZMailboxUtil.java:1979)
at com.zimbra.cs.zclient.ZMailboxUtil.execute(ZMailboxUtil.java:1085)
at com.zimbra.cs.zclient.ZMailboxUtil.main(ZMailboxUtil.java:2917)
I think it is the word accent, is there any way to deal with word accent or special characters in an email in order to add the message?

[Propovednik87]

The issue of lack of memory was resolved for me by adding physical memory to the server. The server consumed 7 GB of RAM, the archive took up 17 GB. 7+17=24 GB of RAM. I allocated 32 GB of RAM. The recovery went without problems.