imapbackup
contains two command line tools:
ib-mailbox
for exporting emails from IMAP mailboxesib-archive
contributes a few functions for the archive data structure
With ib-mailbox
you can archive emails from IMAP mailboxes. The export can
be repeated at regular intervals, no duplicates will be saved as long as you
always export from the same mailbox.
ib-mailbox
needs a configuration file for the backup job, you find the
description below.
First of all, you may want to get an overview of all folders in the mailbox to be able to exclude certain folders from the backup:
$ ib-mailbox list --job example.job
example.org::Trash
example.org::Archive
example.org::Archive/2022
example.org::Archive/2021
example.org::Archive/2020
example.org::INBOX
Now you are ready for the first backup run:
$ ib-mailbox backup --job example.job ./backup
2022-04-12 10:05:52,275 INFO -- START
2022-04-12 10:05:52,276 INFO -- Processing mailbox: example.org
2022-04-12 10:05:52,527 INFO -- example.org::INBOX: found 3 messages
2022-04-12 10:05:52,799 INFO -- example.org::INBOX[1]: NEW: id=25652e390168...a234
2022-04-12 10:05:52,799 INFO -- example.org::INBOX[2]: NEW: id=fa1f63a13f91...c9ee
2022-04-12 10:05:52,799 INFO -- example.org::INBOX[3]: NEW: id=800be881dc38...7fa8
If you run it again, the messages will be downloaded again, but no duplicates will be saved:
$ ib-mailbox --job example.job backup ./backup
2022-04-12 10:09:28,248 INFO -- START
2022-04-12 10:09:28,250 INFO -- Processing mailbox: mailbox.org
2022-04-12 10:09:28,531 INFO -- example.org::INBOX: found 9 messages
2022-04-12 10:09:28,820 INFO -- example.org::INBOX[1]: EXISTS: id=25652e390168...a234
2022-04-12 10:09:28,820 INFO -- example.org::INBOX[2]: EXISTS: id=fa1f63a13f91...c9ee
2022-04-12 10:09:28,820 INFO -- example.org::INBOX[3]: EXISTS: id=800be881dc38...7fa8
With ib-archive
you can move/copy existing eml
files into the archive
directory structure. Let's say you have some mails stored in ./my_mails
and
want to consolidate them into the archive under ./backup
:
$ ib-archive --verbose import ./my_mails ./backup
The import function also has a few options:
$ ib-archive import --help
usage: ib-archive import [-h] [--docuware] [--move] source destination
Import emails from source archive to destination
positional arguments:
source Directory from which you want to copy/move the mails
destination Archive directory
options:
-h, --help show this help message and exit
--docuware Source archive is a Docuware email archive
--move Move emails, i.e. remove emails from source after backup
This tool is work in progress and not yet usable. Its purpose will be to transfer mails between two IMAP mailboxes.
Mails are exported as RFC822 eml
files in a content-addressed directory
structure:
./archive
├── 00
│ ├── 00
│ │ └── 00003c6ec5464cca9...7af8.eml
│ ├── 0f
│ │ └── 000ffe5b49390d9b2...26eb.eml
│ ├── 11
│ │ └── 001124d77ce778289...4fd8.eml
│ ├── 30
│ │ └── 0030f33161416b03e...97aa.eml
The filename is the hash of the eml file itself and at the same time the key to the data structure. It is therefore very easy to determine that the files have not been tampered with by determining their hash value and comparing it with the filename. Finding an email by its sender/recipient or subject is not possible without the help of external tools.
Mails with the same message ID are considered identical by the user, but if
their RFC822 representation is different, they are stored multiple times
because the hash values differ. Especially MS Exchange is very creative in
generating different versions from the same mail, in particular the mails from
the journal differ from those in the mailbox by an additional Received
entry
and replaced MIME multiplart delimiters.
For ib-mailbox
you have to create a configuration file for the backup job.
A simple example for Google Mail:
gmail.com:
server: "imap.gmail.com"
username: "john.doe@gmail.com"
password: "123456"
folders:
- All Mail
This is a YAML file, so you need to pay attention to the correct indentation.
server
(Required): IP address or hostname of the IMAP server.
username
, password
(Required): Login credentials for the IMAP server. Only
login with username and password is currently supported.
port
: If not specified, 993 is used.
tls
: Set to false
if an encrypted connection is not to be established.
Default is true.
folders
: List of names of folder to be exported.
ignore_folder_flags
: With IMAP, flags can be assigned to folders (and
messages). Here you can define a list of flags for folders that you want to
ignore if one of these flags is set.
ignore_folder_names
: List of names of folders you want to ignore. The name
can also be a regular expression like .*/foldername
tls_check_hostname
, tls_verify_cert
: Default true, use these options to
disable the associated checks.
exchange_journal
: This option enables the handling of MS Exchange Journal
messages. In this case, the original mail is found as an attachment in the
journal mail. The handling of these journal mails is not very elaborate, but
in my experience it works reliably - as long as they really are journal mails.
If you don't know exactly if you need this option, then you don't need it.
Default false.
delete_after_export
: Default false. With this option enabled each exported
mail will be removed from the IMAP server. Use with caution.
with_db
: Creates a metadata SQLite database.
incremental
: Downloads only messages since the last backup job. Needs
with_db
option.
Example 1: Here you can see how to exclude folders from the backup.
example.org:
username: "john.doe@example.org"
password: "123456"
server: "imap.example.org"
port: 993
tls: true
ignore_folder_flags:
- Junk
- Drafts
- Trash
ignore_folder_names:
- .*/Calendar/?.*
folders:
- "INBOX"
- "Archive"
Example 2: Exporting MS Exchange journal mails.
exchange.example.org:
server: "exchange.example.org"
username: "john.doe@example.org"
password: "123456"
tls_check_hostname: false
exchange_journal: true
delete_after_export: true
folders:
- INBOX
If requested in the job file with the with_db
option, an SQLite database is
created inside the archive and some fields from the header like date, message
ID, sender, recipient and subject are written to this database.
For an archive without DB, a database can also be created afterwards with
ib-archive db-from-archive --mailbox IDENTIFIER FOLDER
. The mailbox
identifier should match the name of the mailbox in your job file.
A short tutorial on how to create an EXE file for Windows.
First, enter the Python environment by starting CMD.EXE
or PowerShell CLI in
the project folder.
C:\> pipenv install --dev
Now install auto-py-to-exe
:
C:\> pipenv run pip install auto-py-to-exe
C:\> auto-py-to-exe
Select the appropriate py
file from imapbackup/cli
in "Script Location"
field. Activate "One File" option and "Console Based". Now press the button
"CONVERT .PY TO .EXE". You will find the executable in folder ./output
.