/roam-to-git

Automatic RoamResearch backup to Git

Primary LanguagePythonMIT LicenseMIT

Automatic RoamResearch backup

Roam Research backup roam-to-git tests.py

This script helps you backup your RoamResearch graphs!

This script automatically

  • Downloads a markdown archive of your RoamResearch workspace
  • Downloads a json archive of your RoamResearch workspace
  • Unzips them to your git directory
  • Commits and pushes the difference to Github

Demo

See it in action!. This repo is updated using roam-to-git.

Why to use it

  • You have a backup if RoamResearch loses some of your data.
  • You have a history of your notes.
  • You can browse your Github repository easily with a mobile device

Use it with Github Actions (recommended)

Note: Erik Newhard's guide shows an easy way of setting up Github Actions without using the CLI.

Create a (private) Github repository for all your notes

With gh: gh repo create notes (yes, it's private)

Or manually

Configure Github secrets

  • Go to github.com/your/repository/settings/secrets

Add 3 (separate) secrets where the names are

ROAMRESEARCH_USER

ROAMRESEARCH_PASSWORD

ROAMRESEARCH_DATABASE

  • Refer to env.template for more information

  • when inserting the information, there is no need for quotations or assignments

image

Add GitHub action

cd notes
mkdir -p .github/workflows/
curl https://raw.githubusercontent.com/MatthieuBizien/roam-to-git-demo/master/.github/workflows/main.yml > \
    .github/workflows/main.yml
git add .github/workflows/main.yml
git commit -m "Add github/workflows/main.yml"
git push --set-upstream origin master

Check that the Github Action works

  • Go to github.com/your/repository/actions
  • Your CI job should start in a few seconds

Note:

If the backup does not automatically start, try pushing to the repository again

Use it locally

Note: if your file system is not case-sensitive, you will not backup notes that have the same name in different cases

Install Roam-To-Git

With pipx (if you don't know pipx, you should look at it, it's wonderful!)

pipx install git+https://github.com/MatthieuBizien/roam-to-git.git

Create a (private) Github repository for all your notes

With gh: gh repo create notes (yes, it's private)

Or manually

Then run git push --set-upstream origin master

Configure environment variables

  • curl https://raw.githubusercontent.com/MatthieuBizien/roam-to-git/master/env.template > notes/.env
  • Fill the .env file: vi .env
  • Ignore it: echo .env > notes/.gitignore; cd notes; git add .gitignore; git commit -m "Initial commit"

Manual backup

  • Run the script: roam-to-git notes/
  • Check your Github repository, it should be filled with your notes :)

Automatic backup

One-liner to run it with a cron every hours: echo "0 * * * * '$(which roam-to-git)' '$(pwd)/notes'" | crontab -

NB: there are issues on Mac with a crontab.

Debug

Making roam-to-git foolproof is hard, as it depends on Roam, on Github Action or the local environment, on software not very stable (pyppeteer we still love you 😉 ) and on the correct user configuration.

For debugging, please try the following:

  • Check that the environment variables ROAMRESEARCH_USER, ROAMRESEARCH_PASSWORD, ROAMRESEARCH_DATABASE are correctly setup
  • Login into Roam using the username and the password. You may want to ask a new password if you have enabled Google Login, as it solved some user problems.
  • Run roam-to-git --debug to check the authentification and download work
  • Look at the traceback
  • Look for similar issues
  • If nothing else work, create a new issue with as many details as possible. I will try my best to understand and help you, no SLA promised 😇

Task list

Backup all RoamResearch data

  • Download automatically from RoamResearch
  • Create Cron
  • Write detailed README
  • Publish the repository on Github
  • Download images (they currently visible in Github, but not in the archive so not saved in the repository 😕)

Format the backup to have a good UI

Link formatting to be compatible with Github markdown

  • Format [[links]]
  • Format #links
  • Format attribute::
  • Format [[ [[link 1]] [[link 2]] ]]
  • Format ((link))

Backlink formatting

  • Add backlinks reference to the notes files
  • Integrate the context into the backlink
  • Manage / in file names

Other formatting

  • Format {{TODO}} to be compatible with Github markdown
  • Format `{{query}}``

Make it for others

Some ideas, I don't need it, but PR welcome 😀

  • Test it/make it work on Windows
  • Pre-configure a CI server so it can run every hour without a computer Thanks @Stvad for #4!