josegonzalez/python-github-backup

Backup fails when attempting to process public gists on Enterprise 2.20.9

0x6d617474 opened this issue · 2 comments

When attempting to create archives for users in our GitHub Enterprise instance, we noticed that a few users were failing. At first we thought that creating archives with gists was broken, but we noticed that some users with gists were working fine.

After some more investigation, it seems that any user with Public gists fails, while users with only Secret gists succeed.

Here is the commandline args we are using to invoke the tool:

github-backup -t "${GHE_TOKEN}" -H "${GHE_HOST}" --all --gists --prefer-ssh --private "${GHE_USER}" -o "${ARCHIVE_DIR}/${GHE_USER}" > /dev/null

Here's the console output when it fails:

  File "/github-backup/build/scripts-3.7/github-backup", line 42, in <module>
    main()
  File "/github-backup/build/scripts-3.7/github-backup", line 37, in main
    backup_repositories(args, output_directory, repositories)
  File "/github-backup/build/lib/github_backup/github_backup.py", line 780, in backup_repositories
    with codecs.open(output_file, 'w', encoding='utf-8') as f:
  File "/usr/lib/python3.7/codecs.py", line 898, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: '/archives/github_user/gists/8508ab26d5e47bdf4072/gist.json'

I'll probably look into this some more at some point, but wanted to file a report while it was still fresh in my mind.

This appears to be another case of #127

Further investigation:

  • The token we were using didn't have permissions to view secret gists
  • This is why only users with public gists were affected
  • HTTPS cloning appears to be unreliable for enterprise, since it frequently asks for credentials even though they have been supplied in the commandline already
  • When the username/password prompt happened, the clone would wait and then bail, and the script fails

It would be nice if gists would just clone via SSH...