Travis stopped creating Hackage Update jobs

[hesiod]

I'm not quite sure whether this is the right place to report this or if this even is an issue, but it appears that the Travis update jobs for the Hackage mirror, i.e. all-cabal-files, all-cabal-hashes, all-cabal-metadata and all-cabal-packages, have not been triggered since about 20 hours ago. IIUC, they should automatically be triggered every 30 minutes. A link says more than a thousand words, so there you go - note that the aformentioned mirrors have not received updates for some time (relatively speaking).

[snoyberg]

Thanks, it looks like the Travis restart command is failing right now. Looking into it.

[snoyberg]

Looks like we've been affected by this bug: travis-ci/travis.rb#315

[snoyberg]

For the record, this is the shell script which is failing to run if someone else can try it out from their machine with their Github token. It may generate different results, which would be very interesting:

#!/bin/bash

set -eux

travis login --skip-completion-check --skip-version-check --org --github-token $GITHUB_TOKEN

# Trigger the package mirroring first, since it's used by all-cabal-hashes
BUILD=$(travis branches --skip-version-check -r commercialhaskell/all-cabal-packages | grep "^master" | awk "{ print \$2 }")
BUILDNUM=${BUILD###}
echo BUILD=$BUILD
echo BUILDNUM=$BUILDNUM
travis restart --skip-version-check -r commercialhaskell/all-cabal-packages $BUILDNUM

BUILD=$(travis branches --skip-version-check -r commercialhaskell/all-cabal-files | grep "^hackage" | awk "{ print \$2 }")
BUILDNUM=${BUILD###}
echo BUILD=$BUILD
echo BUILDNUM=$BUILDNUM
travis restart --skip-version-check -r commercialhaskell/all-cabal-files $BUILDNUM

# Put in a bit of a delay to allow the all-cabal-packages job to finish. If
# not, no big deal, next job will pick up the change.
sleep 30

BUILD=$(travis branches --skip-version-check -r commercialhaskell/all-cabal-hashes | grep "^hackage" | awk "{ print \$2 }")
BUILDNUM=${BUILD###}
echo BUILD=$BUILD
echo BUILDNUM=$BUILDNUM
travis restart --skip-version-check -r commercialhaskell/all-cabal-hashes $BUILDNUM

BUILD=$(travis branches --skip-version-check -r commercialhaskell/all-cabal-metadata | grep "^hackage" | awk "{ print \$2 }")
BUILDNUM=${BUILD###}
echo BUILD=$BUILD
echo BUILDNUM=$BUILDNUM
travis restart --skip-version-check -r commercialhaskell/all-cabal-metadata $BUILDNUM

[snoyberg]

I've worked around this with a new Cron server that performs the work Travis was doing, as well as adding a daily Travis cron job (via a new Travis feature) that will run daily. For posterity, here's the script that runs on the cron server (creds redacted, and depends on the appropriate GPG and SSH keys being present):

#!/usr/bin/env bash

set -eux

export AWS_ACCESS_KEY=...
export AWS_SECRET_KEY=...

cd ~

for REPO in all-cabal-packages all-cabal-files all-cabal-hashes all-cabal-metadata
do
    if [ ! -d ~/$REPO ]
    then
        git clone git@github.com:commercialhaskell/$REPO
    fi
done

cd ~/all-cabal-packages
git pull
./update.sh

cd ~/all-cabal-files
git fetch
git checkout origin/hackage
git reset --hard
git branch -D hackage || true
git checkout -b hackage
git branch --set-upstream-to=origin/hackage hackage
./update.sh

cd ~/all-cabal-hashes
git fetch
git checkout origin/hackage
git reset --hard
git branch -D hackage || true
git checkout -b hackage
git branch --set-upstream-to=origin/hackage hackage
./update.sh

cd ~/all-cabal-metadata
git fetch
git checkout origin/master
git reset --hard
git branch -D master || true
git checkout -b master
git branch --set-upstream-to=origin/master master
./update.sh

Hopefully this probably won't crop up again. If you notice it, please let me know.

[snoyberg]

Closing, we now have a separate server for doing this.