nxadm/rakudo-pkg

A stable URL for continuous integration

Closed this issue · 8 comments

briandfoy commented

I'd like to have a stable URL that I can fetch from Travis CI or AppVeyor (both Ubuntu 14.04). I don't care about the compiler version as long as it is the latest. Something like rakudo-pkg-travis-ci.deb and rakudo-pkg-appveyor.deb would be nice.

I ran out of time tonight but I figured that could happen in pkg_rakudo.pl if it's the right combination of OS. Or it could be a post step that finds the right package and copies it to the right filename (probably better).

nxadm commented

Thank you @briandfoy,

I'll look into it (and the PR). Keep you posted,

C.

briandfoy commented

I was looking at the travis config and wondering how it could all happen there and be deployed automatically. Maybe just copy the right file to the target name. I won't have time to work that out this weekend but I'll experiment in my own fork. I bet there's something really simple that won't take a lot of work but may require clever thought. :)

zoffixznet commented

A little outside-the-box thinking: @nxadm could host the binaries directly on the new rakudo.org (content of which in a few months will relocate to perl6.org).

The site would offer stable URLs to the latest packages (like how http://hack.p6c.org:4242/latest/star/win64 currently gives latest win64 version of star) and it'd be less disorienting for users to not have to jump from rakudo.org/perl6.org page to a repo on a different site.

If we want to go this route, I can handle the front-end stuff and nxadm should ask moritz for uploading permissions to rakudo@www.p6c.org so binaries could be uploaded (I can setup the keys myself, just need moritz to OK the addition of the new user to the system).

The new rakudo.org site will go live between Apr. 4 and and Apr. 14

nxadm commented

It's something I have been thinking about but haven't finished because of lack of time.

So in short, what we have today are automatically created releases based on a git tag. When travis sees a tag starting with "v", it builds the packages and uploads them to github. I got some help with javascript to create a shortcut to the latest package from the README. This is meant for humans, however, and poor for automated precesses. Firstly for security reasons but also for the lack of time, package creation and release should be in the open and end-to-end automated. The only thing I do manually is bumping the version on the travis file and upgrade a dev release to a regular release (next to figuring how when a Rakudo release fails tests or adding new target OSes).

About @briandfoy's suggestion, I see two options within github:

  • copy the Ubuntu14.04 package to something like rakudo-pkg-travis.deb (uploaded one to the latest release). The problem is that the path in the URL need still be be rewritten. This is not difficult to do (see the JS code in /docs how it's done), but the code should be replicated on remote .travis.yml files. It does not scale.
  • create a 2nd repo with 1 moving tag "latest". The travis file will have the link to the latest deb. The package would be downloaded, renamed and released. I don't know how github handles the combination of moving tags and releases. This needs to be tested.

@zoffixznet's solution is probably the one that makes the most sense, although even half a solution can also work.

  • If moritz is OK, we just just need to figure how to safely upload the files to the rakudo server. Travis supports deployments to Multiple Providers and ssh-key storage, so it should be possible. I spent quite some time already creating the rakudo-pkg travis file, so it shouldn't be to much work any more.
  • Even without remote uploads, a frontend web rewriting could do the same job but point to the latest files (see the JS example). Hosting the files on Rakudo in addition to Github is a good PR move in my opinion.

What I was thinking of but have postponed:

  • create deb/rpm repos. I have an account at Jfrog for this purpose. I need to figure out how the metadata creation works, but I have been lacking time recently. However, for travis this may be overkill (a wget is way easier). On the other hand, a repo can keep historical releases so modules can test against several Rakudo releases. It's low on my TODO, though.

Thank you both for thinking along on this issue.

C.

nxadm commented

@briandfoy Just a small update. I was able to create a repo for the Debian and Ubuntu packages (and applied this by creating a new one for the CI of perl6/doc (it includes the needed modules). ATM I am trying to automate it through Travis. The included bintray deploy integration seems to not be able to work with multiple distros and releases at the same time, but the REST API seems to work fine. You'll get the latest package by doing this:

env:
  global:
    - PATH="/opt/rakudo-pkg/bin:/opt/rakudo-pkg/share/perl6/site/bin:$PATH"
sudo: required
install:
  - sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 379CE192D401AB61
  - echo "deb https://dl.bintray.com/nxadm/rakudo-pkg-debs trusty main" | sudo tee -a /etc/apt/sources.list
  - sudo apt-get update && sudo apt-get install rakudo-pkg

apt-get being apt-get, you can also specify specific version (if in the repo).

C.

nxadm commented

@briandfoy
I will close this issue once 2018.04.1 is out and I release new packages. The new deb repo will be live then:
https://github.com/nxadm/rakudo-pkg/tree/pre2018.04#os-repositories

If this is not what you had in mind, ping me.

nxadm commented

Also, check this issue for rakudo travis support.

nxadm commented

Closing the ticket because the creation of a repo provides the latest package. Feel free to reopen.

C.