Adding power(ppc64le) binaries to releases page
ghatwala opened this issue · 24 comments
Hi All,
I got an issue whilst building "fluentd" dockerfile on ppc64le ( power ) arch as seen here - https://travis-ci.org/ghatwala/fluentd-docker-image/jobs/377640502#L930
The power builds are failing for debian image based fluentd dockerfile only - https://github.com/ghatwala/fluentd-docker-image/blob/master/v0.12/debian/Dockerfile#L4. The power binaries are missing are in releases page here .
I was able to successfully build "dumb-init" from source locally on power machine using the same debian image. let us know if following is possible -
1. Can we help in providing pre-built ppc64le binaries and add this binary to list of available packages on releases page.
2. Do you'll want to build this power binary ( via artifacts ) from jenkins at OSU/OSL - http://osuosl.org/services/powerdev/request_hosting/ . I could help configuring a job here.
let me know your views/suggestions.
Neat! I didn't know that either.
If we can build with Travis, I think that'd be ideal. Definitely happy to add builds for other architectures to the "Releases" tab (assuming we have a way to build them for future releases).
@asottile, @chriskuehl - Yup , travis-ci supports power ( ppc64le) too now , currently we have done a soft beta launch. Hopefully there will be a public announcement about power support on travis-ci shortly.
And yeah we could use travis-ci to create build artifacts for power using below in *.travis.yml -
os:
- linux-ppc64le
And using these instructions here to upload artifacts to github - https://docs.travis-ci.com/user/deployment/releases/ and of course to debug the issues which u have mentioned above.
let me know if u need anything on configuring up a job at travis-ci/power.
We switched to Travis in #163, should be able to try this out now :)
@chriskuehl - sure , please go ahead !! , keep me posted. :)
@chriskuehl - did u try creating a travis-ci job for power , and try creating "dumb-init" binaries ( *.deb ) . I see that your last release was in Dec'17 , Any ETA for the power binary and lemme know if u have any updates for me... Thanks !!
Haven't had a chance to try it out yet -- it's on my list though!
@ghatwala also, if you know how to create the power binary you could also fork this repository, set up Travis CI yourself for building power binaries.
If you get something that works it would be awesome if you contributed your changes back.
@samrocketman , @chriskuehl , @asottile - have been able to create a power binary using this job - https://travis-ci.org/ghatwala/dumb-init/builds/386610930 and using these changes in *.travis.yml - https://github.com/ghatwala/dumb-init/blob/master/.travis.yml ( used only "itest_jessie" env as others were giving failures on power. The tagged release is here - https://github.com/ghatwala/dumb-init/releases/tag/20180601112051-6bf0d8b . let me know your comments about this binary - "dumb-init_1.2.1_ppc64el.deb"
Hi @ghatwala, unfortunately I do not have a ppc based machine so I don’t think I can help validate your binary. On behalf of other ppc users thanks for you help on this effort.
Thanks @samrocketman !!! - was thinking using travis-ci ( having power support now ) , can we now think of having power binaries ( *.deb) now available at the releases page of this "dumb-init" repository . This will help me resolve my original problem statement mentioned at the beginning of this ticket. Any comments on this to conclude on this ticket ?
Also just to confirm that this power binary indeed works ..
Architecture type :
root@packer-vm:~# arch
ppc64le
Download this binary
root@packer-vm:~# wget https://github.com/ghatwala/dumb-init/releases/download/20180601112051-6bf0d8b/dumb-init_1.2.1_ppc64el.deb
--2018-06-04 09:13:50-- https://github.com/ghatwala/dumb-init/releases/download/20180601112051-6bf0d8b/dumb-init_1.2.1_ppc64el.deb
Resolving github.com (github.com)... 192.30.255.112, 192.30.255.113
Connecting to github.com (github.com)|192.30.255.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/135242066/03159208-65bc-11e8-9900-7a51c9a30e47?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180604%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180604T091350Z&X-Amz-Expires=300&X-Amz-Signature=26dcb5b72ecf0938b4dba6119a8103d341ec957a93087b579f115fbf522eb2e2&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Ddumb-init_1.2.1_ppc64el.deb&response-content-type=application%2Foctet-stream [following]
--2018-06-04 09:13:50-- https://github-production-release-asset-2e65be.s3.amazonaws.com/135242066/03159208-65bc-11e8-9900-7a51c9a30e47?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180604%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180604T091350Z&X-Amz-Expires=300&X-Amz-Signature=26dcb5b72ecf0938b4dba6119a8103d341ec957a93087b579f115fbf522eb2e2&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Ddumb-init_1.2.1_ppc64el.deb&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 54.231.97.192
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|54.231.97.192|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 43146 (42K) [application/octet-stream]
Saving to: ‘dumb-init_1.2.1_ppc64el.deb’
dumb-init_1.2.1_ppc64el.deb 100%[===========================================================================================>] 42.13K 265KB/s in 0.2s
2018-06-04 09:13:51 (265 KB/s) - ‘dumb-init_1.2.1_ppc64el.deb’ saved [43146/43146]
Installing this binary on power
root@packer-vm:~# dpkg -i dumb-init_1.2.1_ppc64el.deb
Selecting previously unselected package dumb-init.
(Reading database ... 98635 files and directories currently installed.)
Preparing to unpack dumb-init_1.2.1_ppc64el.deb ...
Unpacking dumb-init (1.2.1) ...
Setting up dumb-init (1.2.1) ...
Processing triggers for man-db (2.7.5-1) ...
Checking the installed version
root@packer-vm:~# whereis dumb-init
dumb-init: /usr/bin/dumb-init /usr/share/man/man1/dumb-init.1.gz
root@packer-vm:~# dumb-init --version
dumb-init v1.2.1
Awesome, thanks for your work on this and producing a working example @ghatwala!
I'll see if I can get this hooked into our regular CI pipeline.
@ghatwala just to be clear I have no control over the dumb-init repository. I just occasionally comment and use dumb-init myself. @chriskuehl does have the ability and judging from his response I’m hopeful that you’ll get a desirable outcome. Just to say that I care but I have no control.
I think it is great that you made the effort to fork this repository and accomplished getting it working.
@chriskuehl I’m not sure if you’re publishing releases automatically but I know that Travis CI supports building tags. There’s also https://github.com/aktau/github-release project which provides a small static binary for publishing releases to GitHub. You can use that github-release project if you would rather automate release from an external system (external to Travis CI).
Just my 2 cents.
@chriskuehl here’s an example of how I automate releasing in one of my projects. https://github.com/samrocketman/jenkins-bootstrap-shared/blob/master/scripts/uploadRelease.sh
Thanks for those examples. I'm looking into what we can do (trying to navigate the process involved around actually getting credentials not tied to an individual's account for this GitHub org).
We may end up using something like https://docs.travis-ci.com/user/uploading-artifacts/ to upload build artifacts from Travis to S3 instead, and then manually copying those to Releases when we make a new release.
Hang tight -- I'm pretty sure we can figure something out within in the next day or two.
@chriskuehl might be worth creating a service account in GitHub as well that is team shared and rotate the login when someone leaves the team. I have a couple of service accounts on GitHub. It’s within the GH terms of service.
@samrocketman - thanks a lot for your 2 cents :-) !!! . @chriskuehl - thanks for trying to push ppc64le artifacts to releases page as well . Saw that as per this PR - #165 travis build artifacts are generated at s3 bucket , which can now be transferred to github -releases page. ( currently amd64 -*deb only ) . Once this is validated, please try and push - ppc64le-*deb as well .
Will be looking here to wait on the next progress !!!
Okay, we have Travis building for ppc64le and uploading artifacts we can use! 🎉
Before I upload these to the "Releases" tab, can anyone (maybe @ghatwala) sanity-check either the binary or the Debian package is working? Besides Travis I don't really know how to test on this architecture :)
Btw, thanks again @ghatwala for pushing for this and helping get Travis set up!
@chriskuehl - Sure, you're welcome !!! Also as requested see below sanity checks done on above binary + debian installer.
Sanity check for Binary
root@f3b43f150ee9:/# arch
ppc64le
root@f3b43f150ee9:/# wget https://yelp-travis-artifacts.s3.amazonaws.com/dumb-init/145/itest_stretch-linux-ppc64le/dist/dumb-init_1.2.1_ppc64el
--2018-06-06 05:04:04-- https://yelp-travis-artifacts.s3.amazonaws.com/dumb-init/145/itest_stretch-linux-ppc64le/dist/dumb-init_1.2.1_ppc64el
Resolving yelp-travis-artifacts.s3.amazonaws.com (yelp-travis-artifacts.s3.amazonaws.com)... 52.219.24.116
Connecting to yelp-travis-artifacts.s3.amazonaws.com (yelp-travis-artifacts.s3.amazonaws.com)|52.219.24.116|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 66968 (65K) [binary/octet-stream]
Saving to: 'dumb-init_1.2.1_ppc64el'
dumb-init_1.2.1_ppc64el 100%[===========================================================================================>] 65.40K --.-KB/s in 0.08s
2018-06-06 05:04:05 (772 KB/s) - 'dumb-init_1.2.1_ppc64el' saved [66968/66968]
root@f3b43f150ee9:/# chmod u+x dumb-init_1.2.1_ppc64el
root@f3b43f150ee9:/# mv dumb-init_1.2.1_ppc64el /usr/bin/dumb-init
root@f3b43f150ee9:/# whereis dumb-init
dumb-init: /usr/bin/dumb-init
root@f3b43f150ee9:/# dumb-init --version
dumb-init v1.2.1
Similar sanity check for debian installer
root@e7cd57f9d157:/# arch
ppc64le
root@e7cd57f9d157:/# wget https://yelp-travis-artifacts.s3.amazonaws.com/dumb-init/145/itest_stretch-linux-ppc64le/dist/dumb-init_1.2.1_ppc64el.deb
--2018-06-06 05:10:58-- https://yelp-travis-artifacts.s3.amazonaws.com/dumb-init/145/itest_stretch-linux-ppc64le/dist/dumb-init_1.2.1_ppc64el.deb
Resolving yelp-travis-artifacts.s3.amazonaws.com (yelp-travis-artifacts.s3.amazonaws.com)... 54.231.235.43
Connecting to yelp-travis-artifacts.s3.amazonaws.com (yelp-travis-artifacts.s3.amazonaws.com)|54.231.235.43|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 43400 (42K) [application/x-debian-package]
Saving to: 'dumb-init_1.2.1_ppc64el.deb'
dumb-init_1.2.1_ppc64el.deb 100%[===========================================================================================>] 42.38K --.-KB/s in 0.04s
2018-06-06 05:10:58 (1022 KB/s) - 'dumb-init_1.2.1_ppc64el.deb' saved [43400/43400]
root@e7cd57f9d157:/# dpkg -i dumb-init_1.2.1_ppc64el.deb
Selecting previously unselected package dumb-init.
(Reading database ... 5069 files and directories currently installed.)
Preparing to unpack dumb-init_1.2.1_ppc64el.deb ...
Unpacking dumb-init (1.2.1) ...
Setting up dumb-init (1.2.1) ...
root@e7cd57f9d157:/# whereis dumb-init
dumb-init: /usr/bin/dumb-init /usr/share/man/man1/dumb-init.1.gz
root@e7cd57f9d157:/# dumb-init --version
dumb-init v1.2.1
From above , sanity checks for both "dumb-init" - binary + installer passed. So you're free to upload these to the releases tab. 💯 . Thanks a lot as well for your efforts too for adding support for power.
Now just awaiting the update to the "Releases" tab.
@chriskuehl - sorry to bother you again , any timeline by when would u be pushing both debian package ( am interested in debian pkg for now ) + binary to releases page.
@ghatwala thanks for the reminder, I've just uploaded them at https://github.com/Yelp/dumb-init/releases/tag/v1.2.1
@asottile - thanks , am already following up with travis folks on this if there's any issue to upload ppc64le build artifacts .
@chriskuehl - thanks a lot for uploading the power binaries to the releases page.
With that , i can close this ticket for now .