is arrow package available [on arm64 aka aarch64] ?
apsteinmetz opened this issue · 9 comments
apsteinmetz commented
after setting up r4u, R still insists on building from source. Source build always fails for me so I was hoping r4u would solve my issue.
install.packages("arrow",source="binary")
Install system packages as root...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Hit http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports jammy InRelease
Hit http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports jammy-updates InRelease
Hit http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports jammy-backports InRelease
Hit https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease
Hit http://ports.ubuntu.com/ubuntu-ports jammy-security InRelease
Ign https://r2u.stat.illinois.edu/ubuntu jammy InRelease
Hit https://r2u.stat.illinois.edu/ubuntu jammy Release
Hit https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease
Fetched 0 B in 0s (0 B/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Fetched 0 B in 0s (0 B/s)
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/arrow_12.0.1.1.tar.gz'
Content type 'application/x-gzip' length 4097373 bytes (3.9 MB)
==================================================
downloaded 3.9 MB
* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** Building on linux aarch64
*** Found local C++ source: 'tools/cpp'
*** Building libarrow from source
...etc.
eddelbuettel commented
I would need more detail about what and how exactly you set things up. Please show me at least apt-cache policy r-cran-arrow. I cannot tell from what you posted why it does not work for you.
Below the fold is the simple counter example -- clearly work for our default Docker container. (It also works 'all the time' for us in CI where arrow is one of many packages installed in each CI run.)
edd@rob:~$ docker run --rm -ti rocker/r2u Rscript -e 'install.packages("arrow")'
Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
Ign https://r2u.stat.illinois.edu/ubuntu jammy InRelease
Get:2 https://r2u.stat.illinois.edu/ubuntu jammy Release [5713 B]
Get:3 https://r2u.stat.illinois.edu/ubuntu jammy Release.gpg [793 B]
Get:4 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:5 https://r2u.stat.illinois.edu/ubuntu jammy/main all Packages [7386 kB]
Get:6 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease [3626 B]
Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [860 kB]
Get:8 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ Packages [43.3 kB]
Get:9 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 Packages [2335 kB]
Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [897 kB]
Get:11 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [980 kB] [49/99554]
Get:12 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [44.0 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
Get:14 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB]
Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:17 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:18 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1241 kB]
Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.8 kB]
Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [923 kB]
Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1136 kB]
Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [25.6 kB]
Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [49.2 kB]
Fetched 36.3 MB in 0s (0 B/s)
Install system packages as root...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Get:1 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-assertthat all 0.2.1-1.ca2204.1 [52.7 kB]
Get:2 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-bit amd64 4.0.5-1.ca2204.1 [1061 kB]
Get:3 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-bit64 amd64 4.0.5-1.ca2204.1 [466 kB]
Get:4 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-glue amd64 1.6.2-1.ca2204.1 [143 kB]
Get:5 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-cli amd64 3.6.1-1.ca2204.1 [1223 kB]
Get:6 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-rlang amd64 1.1.1-1.ca2204.1 [1488 kB]
Get:7 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-lifecycle all 1.0.3-1.ca2204.1 [112 kB]
Get:8 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-magrittr amd64 2.0.3-1.ca2204.1 [199 kB]
Get:9 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-vctrs amd64 0.6.3-1.ca2204.1 [1213 kB]
Get:10 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-purrr amd64 1.0.2-1.ca2204.1 [475 kB]
Get:11 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-r6 all 2.5.1-1.ca2204.1 [83.2 kB]
Get:12 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-withr all 2.5.0-1.ca2204.1 [222 kB]
Get:13 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-tidyselect all 1.2.0-1.ca2204.1 [207 kB]
Get:14 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-cpp11 all 0.4.6-1.ca2204.1 [220 kB]
Get:15 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-arrow amd64 12.0.1.1-1.ca2204.1 [14.5 MB]
Fetched 21.6 MB in 0s (0 B/s)
Selecting previously unselected package r-cran-assertthat.
(Reading database ... 18301 files and directories currently installed.)
Preparing to unpack .../00-r-cran-assertthat_0.2.1-1.ca2204.1_all.deb ...
Unpacking r-cran-assertthat (0.2.1-1.ca2204.1) ...
Selecting previously unselected package r-cran-bit.
Preparing to unpack .../01-r-cran-bit_4.0.5-1.ca2204.1_amd64.deb ...
Unpacking r-cran-bit (4.0.5-1.ca2204.1) ...
Selecting previously unselected package r-cran-bit64.
Preparing to unpack .../02-r-cran-bit64_4.0.5-1.ca2204.1_amd64.deb ...
Unpacking r-cran-bit64 (4.0.5-1.ca2204.1) ...
Selecting previously unselected package r-cran-glue.
Preparing to unpack .../03-r-cran-glue_1.6.2-1.ca2204.1_amd64.deb ...
Unpacking r-cran-glue (1.6.2-1.ca2204.1) ...
Preparing to unpack .../03-r-cran-glue_1.6.2-1.ca2204.1_amd64.deb ...
Unpacking r-cran-glue (1.6.2-1.ca2204.1) ...
Selecting previously unselected package r-cran-cli.
Preparing to unpack .../04-r-cran-cli_3.6.1-1.ca2204.1_amd64.deb ...
Unpacking r-cran-cli (3.6.1-1.ca2204.1) ...
Selecting previously unselected package r-cran-rlang.
Preparing to unpack .../05-r-cran-rlang_1.1.1-1.ca2204.1_amd64.deb ...
Unpacking r-cran-rlang (1.1.1-1.ca2204.1) ...
Selecting previously unselected package r-cran-lifecycle.
Preparing to unpack .../06-r-cran-lifecycle_1.0.3-1.ca2204.1_all.deb ...
Unpacking r-cran-lifecycle (1.0.3-1.ca2204.1) ...
Selecting previously unselected package r-cran-magrittr.
Preparing to unpack .../07-r-cran-magrittr_2.0.3-1.ca2204.1_amd64.deb ...
Unpacking r-cran-magrittr (2.0.3-1.ca2204.1) ...
Selecting previously unselected package r-cran-vctrs.
Preparing to unpack .../08-r-cran-vctrs_0.6.3-1.ca2204.1_amd64.deb ...
Unpacking r-cran-vctrs (0.6.3-1.ca2204.1) ...
Selecting previously unselected package r-cran-purrr.
Preparing to unpack .../09-r-cran-purrr_1.0.2-1.ca2204.1_amd64.deb ...
Unpacking r-cran-purrr (1.0.2-1.ca2204.1) ...
Selecting previously unselected package r-cran-r6.
Preparing to unpack .../10-r-cran-r6_2.5.1-1.ca2204.1_all.deb ...
Unpacking r-cran-r6 (2.5.1-1.ca2204.1) ...
Selecting previously unselected package r-cran-withr.
Preparing to unpack .../11-r-cran-withr_2.5.0-1.ca2204.1_all.deb ...
Unpacking r-cran-withr (2.5.0-1.ca2204.1) ...
Selecting previously unselected package r-cran-tidyselect.
Preparing to unpack .../12-r-cran-tidyselect_1.2.0-1.ca2204.1_all.deb ...
Unpacking r-cran-tidyselect (1.2.0-1.ca2204.1) ...
Selecting previously unselected package r-cran-cpp11.
Preparing to unpack .../13-r-cran-cpp11_0.4.6-1.ca2204.1_all.deb ...
Unpacking r-cran-cpp11 (0.4.6-1.ca2204.1) ...
Selecting previously unselected package r-cran-arrow.
Preparing to unpack .../14-r-cran-arrow_12.0.1.1-1.ca2204.1_amd64.deb ...
Unpacking r-cran-arrow (12.0.1.1-1.ca2204.1) ...
Setting up r-cran-rlang (1.1.1-1.ca2204.1) ...
Setting up r-cran-assertthat (0.2.1-1.ca2204.1) ...
Setting up r-cran-withr (2.5.0-1.ca2204.1) ...
Setting up r-cran-glue (1.6.2-1.ca2204.1) ...
Setting up r-cran-bit (4.0.5-1.ca2204.1) ...
Setting up r-cran-cli (3.6.1-1.ca2204.1) ...
Setting up r-cran-lifecycle (1.0.3-1.ca2204.1) ...
Setting up r-cran-bit64 (4.0.5-1.ca2204.1) ...
Setting up r-cran-cpp11 (0.4.6-1.ca2204.1) ...
Setting up r-cran-r6 (2.5.1-1.ca2204.1) ...
Setting up r-cran-magrittr (2.0.3-1.ca2204.1) ...
Setting up r-cran-vctrs (0.6.3-1.ca2204.1) ...
Setting up r-cran-tidyselect (1.2.0-1.ca2204.1) ...
Setting up r-cran-purrr (1.0.2-1.ca2204.1) ...
Setting up r-cran-arrow (12.0.1.1-1.ca2204.1) ...
edd@rob:~$ eddelbuettel commented
Oh, I spotted it too late aarch64
r2u is x86_64 only. Sorry. We try to state that clearly in the documentation, if you have suggestions for how to state it even more clearly let me know.
eddelbuettel commented
As you are on ec2, 'just' set up another instance that is x86_64 based. I use one too, have r2u on it, and it flies.
eddelbuettel commented
The first full paragraph after the summary bullet points and demo tries to detail this:
apsteinmetz commented
ARM. So that's probably it. I was using ARM because EC2 instances are cheaper per core. Thanks!
eddelbuettel commented
Yep. Still and Intel world here. Arm (ie Graviton) matters too but we just do not have the input binaries, sorry.
I just reworded the README a little more to stress that.
Thanks for your interest in r2u. I hope it will come in handy another time for you on the matching hardware.
eddelbuettel commented
I just checked, and we correctly restrict the architecture in the apt setup:
echo "deb [arch=amd64] https://r2u.stat.illinois.edu/ubuntu jammy main" > /etc/apt/sources.list.d/cranapt.listSo when you then check at the apt level via the suggsted sudo apt-cache policy r-cran-arrow (in essence: asking apt to show which binaries it knows, and in the case of multiple (like r-base-core) detail which it would pick) you would not see a binary which is why R tries to help you and builds from source. Which is all you have here I am afraid.
apsteinmetz commented
r4u is working great now on x86 arch!
eddelbuettel commented
I keep (very) loose tabs on aggregate downloads (every few weeks) and based on the last read we should be in the vicinity of seven million binary packages shipped. It is a big deal, and yes it works. And thanks to you for trying it out!
r2u: fast, easy, reliable. (<-- my new tagline)