JuliaPackaging/Yggdrasil

Meta issue about building packages for experimental platforms (`aarch64-apple-darwin` and `armv6l-linux-*`)

Closed this issue · 10 comments

giordano commented

There is a large demand to build packages for the "experimental" platforms aarch64-apple-darwin and armv6l-linux-*, but there are two NON-NEGOTIABLE constraints

  1. the JLL package must require at least Julia v1.6, because only Julia v1.6+ is able to deal with the presence of these platforms in the Artifacts.toml file
  2. most existing packages are marked as compatible with Julia v1.0, but as a consequence of the point 1 above if we want to rebuild an existing package for the new platform the generated JLL package must have a new version number. No exceptions whatsoever to this rule, otherwise we'll break the ecosystem for Julia v1.5-

Also, if a package has binary dependencies, we need to build all of them before being able to build it. The fact that we need to require Julia v1.6 means that we need to cut out of support old versions of Julia, and not all package maintainers will be happy about that decision. The rebuild of the packages of a consistent part of the ecosystem will reasonably take a few months, certainly not before a new long-term support (LTS) version of Julia is officially announced.

Examples of commonly requested packages:

giordano commented

Note: I've built bzip2 for the new architectures with #2957, but that changed the soname of the library, which means we can't have downstream libraries depending on both v1.0.6 and v1.0.8 (registration has been put on hold: JuliaRegistries/General#36391), they must be rebuilt against v1.0.8. It should be all fixed now

mkborregaard commented

This seems to happen for ImageMagick as well

julia> using ImageMagick
ERROR: InitError: UndefVarError: libwand not defined
Stacktrace:
 [1] magickgenesis
   @ ~/.julia/packages/ImageMagick/b8swT/src/libmagickwand.jl:21 [inlined]
 [2] __init__()
   @ ImageMagick ~/.julia/packages/ImageMagick/b8swT/src/libmagickwand.jl:27
 [3] _include_from_serialized(path::String, depmods::Vector{Any})
   @ Base ./loading.jl:768
 [4] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
   @ Base ./loading.jl:854
 [5] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1097
 [6] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [7] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997
during initialization of module ImageMagick

Is this thread the right place to report?

giordano commented

Is this thread the right place to report?

Yes, but there was already a PR open (and already merged a couple of weeks ago) for this: #3424

  • 👍1
jeremiahpslewis commented

Sorry for creating a duplicate issue, reposting the content:

There are currently 214 instances of platforms = supported_platforms() in the Yggdrasil code base, i.e. cases where (mainly because the script has yet to be updated), experimental platforms like Apple Silicon are not supported. A rough grep of build_tarballs(ARGS suggests this is ~25% (214 / 926) of binary files.

This migration is a very specific change which is repetitive and creates (what I assume to be) an added burden on maintainers.

Would it make sense to get a few people together on a couple of afternoons and do an 'experimental platforms hackathon' to move this number from 25% towards ~0%? This could be planned so that it is maximally convenient for maintainers and so that open PRs don't pollute the repo.

Thoughts? @giordano

giordano commented

Answered in #3805 (comment)

  • ❤️1
mkborregaard commented

Thanks @giordano , I didn't see that issue above. Great!

gbaraldi commented

libhdf5 isn't supported on either the M1 and also lots of other platforms :s . But it looks to be really awful to build.

giordano commented

We can't cross-compile HDF5, we repackage native builds from the Python ecosystem

imciner2 commented

@giordano is there a reason to keep this open now that the list of experimental platforms is actually empty?

giordano commented

Uh, sorry, I missed the message. No, probably this can be closed now.

  • 👍1