andrewwutw/build-djgpp

compiling is broken when compiling against muslc

Neo-Desktop opened this issue · 10 comments

Neo-Desktop commented

basically its a gcc problem but the djgpp doesn't build under a muslc environment (and therefore alpine linux)

I had to include this patch for it to build correctly

https://gitweb.gentoo.org/proj/gcc-patches.git/tree/12.1.0/musl/50_all_calloc_libgccjit.patch?id=caa4485a49e1064181e7ac2df2c5476fbb6e9cf5

side note- I also prefer to use clang even though I'm not in a freebsd environment. The build script overrides both CC and CXX even if I pass it in as an environment variable.

the patches below remove the freebsd check and just override the script to use clang and clang++, this is obviously not necessary but it conforms to my usecase

this is the patch set I use to get it to compile correctly under alpine:

/tmp/patches/djgcc/0001-Add-calloc-patch-to-build-script.patch

diff --git a/script/12.1.0 b/script/12.1.0
index 834e0c2..3d94495 100755
--- a/script/12.1.0
+++ b/script/12.1.0
@@ -36,14 +36,10 @@ AUTOMAKE_VERSION=1.15.1
 CC=gcc
 CXX=g++

-# use gmake under FreeBSD
-if [ `uname` = "FreeBSD" ]; then
-  MAKE=gmake
+# use clang on alpine
   CC=clang
   CXX=clang++
-else
   MAKE=make
-fi
 #CFLAGS="-O2 -g $CFLAGS -std=gnu11"

 export CC CXX CFLAGS MAKE
@@ -288,6 +284,10 @@ if uname|grep "^MINGW32" > /dev/null; then
   (cd $BUILDDIR && patch -p0 < $BUILDDIR/../../patch/patch-gcc12-mingw.txt) || exit 1
 fi

+cp /tmp/patches/djgcc/0002-Fix-attempt-to-use-poisoned-calloc-error-in-libgccji.patch $BUILDDIR/../../patch/
+echo "Patch GCC for alpine"
+(cd $BUILDDIR && patch -p0 < $BUILDDIR/../../patch/0002-Fix-attempt-to-use-poisoned-calloc-error-in-libgccji.patch) || exit 1
+
 # patch gnu/gcc-X.XX/gcc/doc/gcc.texi
 echo "Patch gcc/doc/gcc.texi"
 cd gnu/gcc-*/gcc/doc || exit 1

/tmp/patches/djgcc/0002-Fix-attempt-to-use-poisoned-calloc-error-in-libgccji.patch

https://git.alpinelinux.org/aports/plain/main/gcc/0042-Fix-attempt-to-use-poisoned-calloc-error-in-libgccji.patch
https://bugs.gentoo.org/828580
https://gcc.gnu.org/bugzillgnu-orig/gcc-12.10/show_bug.cgi?id=104799 (additional two hunks thrown in from here for 12)

From 72f32e3383129ad139df76d9a063fac9b03079b1 Mon Sep 17 00:00:00 2001
From: Alex McGrath <amk@amk.ie>
Date: Wed, 13 Oct 2021 23:24:27 +0100
Subject: [PATCH] Fix attempt to use poisoned calloc error in libgccjit

This moves usages of pthread.h to above any usage of system.h as it
included #pragma GCC poison calloc
--- gnu-orig/gcc-12.10/gcc/jit/jit-playback.cc
+++ gnu/gcc-12.10/gcc/jit/jit-playback.cc
@@ -18,6 +18,8 @@ You should have received a copy of the GNU General Public License
 along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
+#include <pthread.h>
+
 #include "config.h"
 #include "system.h"
 #include "coretypes.h"
@@ -41,8 +43,6 @@ along with GCC; see the file COPYING3.  If not see
 #include "diagnostic.h"
 #include "stmt.h"
 
-#include <pthread.h>
-
 #include "jit-playback.h"
 #include "jit-result.h"
 #include "jit-builtins.h"
--- gnu-orig/gcc-12.10/gcc/jit/jit-recording.cc
+++ gnu/gcc-12.10/gcc/jit/jit-recording.cc
@@ -18,6 +18,8 @@ You should have received a copy of the GNU General Public License
 along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
+#include <pthread.h>
+
 #include "config.h"
 #include "system.h"
 #include "coretypes.h"
@@ -25,8 +27,6 @@ along with GCC; see the file COPYING3.  If not see
 #include "pretty-print.h"
 #include "toplev.h"
 
-#include <pthread.h>
-
 #include "jit-builtins.h"
 #include "jit-recording.h"
 #include "jit-playback.h"
--- gnu-orig/gcc-12.10/gcc/jit/libgccjit.cc
+++ gnu/gcc-12.10/gcc/jit/libgccjit.cc
@@ -18,13 +18,14 @@ You should have received a copy of the GNU General Public License
 along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
+#include <pthread.h>
+
 #include "config.h"
 #include "system.h"
 #include "coretypes.h"
 #include "timevar.h"
 #include "typed-splay-tree.h"
 #include "cppbuiltin.h"
-#include <pthread.h>
 
 #include "libgccjit.h"
 #include "jit-recording.h"
--- gnu-orig/gcc-12.10/libcc1/libcc1plugin.cc
+++ gnu/gcc-12.10/libcc1/libcc1plugin.cc
@@ -17,6 +17,7 @@
    along with GCC; see the file COPYING3.  If not see
    <http://www.gnu.org/licenses/>.  */
 
+#include <pthread.h>
 #include <cc1plugin-config.h>
 
 #undef PACKAGE_NAME
--- gnu-orig/gcc-12.10/libcc1/libcp1plugin.cc
+++ gnu/gcc-12.10/libcc1/libcp1plugin.cc
@@ -18,6 +18,7 @@
    along with GCC; see the file COPYING3.  If not see
    <http://www.gnu.org/licenses/>.  */
 
+#include <pthread.h>
 #include <cc1plugin-config.h>
 
 #undef PACKAGE_NAME

--- gnu-orig/gcc-12.10/gcc/cp/mapper-client.cc
+++ gnu/gcc-12.10/gcc/cp/mapper-client.cc
@@ -27,6 +27,7 @@ along with GCC; see the file COPYING3.  If not see
 #define INCLUDE_STRING
 #define INCLUDE_VECTOR
 #define INCLUDE_MAP
+#include <pthread.h>
 #include "system.h"
 
 #include "line-map.h"
--- gnu-orig/gcc-12.10/gcc/cp/module.cc
+++ gnu/gcc-12.10/gcc/cp/module.cc
@@ -204,6 +204,7 @@ Classes used:
 #include "config.h"
 #define INCLUDE_STRING
 #define INCLUDE_VECTOR
+#include <pthread.h>
 #include "system.h"
 #include "coretypes.h"
 #include "cp-tree.h"
--- gnu-orig/gcc-12.10/gcc/cp/mapper-resolver.cc
+++ gnu/gcc-12.10/gcc/cp/mapper-resolver.cc
@@ -25,6 +25,7 @@ along with GCC; see the file COPYING3.  If not see
 #define INCLUDE_VECTOR
 #define INCLUDE_ALGORITHM
 #define INCLUDE_MAP
+#include <pthread.h>
 #include "system.h"
 
 // We don't want or need to be aware of networking
volkertb commented

@Neo-Desktop Do you maybe know if this patch still needs to be applied to GCC 12.2.0? That's the latest DJGPP version that build-djgpp supports at this time of writing, but I can't get that version to build on Alpine Linux ether. But I'm not getting a poisoned calloc error. Or at least, that error wouldn't stand out from the log. It's really failing without a clear compile error.

This is the Dockerfile that I wrote that reproduces the build error:

# SPDX-FileType: SOURCE
# SPDX-FileCopyrightText: Copyright (C) 2022,2024 Volkert de Buisonjé
# SPDX-FileContributor: Volkert de Buisonjé
# SPDX-License-Identifier: Apache-2.0

FROM alpine:3.20.0

ARG DJGPP_RELEASE_VERSION=3.4
ARG GCC_VERSION=12.2.0
ARG DJGPP_TARBALL_NAME=v${DJGPP_RELEASE_VERSION}.tar.gz
ARG DJGPP_TARBALL_SHA256=91caa982f42761d56eae90de5f05ac97b68a24ca8603f5ce42936c19e155da66

# Download, verify and unpack the prebuilt DJGPP binaries for Linux
RUN apk add --no-cache wget
RUN wget -P /tmp https://github.com/andrewwutw/build-djgpp/archive/refs/tags/${DJGPP_TARBALL_NAME}
RUN echo "${DJGPP_TARBALL_SHA256}  /tmp/${DJGPP_TARBALL_NAME}" | sha256sum -c
RUN mkdir /src
RUN tar -xf /tmp/${DJGPP_TARBALL_NAME} -C /src
RUN rm /tmp/${DJGPP_TARBALL_NAME}

RUN ls -lh /src/build-djgpp-${DJGPP_RELEASE_VERSION}
WORKDIR /src/build-djgpp-${DJGPP_RELEASE_VERSION}

# Most projects that need to be built with DJGPP will likely also need GNU Make, so let's include it in this image.
RUN apk add --no-cache make

# In addition to make and wget, install additional programs necessary to build DJGPP:
RUN apk add --no-cache g++ gcc unzip bison flex texinfo patch zlib-dev curl bash

# Maybe `file` is also required to make Alpine build the project successfully?
# FIXME TODO : the script still isn't working. Figure out why.
RUN apk add --no-cache file

RUN ./build-djgpp.sh ${GCC_VERSION}
volkertb commented

@Neo-Desktop Through some googling, I found your gist for building DJGPP 12.2.0 on Alpine.

Thanks for that! I'll try that out.

By the way, under what license are you releasing the script in your gist?

  • 👍1
  • 🎉1
Neo-Desktop commented

@Neo-Desktop Through some googling, I found your gist for building DJGPP 12.2.0 on Alpine.

Thanks for that! I'll try that out.

By the way, under what license are you releasing the script in your gist?

Ah yes, I think that script can be MIT Licensed, though the version that ended up in the UMSKT fork of DJGPP is licensed the same as this repo

  • ❤️1
volkertb commented

Ah, thanks for pointing out the UMSKT fork of build-djgpp. I wasn't aware of that one yet. And I see that it also added support for multicore builds. Nice. 🙂

  • 🎉1
volkertb commented

Hmmm... Strange...

In my Dockerfile, the most recent commit of that fork has a weird issue where running ./build-djgpp.sh lists your 12.2.0-i386 variant as one of the versions I can select. So far, so good.

But if I tell the Dockerfile to run ./build-djgpp.sh 12.2.0-i386 (just like in the GitHub Actions Workflow of that same project), I get the error Unsupported GCC version : 12.2.0-i386. 😕

If I select 12.2.0, it does accept that and start building, but then the build still fails, since the required Alpine patch apparently isn't applied to that existing version script.

I think I'll just try to use your script and patch directly from my Dockerfile, instead of relying on build-djgpp.

  • 👍1
Neo-Desktop commented

@volkertb

But if I tell the Dockerfile to run ./build-djgpp.sh 12.2.0-i386 (just like in the GitHub Actions Workflow of that same project), I get the error Unsupported GCC version : 12.2.0-i386. 😕

IIRC that's due to incorrect permissions (i.e. missing execute bit) on the script - sounds like we didn't really test that repository, we ended up just using the binary builds in the UMSKT build action

  • 🎉1
volkertb commented

Yeah, that was it. Thanks! Running chmod +x script/12.2.0-i386 before running ./build-djgpp.sh 12.2.0-i386 indeed allowed it to start building. 🙂

But it's weird that the GitHub Actions Workflow pipeline in the fork did work without setting the execute permissions.

I'd like to backport the Alpine and multicore fixes to the existing 12.2.0 script for i586 CPU targets as well, but from what I can see from your script, that will trivial. Would a PR be appreciated?

Right now I'm waiting for the build of DJGPP 12.2.0-i386 on Alpine to succeed. 🤞

volkertb commented

It didn't work. Again, it failed at the Building gcc step. 😞

It's getting late here. I'll resume experimenting with this another day. Like I said above, I'll try using the script and patch from your Gist directly in my Dockerfile instead of build-djgpp. If you want, I can share the results of that in this thread.

Your assistance has been quite helpful.

Thanks again and bye for now!

  • 👍1
Neo-Desktop commented

Sounds good, and sure - though just to be perfectly clear, the only difference the -i386 version makes is setting the default -march and -mtune options to be i386 vs anything else. Setting those options when invoking djgpp gcc is more important than changing the defaults (but I wasn't fully aware of that information at the time 😅)

Neo-Desktop commented

It didn't work. Again, it failed at the Building gcc step. 😞

It's getting late here. I'll resume experimenting with this another day. Like I said above, I'll try using the script and patch from your Gist directly in my Dockerfile instead of build-djgpp. If you want, I can share the results of that in this thread.

Your assistance has been quite helpful.

Thanks again and bye for now!

Understood, you're welcome to connect with me at the UMSKT Discord so we aren't going off topic on this thread. But it sounds like there's some fundamental changes to how muslc/Alpine builds gcc and it's broken compatibility with these patches