shadow-maint/shadow

undefined reference to `mallocarray' on Gentoo

alejandro-colomar opened this issue · 18 comments

alejandro-colomar commented

https://bugs.gentoo.org/912446

Reported-by: @cbricart
Reported-by: Robert Marmorstein
Cc: @thesamesam
Cc: @jubalh

I suspect this may have to do with the libmisc and lib separation issue. Maybe moving all of the files to lib/ would fix this. I'll send a PR, and see what happens. Would any of you be able to test the PR on Gentoo before merging it?

cbricart commented

yes - can do for arm.

however, must say I do not experience this on any of my amd64 boxes, like Robert wrote in the bug

jubalh commented

Would any of you be able to test the PR on Gentoo before merging it?

I also can't reproduce this bug for amd64. As a last resort we can still ask the reporter to apply the patch.

alejandro-colomar commented

Well, I guess if I can fix it for ARM, it should get fixed for amd64 too if the cause is the same. Thanks @cbricart ! I'll ping you later when I post a fix.

jubalh commented

BTW the armv7l build in openSUSE seems to build and work fine.

cbricart commented

…what I could suspect here is, that while Gentoo builds make use of
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
the missing reference might get accidentally "optimized-out" of the lib?

alejandro-colomar commented

That's an interesting thing. I thought something like that could be the cause, because I added C99 inline functions, which are externally defined in libmisc, but inline definitions are in lib, so linker errors like that are likely, if the dependencies are not well specified.

I think that moving the function definitions to lib could fix it.

The real issue is that we don't really know very well how the libmisc build system works, but we tried to get it working and couldn't, so my suggestion is to not add any new files to it, and instead add all library functions to lib/, and kind of deprecate libmisc.

cbricart commented

this article might have some ideas for you to look at/for?

alejandro-colomar commented

No, I do understand --as-needed and related linker flags (I actually did learn most of it from Diego's posts about it). Thanks :)

My problem is that I don't understand the autotools code that is used for this specific build system (in general, my autotools experience is ~= 0).

jubalh commented

Robert Förster mentions in https://bugs.gentoo.org/912446#c8 that it solves the problem for him.

lslebodn commented

Robert Förster mentions in https://bugs.gentoo.org/912446#c8 that it solves the problem for him.

libtool: link: armv7a-unknown-linux-gnueabihf-gcc -isystem /usr/include/bsd -DLIBBSD_OVERLAY -march=armv8-a+crc+simd -mtune=cortex-a53 -mlibarch=armv8-a+crc+simd -mfloat-abi=hard -mfpu=vfpv3-d16 -mtls-dialect=gnu -marm -dumpbase -O2 -pipe -Wl,-O1 -o grpck grpck.o  -Wl,--as-needed ../libmisc/.libs/libmisc.a -lcrypt -lsystemd ../lib/.libs/libshadow.a -lbsd
/usr/lib/gcc/armv7a-unknown-linux-gnueabihf/13/../../../../armv7a-unknown-linux-gnueabihf/bin/ld: ../lib/.libs/libshadow.a(libshadow_la-groupio.o): in function `merge_group_entries':
groupio.c:(.text+0x934): undefined reference to `mallocarray'

The function merge_group_entries is defined in lib/groupio.c

The function mallocarray is indeed used in the merge_group_entries via macro MALLOC

But libshadow is not linked with any library

libshadow_la_LIBADD = somelibrary.la

as it is done for other libraries

lslebodn commented

@alejandro-colomar could you link libmisc.la into libshadow.la similar as it is done with libsubid.la ?
rather than moving file here and back as in #792

alejandro-colomar commented
alejandro-colomar commented
lslebodn commented

I prefer removing the library, and merging both of them (libshadow
and libmisc) into one great libshadow.

All files which were linked with libshadow were also linked with libmisc
and there was not any binary which was linked just with libmisc.
So there is not any benefit of 2 libraries :-) :-) :-). Maybe situation was different in past

+1

For the future, I may be even more aggressive and try removing the
autotools code, replacing it by a (GNU)Makefile written from scratch.

I wish you good luck with writing portable (GNU)Makefile written from scratch.
because that was goal of autotools to generate portable makefiles :-)

BTW there is really great book bout Autoconf, Automake, and Libtool
"Autotools: A Practitioner's Guide to GNU Autoconf, Automake, and Libtool"
I highly recommend it. Much better than reinventing wheel.

Or if you decode for "rewrite" build system than try to use existing tools designed to build (cmake, meson ...)
"(GNU)Makefile written from scratch." is a step back even from current version with autotools
(even though you needn't like that :-)

alejandro-colomar commented

I prefer removing the library, and merging both of them (libshadow
and libmisc) into one great libshadow.

All files which were linked with libshadow were also linked with libmisc and there was not any binary which was linked just with libmisc. So there is not any benefit of 2 libraries :-) :-) :-). Maybe situation was different in past

+1

Thanks! I stored that in the commit as the last of a long list of Acked-bys :)

For the future, I may be even more aggressive and try removing the
autotools code, replacing it by a (GNU)Makefile written from scratch.

I wish you good luck with writing portable (GNU)Makefile written from scratch.

Now I can't find the source and exact quote, but I bet it was @madscientist who said something like "don't write portable Makefiles; write GNU Makefiles, and port GNU Make" :)

because that was goal of autotools to generate portable makefiles :-)

A portable Makefile is quite a horrible thing. Each implementation is too different.
I prefer writing high-quality GNUMakefiles, and assume that GNU Make will be available (which is true for all targets I support).

BTW there is really great book bout Autoconf, Automake, and Libtool "Autotools: A Practitioner's Guide to GNU Autoconf, Automake, and Libtool" I highly recommend it. Much better than reinventing wheel.

Heh, I think I already went too far with GNU Make. The Linux man-pages' makefiles, which I rewrote from scratch in the last 3 years, are what I have in mind for this project too some day.

https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/GNUmakefile
https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/share/mk

I admit maybe it's a bit of a monster[1], but the usefulness compensates for it, IMO.

[1]:

$ find ./GNUMakefile ./share/mk/ -type f | xargs wc -l
find: ‘./GNUMakefile’: No such file or directory
   70 ./share/mk/lint/man/man.mk
   16 ./share/mk/lint/man/_.mk
   48 ./share/mk/lint/man/mdoc.mk
   18 ./share/mk/lint/_.mk
  101 ./share/mk/lint/c.mk
   20 ./share/mk/version.mk
   18 ./share/mk/check/_.mk
   51 ./share/mk/check/catman.mk
   40 ./share/mk/compress.mk
   17 ./share/mk/verbose.mk
   40 ./share/mk/cmd.mk
   98 ./share/mk/dist.mk
   50 ./share/mk/src.mk
   47 ./share/mk/build/pre.mk
   68 ./share/mk/build/pdf.mk
   39 ./share/mk/build/groff.mk
   42 ./share/mk/build/html.mk
   68 ./share/mk/build/ps.mk
   52 ./share/mk/build/_.mk
  117 ./share/mk/build/src.mk
   89 ./share/mk/build/catman.mk
  116 ./share/mk/install/man.mk
   36 ./share/mk/install/html.mk
   37 ./share/mk/install/_.mk
 1298 total

Or if you decode for "rewrite" build system than try to use existing tools designed to build (cmake, meson ...)

cmake and meson, like autotools, and any other Makefile generator, have a big problem, IMO:

They are two-step systems. You first configure, then make.

The Linux man-pages set of Makefiles is a one-step build system, and you can just trust make ... to do the right thing at any point in time. No matter how much you changed the source code, or you added new files, or removed some, it will do the "configure" as part of the "make". You don't need to make clean or things like that any more (I already forgot about the last time I ran make clean, but it's measured in years).

"(GNU)Makefile written from scratch." is a step back even from current version with autotools (even though you needn't like that :-)

Please check out that project, have a look at the README, INSTALL, and CONTRIBUTING files, play with the Makefile, and tell me if you still believe that. :-)

https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git

If you miss any feature of other build systems in mine, please report them at linux-man@vger.kernel.org. :)

ikerexxe commented

because that was goal of autotools to generate portable makefiles :-)

A portable Makefile is quite a horrible thing. Each implementation is too different. I prefer writing high-quality GNUMakefiles, and assume that GNU Make will be available (which is true for all targets I support).

I agree with Lukas that autotools was created for a number of reasons and using plain Makefiles would resurrect those problems again. I'm not saying I like autotools, but I definitely prefer it to Makefiles. Maybe we should start using it properly, or maybe we should move on to something else, but before proposing a PR with changes I would like a discussion where the community can express their opinion about this topic.

BTW there is really great book bout Autoconf, Automake, and Libtool "Autotools: A Practitioner's Guide to GNU Autoconf, Automake, and Libtool" I highly recommend it. Much better than reinventing wheel.

A book that I should read, thank you for the recommendation.

  • 👍1
alejandro-colomar commented

@lslebodn

I wish you good luck with writing portable (GNU)Makefile written from scratch.

Now I can't find the source and exact quote, but I bet it was @madscientist who said something like "don't write portable Makefiles; write GNU Makefiles, and port GNU Make" :)

Ahh, it was "Paul's First Rule" what I remembered! :)

because that was goal of autotools to generate portable makefiles :-)

A portable Makefile is quite a horrible thing. Each implementation is too different. I prefer writing high-quality GNUMakefiles, and assume that GNU Make will be available (which is true for all targets I support).

alejandro-colomar commented

because that was goal of autotools to generate portable makefiles :-)

A portable Makefile is quite a horrible thing. Each implementation is too different. I prefer writing high-quality GNUMakefiles, and assume that GNU Make will be available (which is true for all targets I support).

I agree with Lukas that autotools was created for a number of reasons and using plain Makefiles would resurrect those problems again.

I'm sure most of those are either problems of non-GNU make(1), or of ancient versions of GNU make(1), or just FUD. Recent versions of GNU Make are free of the limitations that would make you use autotools, I believe.

Would you mind expressing a list of the problems you'd expect of a hand-written GNUMakefile? I can open a new issue to concentrate this discussion. I'll make sure that a hand-written GNUMakefile is free of all of those problems.

I'm not saying I like autotools, but I definitely prefer it to Makefiles. Maybe we should start using it properly, or maybe we should move on to something else, but before proposing a PR with changes I would like a discussion where the community can express their opinion about this topic.

Sure, I'll open an issue where you can express the minimum requirements that you'd like to see from a build system, and I'll make sure that I include them in my implementation.

Regarding proposing a PR, don't worry. Such a thing would take many months or even a year of development, so you'll be able to see how it's going on before the final review. But I'd like to show you some state-of-the-art GNUMakefile before you discard it as a technology of the past. :D

BTW, I also invite you to check the Linux man-pages project's GNUMakefile. Please read ./README, ./INSTALL, and ./CONTRIBUTING, and run make help and make help-variables, and then play a bit with that build system, and check if your concerns remain. That project doesn't have complex dependencies between source and header files, but I have other makefiles in my private projects that use those, and it's all doable. ;)