Compile fails when exiv2 is not installed
D3vil0p3r opened this issue · 18 comments
I'm trying to compile bulk_extractor on NixOS but I'm getting the following error during make:
g++ -std=c++17 -DHAVE_CONFIG_H -I. -I.. -I../src/be20_api -I../src/be20_api/utfcpp/source -DUTC_OFFSET=+0000 -g -pthread -Wall -MD -Wpointer-arith -Wshadow -Wwrite-strings -Wcast-align -Wredundant-decls -Wdisabled-optimization -Wfloat-equal -Wmultichar -Wmissing-noreturn -Woverloaded-virtual -Wsign-promo -Weffc++ -fPIC -O3 -MT bulk_extractor.o -MD -MP -MF $depbase.Tpo -c -o bulk_extractor.o bulk_extractor.cpp &&\
mv -f $depbase.Tpo $depbase.Po
In file included from bulk_extractor.cpp:49:
bulk_extractor_restarter.h: In member function ‘void bulk_extractor_restarter::restart(Phase1::Config&, scanner_config&)’:
bulk_extractor_restarter.h:91:55: warning: declaration of ‘sc’ shadows a member of ‘bulk_extractor_restarter’ [-Wshadow]
91 | void restart(Phase1::Config &cfg, scanner_config &sc) {
| ~~~~~~~~~~~~~~~~^~
bulk_extractor_restarter.h:24:30: note: shadowed declaration is here
24 | scanner_config ≻
| ^~
bulk_extractor_restarter.h:91:34: warning: declaration of ‘cfg’ shadows a member of ‘bulk_extractor_restarter’ [-Wshadow]
91 | void restart(Phase1::Config &cfg, scanner_config &sc) {
| ~~~~~~~~~~~~~~~~^~~
bulk_extractor_restarter.h:25:30: note: shadowed declaration is here
25 | Phase1::Config &cfg;
| ^~~
bulk_extractor.cpp: In function ‘int bulk_extractor_main(std::ostream&, std::ostream&, int, char* const*)’:
bulk_extractor.cpp:485:22: error: no matching function for call to ‘bulk_extractor_restarter::restart()’
485 | r.restart(); // load the restart file and rename report.xml
| ~~~~~~~~~^~
bulk_extractor_restarter.h:91:10: note: candidate: ‘void bulk_extractor_restarter::restart(Phase1::Config&, scanner_config&)’
91 | void restart(Phase1::Config &cfg, scanner_config &sc) {
| ^~~~~~~
bulk_extractor_restarter.h:91:10: note: candidate expects 2 arguments, 0 provided
make[2]: *** [Makefile:1377: bulk_extractor.o] Error 1
make[2]: Leaving directory '/home/athena/pkgs/bulk-extractor/bulk_extractor/src'
make[1]: *** [Makefile:411: all-recursive] Error 1
make[1]: Leaving directory '/home/athena/pkgs/bulk-extractor/bulk_extractor'
make: *** [Makefile:352: all] Error 2
Solved by adding afflib and exiv2 as dependencies. It could be useful to have them in README or BUILD instructions.
Afflib is no longer supported and should not be used.
Where are the dependencies listed that you did not find them? Exiv should automatically detect and if it is not present the configure script shouldn't have included it.
Are you compiling from sources? It would be nice to fix this so that it doesn't cause other people problems.
I took the list of dependencies from this BlackArch PKGBUILD. I'm compiling from source by downloading the .tar.gz from https://digitalcorpora.s3.amazonaws.com/downloads/bulk_extractor/bulk_extractor-2.0.3.tar.gz
I did a further test. I tried to remove afflib dependency and the compilation still works. If I remove exiv2, I get the error reported above. So I think that exiv2, at least in NixOS, should be necessary.
Furthermore, for BEViewer, is there a specific java version to install? For example jdk11, jdk17 or jdk19?
My only references where that PKGBUILD I linked you before and this .nix file of an old version of bulk_extractor: NixOS/nixpkgs#82265 . On your repository, I referred to https://github.com/simsong/bulk_extractor/blob/main/etc/CONFIGURE_FEDORA36.bash to know what are the dependencies to use.
For [hopefully helpful] context, NixOS is a distro of Linux that uses the nix package manager for controlling the configuration of the system. The nix package manager constructs a Merkle tree of the software packages installed, where both content of source files and configuration options feed into the overall hash of the package. nix then uses an elaborate system of symlinks to set one's PATH and other environment variables so that the environment is somewhat normal. I've played with it a bit a couple years ago. It's neat but (then) was ultimately too frustrating for me to use. The benefit is that everything about the system is well-specified and easily replicated from a very small set of Haskell-based configuration scripts; builds are fully reproducible.
Given the degree to which nix buggers the normal Linux directory structure and environment, the inherent complexity therein, the number of dependencies of bulk_extractor, and the paleolithic technology that is autoconf, it's not surprising that there's a build problem with bulk_extractor on NixOS. Simson, you are correct that the focus should be on configure.ac and what its behavior is when exiv2 isn't found.
Yes, I'm concerned that it didn't work when exiv2 wasn't installed.
Another small non-impacting note is that, at the end of building, also a test_be file is generated inside bin directory (accessible by PATH). Would be cleaner if test_be won't be automatically stored in bin directory?
Well, it's a binary. It just shouldn't be installed. Why is it getting installed on your system? It doesn't get installed on mine, the last time I checked.
Well, it's a binary. It just shouldn't be installed. Why is it getting installed on your system? It doesn't get installed on mine, the last time I checked.
I'm investigating, I think it could be related how nix works. Is there a make config to prevent the creation of this test_be?
What I see in bulk_extractor/src folder after make command in the root repo folder is that, inside generated Makefile, we have:
EXEEXT =
...
bin_PROGRAMS = bulk_extractor$(EXEEXT) test_be$(EXEEXT)
...
install-binPROGRAMS: $(bin_PROGRAMS)
@$(NORMAL_INSTALL)
@list='$(bin_PROGRAMS)'; test -n "$(bindir)" || list=; \
if test -n "$$list"; then \
echo " $(MKDIR_P) '$(DESTDIR)$(bindir)'"; \
$(MKDIR_P) "$(DESTDIR)$(bindir)" || exit 1; \
fi; \
for p in $$list; do echo "$$p $$p"; done | \
sed 's/$(EXEEXT)$$//' | \
while read p p1; do if test -f $$p \
; then echo "$$p"; echo "$$p"; else :; fi; \
done | \
sed -e 'p;s,.*/,,;n;h' \
-e 's|.*|.|' \
-e 'p;x;s,.*/,,;s/$(EXEEXT)$$//;$(transform);s/$$/$(EXEEXT)/' | \
sed 'N;N;N;s,\n, ,g' | \
$(AWK) 'BEGIN { files["."] = ""; dirs["."] = 1 } \
{ d=$$3; if (dirs[d] != 1) { print "d", d; dirs[d] = 1 } \
if ($$2 == $$4) files[d] = files[d] " " $$1; \
else { print "f", $$3 "/" $$4, $$1; } } \
END { for (d in files) print "f", d, files[d] }' | \
while read type dir files; do \
if test "$$dir" = .; then dir=; else dir=/$$dir; fi; \
test -z "$$files" || { \
echo " $(INSTALL_PROGRAM_ENV) $(INSTALL_PROGRAM) $$files '$(DESTDIR)$(bindir)$$dir'"; \
$(INSTALL_PROGRAM_ENV) $(INSTALL_PROGRAM) $$files "$(DESTDIR)$(bindir)$$dir" || exit $$?; \
} \
; done
...so I guess that when sudo make install is run, the function install-binPROGRAMS is run and I think it stores test_be in dest bin dir.
Since in Makefile, test_be is already specified as check_PROGRAMS, the only change should be to remove test_be from bin_PROGRAMS, right? If so, I can open a PR for it.
I tested and the change does not erroneously generate test_be in bin anymore. Here the PR: #434
Just for info, for this small PR will be a small update to a stable version like 2.0.4?
bulk_extractor_restarter.h:24:30: note: shadowed declaration is here
This will be fixed in an upcoming PR