Cannot find zlib.h.

Question

Cannot find zlib.h.

ArnieBerg opened this issue 5 years ago · 15 comments

When I run the 'make test' after a successful setup of the environment and pipeline, I get the following error:
(raseenv) analynano@analynano:/media/analynano/Data500GB/ArnieFiles/rase-pipeline$ prophyle decompress "database/spneumoniae-sparc.k18.tar.gz" database
[prophyle] 2019-10-15 16:15:26 Shell command: make -j -s -C /home/analynano/miniconda3/envs/raseenv/lib/python3.7/site-packages/prophyle
prophyle_assembler.cpp:43:10: fatal error: zlib.h: No such file or directory
#include <zlib.h>
^~~~~~~~
compilation terminated.
make[1]: *** [Makefile:13: prophyle_assembler.o] Error 1
make: *** [Makefile:14: compile_assembler] Error 2
[prophyle] 2019-10-15 16:15:26 Unfinished, an error occurred (error code 2, 0.8 MB used): make -j -s -C /home/analynano/miniconda3/e
nvs/raseenv/lib/python3.7/site-packages/prophyle
ProPhyle Error: A command failed, see messages above.

There is a zlib.h in /home/analynano/miniconda3/envs/raseenv/include. And it is referenced in $CFLAGS:

(raseenv) analynano@analynano:/media/analynano/Data500GB/ArnieFiles/rase-pipeline/rase$ echo $CFLAGS
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/analynano/miniconda3/envs/raseenv/include

What could I be missing?

Answer 1 · 2019-10-16T17:15:40.000Z

Thanks for reporting the problem. What operating system do you use?

Answer 2 · 2019-10-16T17:15:59.000Z

Ubuntu 18.04.

Answer 3 · 2019-10-17T15:34:34.000Z

It's weird that zlib is not installed on your system. Could you run sudo apt-get install libz-dev and try it again?

Answer 4 · 2019-10-17T17:33:33.000Z

It seems that zlib.h is installed on the system, but is not included in the search path for the make environment.

Notice that if it helps you can setup an environment with docker and replicate the issue by running 'docker build' with this file:
https://gist.github.com/sbuller/a7b769b9c6be7162949084d68daaece8

It looks like a problem with the prophyle Makefiles not working with conda environments.
Conda puts zlib.h in $ENV/include, but the compiler is built to expect the system includes under $ENV/x86_64-conda_cos6-linux-gnu. Conda seems to take care of this by setting conventional environment variables for the compiler. These environment variables are picked up by default make rules, but are not present in the rules in the Makefiles in $ENV/lib/python3.7/site-packages/prophyle/*. See https://www.gnu.org/software/make/manual/make.html#Implicit-Rules for more on implicit rules. In particular conda relies on CFLAGS and LDFLAGS being used, but this is just convention due to the aforementioned implicit rules.

Does that help point to a solution?

Answer 5 · 2019-10-17T18:48:49.000Z

Here is what fixed it for my environment. I modified the Makefiles to reflect the include location.

export PROPHYLE=/home/analynano/miniconda3/envs/raseenv/lib/python3.7/site-packages/prophyle

append LDFLAGS to compilation commands

sed -i 's/($(CC).$)/\1 $(LDFLAGS)/' $PROPHYLE/{,/}/Makefile
sed -i 's/($(CXX).$)/\1 $(LDFLAGS)/' $PROPHYLE/{,/}/Makefile

append to CFLAGS rather than overwriting

sed -i 's/CFLAGS=/CFLAGS+=/' $PROPHYLE/prophyle_index/{,bwa/}Makefile
sed -i 's/CXXFLAGS =/CXXFLAGS +=/' $PROPHYLE/prophyle_as*/Makefile

Answer 6 · 2019-10-17T20:56:50.000Z

Hi Arnie, thanks for the solution! I will keep the ticket open as other users might be experiencing the same and the issue needs to be fixed.

Answer 7 · 2019-10-31T16:15:28.000Z

Hi Karel, Now that I have Rase working well, I am faced with an operational issue. Your paper illustrates well how to use Rase with two sample bacteria, but how does one go about creating the input database for a different bacterium? Is there a best practice guide that explains the detailed steps? My background is coming from a long Computer Science career combined with a recent Bioinformatics degree. Thanks for your guidance on this, Arnie From: Karel Břinda <notifications@github.com> Sent: October 17, 2019 2:57 PM To: c2-d2/rase-pipeline <rase-pipeline@noreply.github.com> Cc: ArnieBerg <agb465@mail.usask.ca>; State change <state_change@noreply.github.com> Subject: Re: [c2-d2/rase-pipeline] Cannot find zlib.h. (#14) Hi Arnie, thanks for the solution! I will keep the ticket open as other users might be experiencing the same and the issue needs to be fixed. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#14?email_source=notifications&email_token=AFQ724J44KWTPB4S47Z6PUTQPDGRHA5CNFSM4JBOPNU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBRP4YQ#issuecomment-543358562> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQ724MQX5K7TZQBS25IDCTQPDGRHANCNFSM4JBOPNUQ> .

Answer 8 · 2021-04-06T07:07:56.000Z

Hi Karel and Arnie,

I receive the exact same error also running Ubuntu (20.04) in a VM.

Which of the Makefiles is it that has to be altered to fix the error?

Kind regards,
Morten

Answer 9 · 2021-04-06T17:28:21.000Z

I tried using Arnie's solution as I realized (at least that is what I think) it was supposed to replace all occurrences. However, I receive an error upon execution, can you help:

Thanks!

Answer 10 · 2021-04-09T08:19:43.000Z

@karel-brinda , @ArnieBerg
? :-)

Answer 11 · 2021-04-09T15:05:26.000Z

I think the best solution would be to edit the corresponding file manually (note that you will have to identify the directory of the source codes in your log messages, it no longer will be /home/analynano/miniconda3/envs/raseenv/lib/python3.7/site-packages/prophyle).

On most computers the current way of packaging ProPhyle works, but sometimes there are issues like this one. As ProphAsm (formerly called prophyle_assembler) now exists as a standalone program, future versions of ProPhyle will no longer be distributed with its source codes, but use ProphAsm as a dependency. But it will require changes in the architecture of ProPhyle.

Answer 12 · 2021-04-09T15:07:21.000Z

Concerning the sed error message, my best guess is that on your system there's a slightly different variant of sed with a different syntax (it might be a GNU vs. BSD issue like with many other Unix programs).

Answer 13 · 2021-04-22T07:44:52.000Z

Hi again,
I did locate the 4(?) files that are to be edited, however I am unsure what substitutions the sed command actually makes to the flags, and thus what manual substitutions I should conduct. Could you help me with what exact changes I need to make to these files?

Sincerely, Morten

Prophyle makefile

Prophyle/prophyle_index/bwa makefile

Prophyle/prophyle_assigment makefile

Prophyle/prophyle_assembler makefile

Answer 14 · 2021-04-22T09:05:53.000Z

Or maybe a hint to what version of sed will work. This is my installed version:

Answer 15 · 2021-04-23T17:58:58.000Z

I am sorry for the difficulties. Maybe try if some params like -E are not necessary: https://unix.stackexchange.com/questions/615977/invalid-reference-1-on-s-commands-rhs-error-for-sed-script. On some systems, these might be passed automatically (e.g., through aliasing).