d2iq-archive/spark-build

curl not working

Opened this issue · 2 comments

dirkjonker commented

With the latest release curl doesn't work.

docker run --rm -it mesosphere/spark:2.0.1-2.2.0-1-hadoop-2.7 /bin/bash
root@c91f29a62179:/opt/spark/dist# curl
curl: /opt/mesosphere/libmesos-bundle/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by curl)

This is because curl apparently follows LD_LIBRARY_PATH where there is a conflicting version of the curl library.

The fix I'm using is to run RUN LD_LIBRARY_PATH= curl ... when building an image on top of this one, but it feels a bit hacky.

skonto commented

@dirkjonker The other option is: LD_PRELOAD=libcurl.so.3 curl --help
Another option is to build curl and use that instead of installing it from the ubuntu repo (during image built):
https://gist.github.com/fideloper/f72997d2e2c9fbe66459
This worked fine for me. Also an upgrade: FROM ubuntu:16.04 did work for me, any reason we want to stick to 14.04?
@ArtRand @susanxhuynh thoughts?

  • 👍3
nathanloyer commented

This problem still exists in mesosphere/spark:2.3.0-2.2.1-2-hadoop-2.7. Adding ENV LD_PRELOAD=libcurl.so.3 to my own Dockerfile seems to fix it.