gopherdata/gophernotes

Error importing packages from docker build of gophernotes

cosmos72 opened this issue · 5 comments

cosmos72 commented

@snowzach reported the following in cosmos72/gomacro#109

I am not expert with docker, so I moved the issue here as it's about gophernotes + docker
(I tested on a plain Linux installation of latest gophernotes without docker, and I could not reproduce this issue)

Any suggestion / help is appreciated.

I used go 1.16.3... Here's the Dockerfile I am using to add gophernotes to jupyter.
It pulls gophernotes master (which I think is 1.0RC) which depends on github.com/cosmos72/gomacro v0.0.0-20210413180755-590dbc3d0bcd which looks very recent.
I also tried forcing module mode but that didn't seem to help.

FROM jupyter/datascience-notebook:latest

# Need to be root to do all this stuff
USER root

# Base image packages
RUN apt-get update \
    && sudo apt-get install -y \
    libpq-dev less \
    && rm -rf /var/lib/apt/lists/*

# Python packages
RUN pip install \
    psycopg2

# Install Go
RUN wget -O /tmp/go.tar.gz https://golang.org/dl/go1.16.3.linux-amd64.tar.gz \
    && tar -C /usr/local -xzf /tmp/go.tar.gz \
    && rm /tmp/go.tar.gz
ENV PATH="${PATH}:/usr/local/go/bin:$HOME/go/bin"
ENV GO111MODULE=on

# Install gophernotes
RUN cd /tmp && git clone https://github.com/gopherdata/gophernotes \
    && cd /tmp/gophernotes && GOPATH=/go go install \
    && mv /go/bin/gophernotes /usr/local/bin/gophernotes \
    && mkdir -p /opt/conda/share/jupyter/kernels/gophernotes \
    && cp kernel/* /opt/conda/share/jupyter/kernels/gophernotes \
    && chmod +w /opt/conda/share/jupyter/kernels/gophernotes/kernel.json \
    && chown -R jovyan:users /opt/conda/share/jupyter/kernels/gophernotes \
    && cd && rm -Rf /tmp/gophernotes && rm -Rf /go

# Switch back to jupyter default user
USER jovyan

When I try using Go it just says:

[I 14:48:07.597 NotebookApp] 302 GET / (172.17.0.1) 0.970000ms
[I 14:48:07.642 NotebookApp] 302 GET /tree? (172.17.0.1) 0.990000ms
[I 14:48:12.580 NotebookApp] 302 POST /login?next=%2Ftree%3F (172.17.0.1) 1.880000ms
[I 14:48:19.120 NotebookApp] Creating new notebook in
[I 14:48:19.208 NotebookApp] Writing notebook-signing key to /home/jovyan/.local/share/jupyter/notebook_secret
[I 14:48:25.080 NotebookApp] Kernel started: 7e12e118-ad92-40a3-87eb-60f398cd57c2, name: gophernotes
[I 14:48:25.209 NotebookApp] Adapting from protocol version 5.0 (kernel 7e12e118-ad92-40a3-87eb-60f398cd57c2) to 5.3 (client).
2021/04/16 14:48:27 Unhandled shell message:  comm_info_request
// debug: running "go get github.com/davecgh/go-spew/spew" ...
go: downloading github.com/davecgh/go-spew v1.1.1
go get: added github.com/davecgh/go-spew v1.1.1

And then the console shows
image

I did get it to work at one point when I got it to print compiling in that log output.. I never restarted it. It literealy didn't work several times, then I did something and it printed compiling in the log output and after that it seemed to work. Not sure what I did to get that compiling stage to work though.

snowzach commented

This might better be titled Having issues with third party packages and go1.16.3 Docker is just the way I chose to install it.

cosmos72 commented

Well, I could not reproduce this issue on the following setups:

  • latest gophernotes, compiled on Linux/amd64 with Go 1.16.1
  • latest gophernotes, compiled on Linux/amd64 with Go 1.16.3

hence my guess that it's somehow related to the docker installation procedure

snowzach commented

There is really nothing special about the docker installation procedure other than it starts with a clean slate. Would you be able to try with an empty GOPATH? Is it possible you're starting with a cache in place of some sort? Could it matter that this is using the Jupyter Notebook single-image?

cosmos72 commented

There is really nothing special about the docker installation procedure other than it starts with a clean slate.

That's what puzzles me.

The sequence to load packages in module mode with Go >= 1.16 is quite convoluted:

  1. create a dummy go.mod file in a new, empty directory
  2. echo the line // debug: running "go get github.com/davecgh/go-spew/spew" ...
  3. run go get pkg/full/path in the directory created at 1
  4. call golang.org/x/tools/go/packages/Load("pattern="+pkgpath)
  5. create a file pkgshortname.go in the directory created at 1. The file contains all package's symbols retrieved at 4
  6. echo the line // debug: running "go mod tidy" ...
  7. run go mod tidy in the directory created at 1
  8. echo the line // debug: compiling "pkg/full/path" ...
  9. run go build -buildmode=plugin in the directory created at 1
  10. call plugin/Open() and plugin/Plugin.Lookup() to load and use the plugin

your setup fails at step 4 because the echo at step 6 is not visible in your logs.
And golang.org/x/tools/go/packages/Load() is a big black box that, among other things, internally executes the go command.

butuzov commented

@cosmos72 I was able to run third party with no much hassle when using your gomacro 0.7.3 after the recent update. However, I didn't use gophernotes/gophernotes image, but rather mine. And all I change to make it working is to instal build-essentials (gcc/make/etc...).

Docekrfiles can be found here - https://github.com/butuzov/dots/tree/master/docker-images

image