Cannot connect to jupyter/ipython instance
Closed this issue · 122 comments
When attempting to run a jupyter notebook or qtconsole, I get error messages like this:
Invalid argument (bundled/zeromq/src/tcp_address.cpp:171)
The normal ipython prompt works fine.
Thanks for trying this out. Could you collect a strace for the failing command to what might be failing?
I can confirm this bug. You can find the strace here: https://gist.github.com/chrisfilo/2dd2c47592423291bf16db6348d26c6a
It seems that this bug is caused by jupyter/notebook#1331
Trying to get the same thing to run too, and with the same result. I'm on a Surface 3 4GB/128GB and happy to aid with testing/pinning things down. Jupyter is such an excellent tool and being able to run this on Bash on Windows would be amazing.
Same setup as @arthurhjorth here and also happy to help out. Loving bash on windows so far. Getting jupyter up and running would be the icing on the cake.
I came up with a workaround based on @MadcowD 's investigation on jupyter/notebook#1331 . I submitted it upstream as zeromq/libzmq#2013 , and posted an updated Ubuntu 14.04 package to a PPA in case anyone would like to try to use it.
Standard disclaimers -- I make no guarantees about either security or correctness; if it eats your data or steps on your cat I take no responsibility. But it seems to work for me.
Of course, getting a fix into WSL would be ideal. My cursory understanding of this code is that WSL would need to implement getifaddrs(), and would I think need to at minimum report on the existence of a loopback interface.
@aseering can you please explain how to use this fix? I can 'make install' for libzmq from repo source (including this fix) but how do I setup Jupyter to use the custom libzmq build? - it is still giving the same error that is quoted in the original post. Thank you!
@sgraphics are you familiar with Ubuntu PPAs? I would suggest using the PPA that I created that contains the fix. The simplest way to do so would be, I believe:
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3
That last command would normally install Ubuntu's libzmq; but because you've added my PPA containing the updated package, it will get the updated package instead.
If you want to inspect the source code, you're welcome to download it from the PPA's website (see the link above). You can also follow the first two steps above, then run apt-get source libzmq3, which will download the source code for the package to the current directory. Note that PPAs can contain updated versions of any package; you should only add PPAs that you trust.
Unfortunately, make install will not work in this case: The upstream version that contains the fix is also too new to work on Ubuntu 14.04. The PPA contains the same fix but applied to the source code used by the official Ubuntu 14.04 package. (I downloaded the Ubuntu source using apt-get source, then used quilt to apply the diff containing my fix, and uploaded the updated package.)
@aseering
I am getting an error 'Invalid argument' in line 190 of tcp_address.cpp even after updating the libzmq3 as you described above.
I couldn't build from source because it was asking for source URI when I was trying to download source.
I had the same issue as @pulinagrawal.
On Jul 13, 2016, at 11:22 AM, Pulin Agrawal notifications@github.com wrote:
I am getting an error 'Invalid argument' in line 190 of tcp_address.cpp even after updating the libzmq3 as you described above.
I couldn't build from source because it was asking for source URI when I was trying to download source.—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
@pulinagrawal @jzuhone -- hm... Could you post the output of dpkg -l libzmq3? It should indicate that you have version 4.0.4+dfsg-2wsl1 installed. (The wsl1 at the end is the key.)
I've posted a bug report about this issue to the Ubuntu libzmq3 bugtracker:
https://bugs.launchpad.net/ubuntu/+source/zeromq3/+bug/1602900
The fix is known; hopefully they will agree to backport it so that WSL users don't have to keep doing this dance to install packages from a PPA.
this fix doesn't work for my Surface 3...
I can still see the message
Invalid argument (bundled/zeromq/src/tcp_address.cpp:171)
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-========================-=================-=================-=====================================================
ii libzmq3:amd64 4.0.4+dfsg-2wsl1 amd64 lightweight messaging kernel (shared library)
@mrsiamen -- it sounds like you have the new package but it's not getting used for some reason. Is this a clean install of WSL? Did you try installing libzmq3 yourself before using the PPA?
Could you post the results of the following commands?:
ldd /usr/lib/python2.7/dist-packages/zmq/backend/cython/utils.x86_64-linux-gnu.so | grep zmq
md5sum /usr/lib/x86_64-linux-gnu/libzmq.so.3.1.0
For reference, on my machine, I see:
$ ldd /usr/lib/python2.7/dist-packages/zmq/backend/cython/utils.x86_64-linux-gnu.so | grep zmq
libzmq.so.3 => /usr/lib/x86_64-linux-gnu/libzmq.so.3 (0x00007f72cc2a0000)
$ md5sum /usr/lib/x86_64-linux-gnu/libzmq.so.3
87b97738b28a38a8e89796ecd41862b0 /usr/lib/x86_64-linux-gnu/libzmq.so.3.1.0
on dpkg on libzmq3 I do have wsl1 at the end
ldd gives
ldd: /usr/lib/python2.7/dist-packages/zmq/backend/cython/utils.x86_64-linux-gnu.so: No such file or directory
md5sum is the same as your output
@pulinagrawal -- Sounds like you're not using Ubuntu's iPython package? This fix only fixes Ubuntu's libzmq package. If you're using an iPython that pulls in libzmq from elsewhere, they will need to apply the fix as well.
To be specific, I'm testing as follows:
- Uninstall and fully remove any third-party ipython versions that you may have installed
- Run the command
sudo apt-get install ipython - Once ipython has successfully installed, run
ipython notebook --no-browser. It should print out a URL. Open a browser in Windows; navigate to that URL.
If you want to track this down further, you could run a command like
find /usr $HOME -type d -name zmq | grep -v IPython
to try to figure out the actual path to the Python zmq module that you're running. Then update the first command above with the appropriate path and re-run ldd; that should tell you where the third-party libzmq that you're using lives.
Another approach, just a guess: Are you getting ipython through Anaconda? I don't personally use Anaconda on Linux (I do on Mac/Windows) because it tends to cause this sort of confusion, and because there's less need for it -- apt-get install and pip install work well on their own.
@aseering Thanks for keep tracking on this:). looks like I'm facing the same issue as pulinagrawal does.
Will try to confirm the path.
I try to install the fixed libzmq3 before installing ipython notebook and it works now.
the kernel doesn't crash anymore. thank you guy for helping on this :)
libzmq.so.3 => /usr/lib/x86_64-linux-gnu/libzmq.so.3 (0x00007fb3722e0000)
87b97738b28a38a8e89796ecd41862b0 /usr/lib/x86_64-linux-gnu/libzmq.so.3.1.0
@aseering Could you possibly package this for conda??? unfortunately I have to use a python distribution that is through conda (specific not available packages). Or prehaps tell me how to link it?
Thank You for the help!! It is fixed now.
I uninstalled ipython and jupyter. I removed the zmq folder from dist-packages.
I installed libzmq3 then installed ipython. It did not run yet because I guess removing zmq removed other stuff as well. So, I installed python-zmq. Now it is running.
I might have broken other stuff by just deleting the zmq folder. So I guess better method would have been to use uninstall on python-zmq. But I did not know the name of the package then.
But it works for now.
@tanguita , sorry for the delay -- I believe Anaconda manages their own packages so they would have to incorporate this patch themselves. Feel free to go to Anaconda's bug tracker and file a bug there. If they have questions, I'd be happy to help them with the process.
If you want to get the source code for the patched libzmq, first run the commands above to get the updated package; then run
apt-get source libzmq
This will download the source package from my PPA. You can "cd" into it, read zmq's documentation, and see how to build it.
Alternatively, you can just install the binary package and take the pre-built libzmq that it installs, and try replacing Anaconda's libzmq with Ubuntu's. But that will only work if the two are the same or similar versions; I haven't tested it myself.
@aseering, This is not strictly true--one can create a package for Anaconda and upload it to one's own conda channel:
Then others can install it in their conda distribution.
This is what I thought but do not know how to do it..
On 21/07/16 11:25, John ZuHone wrote:
@aseering https://github.com/aseering, This is not strictly
true--one can create a package for Anaconda and upload it to one's own
conda channel:—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#185 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AID11O3GdlLali0cfqZHhW1dxThfSg9mks5qX49tgaJpZM4IG4pi.
Ah, thanks @jzuhone , I didn't realize people can create their own packages. I'll look into that, but I'm a bit busy these days and I'm not a big Anaconda user so I'd have a bunch to learn...
The actual code change to libzmq is quite small; you can see it here. If anyone here knows more about Anaconda packaging, feel free to apply that patch and add it to a channel.
@jzuhone Is this something you could do or point me on how? I have no idea where to start...
@tanguita and others:
I compiled zeromq with the patch as an anaconda package, you can get it by doing:
conda install -c jzuhone zeromq=dev
At best I will keep it updated infrequently, but this should hold everyone over until the stable releases have the fix hopefully.
Sorry, found some issues with the above. Try:
conda install -c jzuhone zeromq=4.1.dev0
instead
@aseering
I've actually done all of the above. ldd and md5sum are like yours. My problem is that ipython can't detect my python installation. I've installed ipython using sudo apt get install ipython ipython-notebook.
And there are also some modules that I had to install using pip.
So how do I solve no native kernel is detected issue?
@adonese -- it sounds like your Python package directory has gotten corrupted. You should not be needing to pip install anything to get iPython Notebook working.
Do you have any important state (configuration, files, etc) inside your Bash environment? The simplest solution, though it's a bit heavyweight, would be to simply uninstall it per the instructions here:
https://msdn.microsoft.com/en-us/commandline/wsl/faq
then reinstall it per the instructions here:
https://msdn.microsoft.com/en-us/commandline/wsl/install_guide
and follow the steps above again to install the libzmq fix and install iPython Notebook.
Thanks @aseering. I've finally got it to work. I had to uninstall WSL, the reinstalling it again. IPython is now fully working (It's IPython though, not Jupyter, but I guess the previous will work also on Jupyter).
You also have to install ipython notebook through sudo apt-get install ipython ipython-notebook.
It's not working for me.
I tried finding my zmq directory like this:
$ find /usr $HOME -type d -name zmq
/usr/lib/python2.7/dist-packages/IPython/kernel/zmq
/usr/local/lib/python2.7/dist-packages/zmq
There is no directory such as /usr/lib/python2.7/dist-packages/zmq
On the other hand:
$ ldd /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/utils.so
linux-vdso.so.1 => (0x00007fffc4f64000)
libzmq-feeff037.so.5.0.1 => /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/../../.libs/libzmq-feeff037.so.5.0.1 (0x00007f8073273000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8073060000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8072e30000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8072a60000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8072750000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8072430000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8072210000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8073800000)
md5sum for this is:
$ md5sum /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/../../.libs/libzmq-feeff037.so.5.0.1
cdcd394e58afa01a7c9981ac9cc90c97 /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/../../.libs/libzmq-feeff037.so.5.0.1
But I do have the correct version of libzmq3:
$ dpkg -l libzmq3
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-====================================-=======================-=======================-==============================================================================
ii libzmq3:amd64 4.0.4+dfsg-2wsl1 amd64 lightweight messaging kernel (shared library)
Can anyone help me understand what's going on?
@yasirs -- you are not using the Ubuntu version of python-zmq. How did you install it?
@aseering I think I may have installed it using pip. In any case, this is what I did:
$ sudo apt-get install python-zmq
$ sudo -H pip uninstall pyzmq
This solved the problem. Thanks!
@aseering Thanks for the fix, your changes worked perfectly on a new WSL install. Installed pip, pyhon-dev, scipy stack, ipython, ipython-notebook and then pip install jupyter. Works great!
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3
Seems like this doesn't work with Python 2.7.12 built from source and linked in via update-alternatives. After sudo apt-get install ipython ipython-notebook, running ipython notebook results in
Traceback (most recent call last):
File "/usr/bin/ipython", line 5, in <module>
from IPython import start_ipython
ImportError: No module named IPython
Installing IPython using the pip bundled with that Python distro results in the ipython notebook command working, but then the zeromq error comes up.
@RitwikGupta Here's what I did to have it working with the pip version of jupyter.
pip uninstall pyzmq
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3 libzmq3-dev
export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
pip install --no-use-wheel -v pyzmq
pip install jupyter
@gglanzani I tried your method, but this is what I get.
`
me@LAPTOP-C55:/mnt/c/Windows/System32$ jupyter notebook
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 7, in
from notebook.notebookapp import main
File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 31, in
from zmq.eventloop import ioloop
ImportError: No module named zmq.eventloop
`
I tried sudo apt-get install python-zmq instead and it worked now.
@gglanzani Your fix worked! Though, running jupyter results in zsh: command not found: jupyter, but that may be due to my local pip installation. ipython notebook --no-browser works perfectly.
import sys
print sys.version_info
>>> sys.version_info(major=2, minor=7, micro=12, releaselevel='final', serial=0)@gglanzani This works. Thanks a bunch mate! Now onto getting tensorflow to talk to my sweet GTX960M GPU.
Thank you so much for the help, everybody. Jupyter is working on my computer now, running on a python3 kernel.
It took me a while to piece together all the individual pieces of advice/directions, because I'm a Linux/apt-get newbie.. So just in case anyone else is confused, hopefully I can help save some time with these instructions:
- Clean install.
- This:
sudo apt-get update
sudo apt-get install python-pip ipython ipython-notebook ipython3 ipython3-notebook
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3
sudo apt-get install python3-pip
sudo pip3 install jupyter
sudo pip install jupyter
Will get it up and running. Some of this may be redundant - I don't know. But it made it work, so figured I would share.
All, I followed the instructions above and unfortunately things did not work for me.
[W 12:02:11.358 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20160822120145 (127.0.0.1) 8.01ms referer=http://localhost:8888/notebooks/Untitled.ipynb?kernel_name=python2
[I 12:02:11.613 NotebookApp] Kernel started: 75e0b050-5e54-4499-86f1-8db750790279
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:14.615 NotebookApp] KernelRestarter: restarting kernel (1/5)
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:17.641 NotebookApp] KernelRestarter: restarting kernel (2/5)
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[W 12:02:21.647 NotebookApp] Timeout waiting for kernel_info reply from 75e0b050-5e54-4499-86f1-8db750790279
[I 12:02:23.664 NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel 75e0b050-5e54-4499-86f1-8db750790279 restarted
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:26.690 NotebookApp] KernelRestarter: restarting kernel (2/5)
WARNING:root:kernel 75e0b050-5e54-4499-86f1-8db750790279 restarted
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:29.727 NotebookApp] KernelRestarter: restarting kernel (3/5)
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
Very disappointing. Not sure if the issue lies on the Ubuntu side or the windows side. I don't think Jupyter is directly involved in that issue.
I want to report that the only way to get this to work was through a clean install. ICYMI:
In a good old windows cmd.exe prompt type: lxrun /uninstall /all It will raise a warning letting you that all data (including your home dir) will be wiped.
After that process is done (it should take 2-3 minutes), type: lxrun /install to re-install the ubuntu subsystem from scratch.
Once that's done here's what I did:
1 sudo apt-get update
2 sudo add-apt-repository ppa:aseering/wsl
3 sudo apt-get update
4 sudo apt-get install python-pip ipython ipython-notebook
5 ipython notebook --no-browser
It now works flawlessly, but with the old iPython UI (which I'm totally OK with given it works)
Hi guys,
This thread has been very useful for me to set up Jupyter kernel and stuff.
I used Windows 10 bash to install it and tensorflow. I run Jupyter in bash and open Windows browser as http://localhost:8888. Jupyter works and my files are there. My intention is to right a tensorflow code and execute it but I get the following errors:
ImportError Traceback (most recent call last)
in ()
----> 1 import tensorflow as tf
2 import numpy as np
ImportError: No module named tensorflow
What can be the possible reason?
@allblues1001 can you run python in a bash prompt, then when you get the python REPL prompt >>> type import tensorflow and let us know the result of that is?
@rdodev Thanks for the response. Yes, I am able to run python and import tensorflow in the bash prompt. Absolutely no problem in that. It is only when I use Windows browser (Google Chrome), it fails to import tensorflow, numpy etc. Interestingly, the python commands are working fine like adding two numbers etc. It is only when I try to import some library like tensorflow and numpy, I get these errors.
Can there be some communication problem?
Did you install TensorFlow via pip?
Could you post the result of python --version and pip --version?
On Wed, Aug 24, 2016 at 9:01 AM -0400, "allblues1001" <notifications@github.commailto:notifications@github.com> wrote:
@rdodevhttps://github.com/rdodev Thanks for the response. Yes, I am able to run python and import tensorflow in the bash prompt. Absolutely no problem in that. It is only when I use Windows browser (Google Chrome), it fails to import tensorflow, numpy etc. Interestingly, the python commands are working fine like adding two numbers etc. It is only when I try to import some library like tensorflow and numpy, I get these errors.
Can there be some communication problem?
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//issues/185#issuecomment-242054527, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEurpdNlSNKjKTTqy95y2BASscw6s3PJks5qjEChgaJpZM4IG4pi.
@RitwikGupta : Yes, I used pip to install Tensorflow. The results are:
Python 2.7.6 and pip 1.5.4
Could you please also list the complete path of the installation of Python and pip? Just confirming that you're not using a version of pip provided by a separate Python installation and then launching ipython notebook provided by apt.
On Wed, Aug 24, 2016 at 9:15 AM -0400, "allblues1001" <notifications@github.commailto:notifications@github.com> wrote:
@RitwikGuptahttps://github.com/RitwikGupta : Yes, I used pip to install Tensorflow. The results are:
Python 2.7.6 and pip 1.5.4
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//issues/185#issuecomment-242058810, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEurpcxvr_fulTt1ABh2dcis29rFqUrLks5qjEPKgaJpZM4IG4pi.
Sure, it says:
pip 1.5.4 from /home/allblues1001/tensorflow/local/python2.7/site-packages (python 2.7)
@allblues1001 and forgive me if this is obvious, but the kernel you're starting in ipython is the same version as the one in bash? try this on both the python REPL and the ipython notebook:
import sys
print sys.version_info
and compare.
@rdodev For python REPL and jupyter notebook it is the same:
sys.version_info (major=2, minor=7, micro=6, releaselevel='final', serial=0)
For python:
['', '/home/allblues/tensorflow/lib/python2.7', '/home/saad-omar/tensorflow/lib/python2.7/plat-x86_64-linux-gnu', '/home/allblues/tensorflow/lib/python2.7/lib-tk', '/home/saad-omar/tensorflow/lib/python2.7/lib-old', '/home/allblues/tensorflow/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/home/allblues/tensorflow/local/lib/python2.7/site-packages', '/home/allblues/tensorflow/lib/python2.7/site-packages', '/usr/local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7']
For jupyter notebook:
['',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/PILcompat',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7',
'/usr/local/lib/python2.7/dist-packages/IPython/extensions',
'/home/allblues/.ipython']
@allblues1001 you left a few 'saad-omar' there :p That said, you have an issue. Both paths should look more or less the same. It would seem tensorflow installed its own version of python? Either way, your issue is not related to Bash on Windows or ipython. I would suggest seeking help from the folks at tensorflow.
thank you!
@RitwikGupta ipython notebook --no-browser says 0 active kernel. I opened http://localhost:8888/tree# in my browser and then when I open a new notebook (python[root]) it says connecting to kernel and closes automatically. Can you please let me know what is the possible solution for this?
Thanks!
Anand Bhattad
this is what I get
Check your console and ensure there are no errors first. If getting the socket error, follow the steps above.
On Wed, Aug 31, 2016 at 11:37 PM -0400, "bhattad2" <notifications@github.commailto:notifications@github.com> wrote:
@RitwikGuptahttps://github.com/RitwikGupta ipython notebook --no-browser says 0 active kernel. I opened http://localhost:8888/tree# in my browser and then when I open a new notebook (python[root]) it says connecting to kernel and closes automatically. Can you please let me know what is the possible solution for this?
Thanks!
Anand Bhattad
this is what I get
[image]https://cloud.githubusercontent.com/assets/16907734/18154563/5a082604-6fcb-11e6-8ab8-1a488becb7cf.png
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//issues/185#issuecomment-243967211, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEurpbB0-cTP5_8ULFspO1E6Ytd5OOURks5qlkhMgaJpZM4IG4pi.
There are no console errors. And I'm able to use ipython from the terminal and I am able to import tensorflow, numpy without any error
This seems like an iPython issue rather than BashOnWindows, so I would suggest taking this issue to that repo, but I think the error is in your kernel config. Is python[root] your kernel? It should be called “Python 2” or “Python 3” for a fresh install.
@aseering Please clarify few things about the solution:
- does it work only for Python 2.7 or does it cover Python 3.x?
- you keep referring to IPython rather than Jupyter - does this mean Jupyter is not covered?
- how should we integrate the solution in case of virtualenv or anaconda environments?
Thank you
Also, shouldn't Microsoft be moving towards an actual solution to this
rather than just @aseering's workaround?
On Thu, Sep 1, 2016 at 5:11 PM, bobev18 notifications@github.com wrote:
@aseering https://github.com/aseering Please clarify few things about
the solution:
- does it work only for Python 2.7 or does it cover Python 3.x?
- you keep referring to IPython rather than Jupyter - does this mean
Jupyter is not covered?- how should we integrate the solution in case of virtualenv or
anaconda environments?Thank you
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#185 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYcmsyDi5NKMFW7XuLE6fuq5Pb3ckW0ks5qlz95gaJpZM4IG4pi
.
@bobev18 , maybe this helps to understand the issue: This bug is not a bug in Python. IPython (and Jupyter) Notebook are not pure Python; they contain some native code as well, and that native code links against some third-party native systemwide C libraries. The problem is a bug in one of those libraries, "libzmq". All versions of Notebook link libzmq. Because it's an independent system library, it doesn't matter what version of iPython/Jupyter you're using, nor what version of Python.
But which libzmq do they link? The iPython that comes with Ubuntu uses the libzmq that comes with Ubuntu. I believe pip will try to use the system libzmq as well. My updated patch fixes Ubuntu's version of the library. Anaconda bundles their own copy of libzmq, so you have to use @jzuhone 's solution to get their fix.
@yasirs -- for what it's worth, this is actually a bug in Ubuntu's package. The bug has been acknowledged and fixed upstream. It happens to be much more disruptive on WSL, but it's a bug either way. My "workaround" is simply to backport the fix to Ubuntu 14.04.
Sure, Microsoft should make their code more robust to buggy code. I hope they've gotten that memo :-) But this is a known bug whose fix has been submitted to Ubuntu. They could accept the patch today and ... well, sure, some important edge cases, but it would help a lot of people. So, could some of the enthusiastic folks here go upvote / encourage them to do that?:
https://bugs.launchpad.net/ubuntu/+source/zeromq3/+bug/1602900
@aseering Do you mind sharing latest instructions for clean install please? The thread is quite long, and it's not clear if things are changed since your last instructions post
This worked for me a few weeks ago from a fresh install:
Sudo apt-get update
Sudo apt-get install python-pip ipython ipython-notebook ipython3
ipython3-notebook
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3
sudo apt-get install python3-pip
sudo pip3 install jupyter
sudo pip install jupyter
jupyter notebook --no-browser
No reason to believe it wouldn't work now. But it would be great with
confirmation if you decide to try this out.
On Tue, Sep 20, 2016, 5:43 PM fedorzh notifications@github.com wrote:
@aseering https://github.com/aseering Do you mind sharing latest
instructions for clean install please? The thread is quite long, and it's
not clear if things are changed since your last instructions post—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#185 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABxk66QF7rV31K3PrktrGIdNJDldVELrks5qsGGcgaJpZM4IG4pi
.
(Email client on phone capitalized s couple of sudo there, sorry. Just
lowercase the S.)
On Tue, Sep 20, 2016, 5:50 PM Arthur Hjorth arthur.hjorth@stx.oxon.org
wrote:
This worked for me a few weeks ago from a fresh install:
Sudo apt-get update
Sudo apt-get install python-pip ipython ipython-notebook ipython3
ipython3-notebooksudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3sudo apt-get install python3-pip
sudo pip3 install jupyter
sudo pip install jupyter
jupyter notebook --no-browserNo reason to believe it wouldn't work now. But it would be great with
confirmation if you decide to try this out.On Tue, Sep 20, 2016, 5:43 PM fedorzh notifications@github.com wrote:
@aseering https://github.com/aseering Do you mind sharing latest
instructions for clean install please? The thread is quite long, and it's
not clear if things are changed since your last instructions post—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#185 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABxk66QF7rV31K3PrktrGIdNJDldVELrks5qsGGcgaJpZM4IG4pi
.
@arthurhjorth Thanks! I am actually trying to install with python 2, as I want to use tensorflow and gym, and they don't support 3 as well as 2 yet.
I have tried
apt-get install python-pip ipython ipython-notebook
add-apt-repository ppa:aseering/wsl
apt-get update
apt-get install libzmq3
pip install jupyter
jupyter notebook --no-browser
(no sudos didn't give any errors during installation, and I wanted to do anything as myself to avoid permission problems in the future) but that didn't work (as in still the same error "Invalid argument (bundled/zeromq/src/tcp_address.cpp:190")
Using @rdodev 's advice exactly (without sudo as I logged in as root) #185 (comment), with full reinstall of Ubuntu, ipython notebook worked!
Further pip install jupyter worked for running jupyter instead.
After upgrading to 14936, jupyter notebook no longer works on my machine. If you're in this thread, you probably care about that, so don't upgrade unless you have to.
Reverting back fixed it.
On the new build, I got an error about unsupported operation at tcp_address.cpp:169 and the kernel kept restarting until it had tried five times, and then it gave up.
I am using my computer to do a presentation including some coding, so I can't help with debugging atm or offer any other information.
FWIW, I'm seeing the exact same thing as @arthurhjorth with python and jupyter installed from conda
@arthurhjorth +1
In 14936,
[I 23:55:53.388 NotebookApp] Kernel started: 99311cb2-a24a-4f51-983b-7187e3fe940d
Operation not supported (tcp_address.cpp:169)
[I 23:55:56.383 NotebookApp] KernelRestarter: restarting kernel (1/5)
Operation not supported (tcp_address.cpp:169)
[I 23:55:59.465 NotebookApp] KernelRestarter: restarting kernel (2/5)
Operation not supported (tcp_address.cpp:169)
[I 23:56:02.523 NotebookApp] KernelRestarter: restarting kernel (3/5)
^C[I 23:56:03.049 NotebookApp] interrupted
Serving notebooks from local directory: /root
1 active kernels
The Jupyter Notebook is running at: http://localhost:8192/
Shutdown this notebook server (y/[n])? ^C[C 23:56:03.195 NotebookApp] recei
Fix:
In 14936, kernel use EOPNOTSUPP instead of EINVAL.
So, after using https://launchpadlibrarian.net/261891465/zeromq3_4.0.4+dfsg-2_4.0.4+dfsg-2wsl1.diff.gz this patch, EOPNOTSUPP is not ignored.
Line 161 should be if (rc != 0 && (errno == EINVAL || errno == EOPNOTSUPP))
and it should works.
PS: you may use this libzmq3 (if you trust me), or you can modify that line and build the package
Hint: If you are building & packing this library under WSL, you should skip all tests (some tests may not pass under WSL)
@Yangff Thanks for the information! Very insightful.
I'm less interested in getting this running now than I am in poking the right people (e.g., jupyter, zeromq) so that they are aware of the issue and can support WSL
Ideally people shouldn't have to make changes to support WSL. It's more a matter of us making sure to return the right error code in this condition. I'll take a look at this tomorrow and see what the problem is.
@benhillis Thanks for making sure this issue is taken care of 😄
Thanks! Can @aseering update his branch with this patch, so people can use
the the same instructions to use the patched package?
On Sun, Oct 2, 2016 at 7:57 PM, Ritwik Gupta notifications@github.com
wrote:
@benhillis https://github.com/benhillis Thanks for making sure this
issue is taken care of 😄—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#185 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYcmpleOGQ48MJFtbvLRHzD8F8yZEy_ks5qwEUGgaJpZM4IG4pi
.
@Yangff thanks so much! I have too much work this week to dare do anything other than just stick with the old build for now. Unless there's a WSL fix in sight by some time after next week, I'll use your libzmq. Thank you!!
I'm having trouble reproducing this issue. Could somebody give me a list of steps / instructions? Here's what I did:
apt-get update
apt-get install python-pip ipython ipython-notebook libzmq3
pip install jupyter
jupyter notebook --no-browser
I then launched a browser and navigated to the URL that was spit out by the jupyter command.
@benhillis can you then start a notebook and execute python?
IME, launching the server seems to work fine. The problems arise when you create a new notebook and try to execute even the most basic code. For example:
import os
print(os.getcwd())
Should trigger a restart to the kernel (if it let's you get that far).
On the site New, then Terminal and ran python inside of that. Is that what you mean?
Hm, it may be that the Ubuntu trunk libzmq may be working properly now with WSL. @benhillis, can you try the steps listed on #185 (comment) and see if that breaks it? If so, the libzmq on @aseering's PPA might be borked.
EDIT: Disregard, I misidentified the issue. See @phobson's comment below for repro.
You need to make a new notebook, not a terminal:
Then you get stuck in a loop of dead/restarting kernels:
Console output looks like this:
(jup) paul@POR-PHOBSON10:/mnt/c/Users/phobson$ jupyter notebook --no-browser
[W 11:13:39.836 NotebookApp] Widgets are unavailable. Please install widgetsnbextension or ipywidgets 4.0
[I 11:13:39.851 NotebookApp] Serving notebooks from local directory: /mnt/c/Users/phobson
[I 11:13:39.852 NotebookApp] 0 active kernels
[I 11:13:39.852 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/
[I 11:13:39.852 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 11:14:20.160 NotebookApp] 302 GET / (127.0.0.1) 4.28ms
[I 11:15:47.918 NotebookApp] Creating new notebook in
[W 11:15:48.907 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20161003111339 (127.0.0.1) 14.75ms referer=http://127.0.0.1:8888/notebooks/Untitled.ipynb?kernel_name=python3
[I 11:15:49.166 NotebookApp] Kernel started: 03a01c40-5d09-444d-a173-bf97447956bd
Invalid argument (src/tcp_address.cpp:172)
[I 11:15:52.165 NotebookApp] KernelRestarter: restarting kernel (1/5)
Invalid argument (src/tcp_address.cpp:172)
[I 11:15:55.232 NotebookApp] KernelRestarter: restarting kernel (2/5)
Invalid argument (src/tcp_address.cpp:172)
[I 11:15:58.295 NotebookApp] KernelRestarter: restarting kernel (3/5)
[W 11:15:59.202 NotebookApp] Timeout waiting for kernel_info reply from 03a01c40-5d09-444d-a173-bf97447956bd
Invalid argument (src/tcp_address.cpp:172)
[I 11:16:01.380 NotebookApp] KernelRestarter: restarting kernel (4/5)
WARNING:root:kernel 03a01c40-5d09-444d-a173-bf97447956bd restarted
Invalid argument (src/tcp_address.cpp:172)
[W 11:16:04.456 NotebookApp] KernelRestarter: restart failed
[W 11:16:04.458 NotebookApp] Kernel 03a01c40-5d09-444d-a173-bf97447956bd died, removing from map.
ERROR:root:kernel 03a01c40-5d09-444d-a173-bf97447956bd restarted failed!
[W 11:16:04.504 NotebookApp] Kernel deleted before session
[W 11:16:04.509 NotebookApp] 410 DELETE /api/sessions/84ce5660-b868-4824-805b-200a40c62596 (127.0.0.1) 12.75ms referer=http://127.0.0.1:8888/notebooks/Untitled.ipynb?kernel_name=python3
[W 11:16:22.342 NotebookApp] Session not found: session_id='84ce5660-b868-4824-805b-200a40c62596'
[W 11:16:22.342 NotebookApp] 404 DELETE /api/sessions/84ce5660-b868-4824-805b-200a40c62596 (127.0.0.1) 2.19ms referer=http://127.0.0.1:8888/notebooks/Untitled.ipynb?kernel_name=python3
[I 11:16:22.398 NotebookApp] Kernel started: 1da56c55-9360-4f85-89b1-76da21623cd6
Invalid argument (src/tcp_address.cpp:172)
[I 11:16:25.397 NotebookApp] KernelRestarter: restarting kernel (1/5)
Invalid argument (src/tcp_address.cpp:172)
[I 11:16:28.432 NotebookApp] KernelRestarter: restarting kernel (2/5)
Invalid argument (src/tcp_address.cpp:172)
[I 11:16:31.489 NotebookApp] KernelRestarter: restarting kernel (3/5)
Invalid argument (src/tcp_address.cpp:172)
[W 11:16:32.421 NotebookApp] Timeout waiting for kernel_info reply from 1da56c55-9360-4f85-89b1-76da21623cd6
[I 11:16:34.568 NotebookApp] KernelRestarter: restarting kernel (4/5)
WARNING:root:kernel 1da56c55-9360-4f85-89b1-76da21623cd6 restarted
Invalid argument (src/tcp_address.cpp:172)
[W 11:16:37.647 NotebookApp] KernelRestarter: restart failed
[W 11:16:37.649 NotebookApp] Kernel 1da56c55-9360-4f85-89b1-76da21623cd6 died, removing from map.
ERROR:root:kernel 1da56c55-9360-4f85-89b1-76da21623cd6 restarted failed!
[W 11:16:37.691 NotebookApp] Kernel deleted before session
[W 11:16:37.694 NotebookApp] 410 DELETE /api/sessions/93443e32-9137-48b4-ba40-2ee6c963ca34 (127.0.0.1) 7.89ms referer=http://127.0.0.1:8888/notebooks/Untitled.ipynb?kernel_name=python3
ipython kernel keeps dying (bash on ubuntu on windows)
I follow this step, yet the kernel keeps dying
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3 libzmq3-dev
export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
pip install --no-use-wheel -v pyzmq
pip install jupyter
Kernel started: c81deaed-dfc4-4733-8ba7-173dc82d646c
Invalid argument (src/tcp_address.cpp:190)
KernelRestarter: restarting kernel (1/5)
Invalid argument (src/tcp_address.cpp:190)
KernelRestarter: restarting kernel (2/5)
Invalid argument (src/tcp_address.cpp:190)
KernelRestarter: restarting kernel (3/5)
Invalid argument (src/tcp_address.cpp:190)
Timeout waiting for kernel_info reply from c81deaed-dfc4-4733-8ba7-173dc82d646c
KernelRestarter: restarting kernel (4/5)
WARNING:root:kernel c81deaed-dfc4-4733-8ba7-173dc82d646c restarted
Invalid argument (src/tcp_address.cpp:190)
![applicationframehost_2016-10-18_18-37-29]
KernelRestarter: restart failed
Kernel c81deaed-dfc4-4733-8ba7-173dc82d646c died, removing from map.
ERROR:root:kernel c81deaed-dfc4-4733-8ba7-173dc82d646c restarted failed!
Kernel deleted before session
410 DELETE /api/sessions/21a9b67b-1a2c-45cd-8df7-41885ae62f2e (127.0.0.1) 4.49ms referer=http://localhost:8888/notebooks/a.ipynb
By the way, if you are mostly using bash to run TensorFlow there is now an alpha Windows support to run it natively on CPU only now
Most people still have problem getting the CPU only version to work :-(
@JimSEOW The native TensorFlow windows support is an alpha as you've read I mentioned, so you are expected to face some instability although it's made progress since then and seems to be working fine.
For more information about it check their thread here.
There is also a build now running with bazel. I have it running perfectly natively.
Now regarding bash and ipython, did you try to install on a clean bash installation?
In @benhillis case:
Invalid argument (src/tcp_address.cpp:172)
In my case:
Invalid argument (src/tcp_address.cpp:190)
Is this related to Bash on Windows? Any suggestion?
@Carmezim
Yes, it is a clean bash installation. Do you have stable Juypter running on bash on Window? If so, I would try again, before moving to Hyper-V Ubuntu.
@JimSEOW Yes, I had problems to get it working at first but after a fresh installation and following some instructions above it worked normally.
@Yangff (or others that can help): I don't know what to do with the modified libzmq3 files you uploaded. I tried installing them using sudo dpkg -i filename and then sudo apt-get install -f afterwards, but it didn't seem to work.
How should this library get installed?
For what it's worth, the latest WSL Insider build contains a change that might work around this issue. I'm not at a Windows machine right now but I would be interested to know if the stock libzmq does indeed work now.
@aseering forgive my ignorance. how does one update to the latest build? (I presume it takes more than just sudo apt-get update && sudo apt-get upgrade --yes)
@phobson -- I'm guessing based on your question that you're running the standard stable version of Windows, and are not part of the Insider program? You can learn more about the program here:
It's easy (and free) to join. Once you join, updates will automatically appear through Windows Update. But you can't selectively update (you can't just upgrade WSL, for example); you're opting in to receive a pre-release version of all of Windows. This new version contains new features; also new bugs :-)
Features added to the Insider releases eventually make it into the next major Windows release. So, if you don't choose to join the Insider program, you'll get the new features when the next major Windows version ships, probably some number of months from now.
It works for me, but I need to run all pip install or pip3 install with sudo -H flag so that everything installs in the home directory. I do not use the -H flag when installing things with apt-get. I don't know enough about all this stuff to make sense of why, but doing that makes everything run "out of the box" for me!
(If anyone understands why this would work and can spare the time to explain it or point to a good resource on it, I would love to learn. I've googled around and read stuff online but still can't put together a coherent explanation in my head.)
Hi @arthurhjorth -- what arguments are you running pip with?
sudo -H will not, by itself, cause things to be installed to your home directory. sudo (by default) runs commands as the root user rather than as your normal user account; root has elevated privileges to install things systemwide. sudo -H says, even though you're running as the root user, set the $HOME environment variable to point at your regular user's home directory. This affects programs that try to write to "the current user's home directory".
apt-get always installs software systemwide. (It's the Ubuntu package manager; it installs packages for your Ubuntu system.) Therefore, by default it must be run as root, and the value of the $HOME environment variable (and therefore the -H flag to sudo) doesn't affect it.
pip (and pip3) can install either systemwide or just for your current user account. By default, it decides where to install based on whether you run pip with the --user flag: With that flag, it will install to your home directory; without that flag, it will install systemwide. If you install systemwide, by default it will need to run as root. If you install to your home directory, you should not need to install as root -- this is very useful for users on shared machines who don't have sudo access.
There are a few weird edge cases to consider:
- If you run
sudo pip install --user <some package>:sudomeans "run the following command asroot".pip install --usermeans "install to the current user's home directory. But the current user isroot. So this will install the package toroot's home directory, and it will only be available if you run asroot. Which is probably not useful, unless you really know what you are doing. - If you run
sudo -H pip install --user <some package>: This is the same as above, except, you're tellingpipthatroot's home directory is your current directory. That's actually bad: It means thatpipwill install the package to your home directory, but your user's package repository will be corrupted / updated to be owned byroot. By default, this means that you can run packages, but the next time you try to install usingpipwithoutsudo, you won't be allowed to do so. You can fix this by runningsudo chown -R $USER ~/.local. - What happens if you install via both
apt-getandpip? Some packages will conflict. In this case,apt-get, which is a general-purpose package manager and has only a minimal understanding of Python in particular, will simply error out and say (I'm paraphrasing here) "go fix your package!"pipis a little more clever; it will try to resolve the dependency in favor of whatever version it is installing. However, most packages won't conflict. This is because Python has a "search path": When you go toimporta package in a Python script, first, Python checks for packages installed to your user account's home directory (wherepip install --userputs them):$HOME/.local/lib/python2.7. If it doesn't find a package there, next it looks in/usr/local/lib/python2.7(or similar for Python 3), which is wheresudo pip installputs them. If it still doesn't find the package, then it looks in/usr/lib/python2.7, which is whereapt-getinstalls its packages. If it still doesn't find the package, only then does it error out. So if you install a package multiple times using different tools or commands, Python will pick whichever it sees first. This can cause a lot of confusion -- for example, if you install usingpip, then you decide you want the official Ubuntu version so you install usingapt-get, you'll keep getting thepipversion because it's first in the search path. Also, if you have a broken version of a package early in the search path that won't uninstall for whatever reason, installing a correct package may not fix it if the correct one is installed later in the search path.
All of the above assumes that you're using the standard version of Python packaged for Ubuntu Linux (which is what WSL uses). If you're running on CentOS, or Mac, or on Ubuntu using an alternative version of Python such as Anaconda or Linuxbrew, or if you compile and install Python yourself from source, then at minimum the number and location of paths in Python's search path will change; also, some of those installations will reconfigure your system to not require root access to install Python packages systemwide.