IPython development version compatibility issues

Question

IPython development version compatibility issues

Closed this issue 10 years ago · 53 comments

bpipev commented 11 years ago

The most recent development version of IPython breaks nbdiff/merge in several ways.

bpipev commented 11 years ago

It works!

Answer 1 · 2014-02-18T02:14:54.000Z

This is what I have in console:

failed to load resource: the server responded with a status of 404 (NOT FOUND)
http://127.0.0.1:5000/api/notebooks?_=1392683124695

Selena and I already knew about these errors but didn't file issues for them. Typically they don't actually interfere with the function of nbmerge. Are they actually causing the diffs not to load on your machine?

Answer 2 · 2014-02-18T02:43:03.000Z

I don't know if that's the cause, but I don't see the merge page.

Answer 3 · 2014-02-18T02:46:56.000Z

Hmm. Can you provide steps to reproduce? I'm running the make_merge_conflict.py script and attempting to resolve the conflict and it works for me :(

Answer 4 · 2014-02-18T03:18:08.000Z

How did you install nbdiff? Did you use python setup.py install or python setup.py develop?

Answer 5 · 2014-02-18T04:15:39.000Z

I run the make_merge_conflict.py
I go to folder
I type nbmerge
that's it.

I installed nbdiff using python setup.py develop

Answer 6 · 2014-02-18T04:16:10.000Z

sudo python setup.py develop

Answer 7 · 2014-02-18T04:18:48.000Z

bpipev commented 11 years ago

Answer 8 · 2014-02-18T04:20:07.000Z

Do you have the latest master?

If you open the chrome debugger and look in the "networks" tab, do you see an entry with a url resembling:

localhost:5000/notebooks/test_notebook

?

Answer 9 · 2014-02-18T04:22:22.000Z

Just to be clear, is this on your vcs_adapter branch or on master?

Answer 10 · 2014-02-18T04:23:49.000Z

Neither works

Answer 11 · 2014-02-18T04:28:49.000Z

And no I don't see that

Answer 12 · 2014-02-18T04:31:52.000Z

Can you post the output of nbmerge in the terminal? (Not the chrome console)

Here's an excerpt from mine:

127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /static/notebook/js/config.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /static/notebook/js/main.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /static/notebook/js/contexthint.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /static/notebook/js/celltoolbarpresets/default.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /static/notebook/js/celltoolbarpresets/slideshow.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /nbdiff/nbdiff.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /static/custom/custom.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /static/components/marked/lib/marked.js HTTP/1.1" 304 -
127.0.0.1 - - [17/Feb/2014 23:29:42] "GET /notebooks/test_notebook?_=1392697782328 HTTP/1.1" 200

You should see GET /notebooks/test_notebook somewhere in there, and if its status isn't 200 that might help.

Answer 13 · 2014-02-18T04:34:47.000Z

Request URL:http://127.0.0.1:5000/api/notebooks?_=1392697946766
Request Method:GET
Status Code:404 NOT FOUND

Answer 14 · 2014-02-18T04:42:12.000Z

the /api/ url is supposed to fail. Look for /notebooks/test_notebook.

Answer 15 · 2014-02-18T04:46:11.000Z

Can't find it

Answer 16 · 2014-02-18T04:47:31.000Z

67 requests

Answer 17 · 2014-02-18T04:48:16.000Z

It's not there

Answer 18 · 2014-02-18T05:03:20.000Z

Can you paste the full output of nbmerge?
Can you paste a screenshot of the networks tab of the chrome debugger?

Answer 19 · 2014-02-18T05:06:08.000Z

Also, please share the output of git log -1

Answer 20 · 2014-02-18T05:25:37.000Z

127.0.0.1 - - [18/Feb/2014 00:25:08] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/components/jquery-ui/themes/smoothness/jquery-ui.min.css HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /nbdiff/nbdiff.css HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/components/codemirror/lib/codemirror.css HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/style/style.min.css HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/notebook/css/override.css HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/custom/custom.css HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/components/requirejs/require.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/components/jquery/jquery.min.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/components/jquery-ui/ui/minified/jquery-ui.min.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/components/bootstrap/bootstrap/js/bootstrap.min.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/base/js/namespace.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/base/js/page.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/auth/js/loginwidget.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/components/codemirror/lib/codemirror.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:08] "GET /static/base/images/ipynblogo.png HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /nbdiff/logo.png HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/addon/mode/loadmode.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/addon/mode/multiplex.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/addon/mode/overlay.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/addon/edit/matchbrackets.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/addon/comment/comment.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/htmlmixed/htmlmixed.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/xml/xml.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/javascript/javascript.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/css/css.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/rst/rst.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/markdown/markdown.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/gfm/gfm.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/codemirror/mode/python/python.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/codemirror-ipython.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/highlight.js/build/highlight.pack.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/dateformat/date.format.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/base/js/events.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/base/js/utils.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/base/js/dialog.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/services/kernels/js/kernel.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/services/kernels/js/comm.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/services/sessions/js/session.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/layoutmanager.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/mathjaxutils.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/outputarea.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/cell.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/celltoolbar.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/codecell.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/completer.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/textcell.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/savewidget.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/quickhelp.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/pager.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/menubar.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/toolbar.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/maintoolbar.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/notebook.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/notificationwidget.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/notificationarea.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/tooltip.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/config.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/main.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/contexthint.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/celltoolbarpresets/default.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/notebook/js/celltoolbarpresets/slideshow.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /nbdiff/nbdiff.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/custom/custom.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/components/marked/lib/marked.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /api/notebooks?_=1392701109029 HTTP/1.1" 404 -
127.0.0.1 - - [18/Feb/2014 00:25:09] "GET /static/base/images/favicon.ico HTTP/1.1" 200 -

Answer 21 · 2014-02-18T05:26:31.000Z

[3298:3378:0218/002509:ERROR:download.cc(330)] PostClientToServerMessage() failed during GetUpdates

Answer 22 · 2014-02-18T05:28:27.000Z

commit 4aec2bd51946bc2d7a1ded81c3cb2dfa24a0572e
Author: bsplay bsplay@gmail.com
Date: Mon Feb 17 23:06:14 2014 -0500

Answer 23 · 2014-02-18T05:35:24.000Z

Can you push what you have to a separate branch? I need to be able to download your code to test what's going on.

Answer 24 · 2014-02-18T05:41:13.000Z

I get the same thing with master, so I don't think my branch is the problem

Answer 25 · 2014-02-18T05:44:32.000Z

If you type which nbmerge in the terminal, what do you get?

Answer 26 · 2014-02-18T05:52:08.000Z

/usr/local/bin/nbmerge

Answer 27 · 2014-02-18T05:53:20.000Z

I installed it on my laptop and I get:
Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Answer 28 · 2014-02-18T05:56:17.000Z

OK, run the following:

sudo rm /usr/local/bin/nbmerge
sudo rm /usr/local/bin/nbdiff

And see if you get anything different.

Answer 29 · 2014-02-18T06:01:28.000Z

Nothing is different

Answer 30 · 2014-02-18T06:06:29.000Z

OK, install virtualenv and try installing this and its dependencies in a fresh virtualenv. See if that makes a difference? This is pretty frustrating. :(

Answer 31 · 2014-02-18T06:37:53.000Z

Surprise, surprise it still doesn't work. Now I get my laptop's error.
Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

That is, if I did everything properly

Answer 32 · 2014-02-18T13:11:36.000Z

Change app.run(debug=False) to app.run(debug=True) in commands.py and see if you can get a more useful error message? At least you're getting a consistent message now. An internal server error would suggest a problem with our python code somewhere.

Answer 33 · 2014-02-18T16:43:37.000Z

jinja2.exceptions.UndefinedError
UndefinedError: 'static_url' is undefined

Traceback (most recent call last)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in call
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functionsrule.endpoint
File "/home/lenovo/git/nbdiff/nbdiff/server/local_server.py", line 34, in home
base_kernel_url='/', notebook_id='test_notebook')
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 128, in render_template
context, ctx.app)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 110, in _render
rv = template.render(context)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/lenovo/git/nbdiff/nbdiff/server/templates/nbdiff.html", line 1, in top-level template code
{% extends "notebook.html" %}
File "/home/lenovo/git/ipython/IPython/html/templates/notebook.html", line 1, in top-level template code
{% extends "page.html" %}
File "/home/lenovo/git/ipython/IPython/html/templates/page.html", line 10, in top-level template code

UndefinedError: 'static_url' is undefined

Answer 34 · 2014-02-18T16:59:17.000Z

In my version of ipython, the static_url(name) macro is defined at the top of /IPython/html/templates/page.html. Do you have the same?

<!DOCTYPE HTML>
{% macro static_url(name) -%}
   {{ base_project_url }}static/{{ name }}
{%- endmacro %}
<html>

Answer 35 · 2014-02-18T17:05:12.000Z

<!DOCTYPE HTML>
<html>

<head>
    <meta charset="utf-8">

    <title>{% block title %}IPython Notebook{% endblock %}</title>
    <link rel="shortcut icon" type="image/x-icon" href="{{static_url("base/images/favicon.ico") }}">
    <meta http-equiv="X-UA-Compatible" content="chrome=1">
    <link rel="stylesheet" href="{{static_url("components/jquery-ui/themes/smoothness/jquery-ui.min.css") }}" type="text/css" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">

    {% block stylesheet %}
    <link rel="stylesheet" href="{{ static_url("style/style.min.css") }}" type="text/css"/>
    {% endblock %}
    <link rel="stylesheet" href="{{ static_url("custom/custom.css") }}" type="text/css" />
    <script src="{{static_url("components/requirejs/require.js") }}" type="text/javascript" charset="utf-8"></script>
    <script>
      require.config({
          baseUrl: '{{static_url("")}}',
          paths: {
            nbextensions : '{{ base_project_url }}nbextensions'
          }
      });
    </script>

    {% block meta %}
    {% endblock %}

</head>

<body {% block params %}{% endblock %}>

Answer 36 · 2014-02-18T17:09:57.000Z

There's no macro

Answer 37 · 2014-02-18T17:30:21.000Z

My ipython was a little behind. I now get the same error, hooray!

Answer 38 · 2014-02-18T17:56:16.000Z

Yay! Now I'm not the only one!

Answer 39 · 2014-02-18T18:13:46.000Z

I'm not sure how well this will work if we're trying to support older versions too, but I can create a second handler for the new API in local_server.py:

@app.route('/api/notebooks/test_notebook', methods=['GET', 'PUT'])
def notebookjson2():
    if request.method == 'PUT':
        app.shutdown(request.data)
        request.environ.get('werkzeug.server.shutdown')()
        return ""
    else:
        parsed = {
            'content': app.notebooks[0],
            'name': 'test_notebook',
        }
        return json.dumps(parsed)

I also have to change the main handler to add the static_url function to the template:

@app.route('/')
def home():
    return render_template('nbdiff.html', project='/', base_project_url='/',
                           base_kernel_url='/', notebook_id='test_notebook', notebook_name='test_notebook',
                           static_url = lambda name: "http://localhost:5000/static/" + name)

And now I get this fun stack trace:

Traceback (most recent call last):
   ... SNIP
  File "/home/tavish/capstone/nbdiff/nbdiff/server/local_server.py", line 52, in notebookjson2
    app.shutdown(request.data)
  File "/home/tavish/capstone/nbdiff/nbdiff/commands.py", line 160, in save_notebook
    parsed = nbformat.reads(notebook_result, 'json')
  File "/home/tavish/code/ipython/IPython/nbformat/current.py", line 124, in reads
    return reads_json(s, **kwargs)
  File "/home/tavish/code/ipython/IPython/nbformat/current.py", line 79, in reads_json
    return convert(reader_reads(s), current_nbformat)
  File "/home/tavish/code/ipython/IPython/nbformat/convert.py", line 63, in convert
    converted = convert_function(nb)
  File "/home/tavish/code/ipython/IPython/nbformat/v2/convert.py", line 38, in upgrade
    if orig_version == 1:
NameError: global name 'orig_version' is not defined

Answer 40 · 2014-02-18T18:20:59.000Z

@BSPlay: Run the following inside the ipython repository to set yourself up to a known-good revision:

git checkout -b nbmerge-last-good 029c1a383a40419a37352d1079f375d2adc918b1

That way you can happily work on your stuff while I sort out compatibility issues.

Answer 41 · 2014-02-18T18:33:49.000Z

Which is the last good version

Answer 42 · 2014-02-18T18:47:35.000Z

OK, run this instead:

git checkout -b nbmerge-last-good-really rel-1.1.0

Our code seems to work with rel-1.1.0.

Answer 43 · 2014-02-18T18:47:56.000Z

You might have to run python setup.py develop again inside the ipython repository.

Answer 44 · 2014-02-18T18:59:40.000Z

HUZZAH. I'll keep this issue open for compatibility issue fixing.

Answer 45 · 2014-02-19T01:00:56.000Z

I got this running localhost:5000

Answer 46 · 2014-02-19T01:03:27.000Z

console details

Answer 47 · 2014-02-19T04:42:10.000Z

@NightlyBuilds did you try the workaround that worked for @BSPlay ?

Answer 48 · 2014-02-19T04:54:08.000Z

@NightlyBuilds if you installed it through conda that would be a separate issue -- please let us know if that's the case.

Answer 49 · 2014-02-19T05:40:01.000Z

I don't think i installed it on conda. I can't git checkout git checkout -b nbmerge-last-good-really rel-1.1.0, says cannot update paths and switch to branch 'nbmerge-last-good-really' at the same time.

Answer 50 · 2014-02-19T05:49:28.000Z

OK, try what I suggested in this comment and see if you can get a more useful error message. Judging by that screenshot, it's not the same issue.

Answer 51 · 2014-02-19T06:02:22.000Z

so i did nbdiff\scripts>nbmerge example.ipynb example-local.ipynb example-remote.ipynb
then went to localhost:5000
it now gets me Unable to connect.

Answer 52 · 2014-04-15T19:18:12.000Z

Fixed in v1.0.3