ines/course-starter-python

Use with (local) docker container

vnijs opened this issue · 11 comments

vnijs commented

Very interested to try this out with my students this summer but would like to use a (local) docker container. I tried several versions of "serverSettings" but get the same error in chrome developer tools? Any recommendations on what to try? Thx @ines

image

Section with changes from src/components/juniper.js

// url: 'http://localhost:8989',
url: '',
// url: 'https://mybinder.org',
serverSettings: {
    baseUrl: 'http://localhost:8989',
    token: 'jupyter'
},
kernelType: 'python3',
lang: 'python',
theme: 'default',
isolateCells: true,
// useBinder: true,
useBinder: false
ines commented

Hi! So what exactly are you running locally on port 8989? And does it allow connections from a different host/port?

If you set useBinder: false, the serverSettings will be passed to JupyterLab directly and it will not expect a BinderHub deployment like mybinder.org or your own. (You can find more details on the Juniper options here btw. You probably also want to set useStorage: false to prevent it from storing invalid server settings in your browser's local storage.)

Basically, what you pass in as the serverSettings needs to be what JupyterLab's ServerConnection.makeSettings expects. So in theory, the most minimal example that should work (independent of my Juniper implementation, the course framework etc.) is this:

import { Kernel, ServerConnection } from '@jupyterlab/services'

const settings = { }  // your server settings
const kernelType = 'python3'

const serverSettings = ServerConnection.makeSettings(settings)
const kernel = Kernel.startNew({ type: kernelType, name: kernelType, serverSettings })
    .then(kernel => {
        console.log('ready!')
        return kernel
    })

If this works, chances are that everything else will work as well. I'm not JupyterLab expert, but if you do get stuck with getting the server settings and container setup right, maybe you could also ask the community at https://discourse.jupyter.org?

vnijs commented

Hi @ines. I'm running a docker container with jupyterlab and some other things. I'm able to connect to it from different editors (e.g., Atom or VSCode) and run code so I assume that part should be fine. I tried compiling the TS you posted but that produces some new errors (see below).

I have not used TS before and my JS foo is, unfortunately, also limited. If the TS does compile to JS, how would I run it to check if it is working? I'm so used to Jupyter and Rstudio, that trying out JS code feels like an alien world sometimes :) Any specific resources you could recommend to be able to work effectively with tools like yours?

I just setup the TS kernel in a Jupyter so I'll do my best to figure out why this isn't work.

image

import { Kernel, ServerConnection } from '@jupyterlab/services'

const settings = { baseUrl: 'http://localhost:8989', token: 'jupyter' }  // your server settings
const kernelType = 'python3'

const serverSettings = ServerConnection.makeSettings(settings)
const kernel = Kernel.startNew({ type: kernelType, name: kernelType, serverSettings })
    .then(kernel => {
        console.log('ready!')
        return kernel
    })
ines commented

My code itself is just basic JavaScript, but the JupyterLab source is typed and the error is telling you that my code example was bad and type: kernelType shouldn't actually exist there and isn't a valid option. So removing that part should make that error go away.

Btw, to be honest, even though I work a lot with JavaScript, I find that just quickly writing and testing some code with external dependencies much more convenient in Python. Setting up a build pipeline in JS is kinda annoying – but it is quite fun if you can actually work in the browser. Which just gave me an idea – try this for debugging:

<!DOCTYPE html>
<html>
    <body>
        <script src="https://unpkg.com/@jupyterlab/services@3.2.1/dist/dist/index.js"></script>
        <script>
            const { Kernel, ServerConnection } = window['@jupyterlab/services']

            const settings = { baseUrl: 'http://localhost:8989', token: 'jupyter' } // your server settings
            const kernelType = 'python3'

            const serverSettings = ServerConnection.makeSettings(settings)
            const kernel = Kernel.startNew({
                name: kernelType,
                serverSettings
            }).then(kernel => {
                console.log('ready!')
                return kernel
            })
        </script>
    </body>
</html>

No compilers, no weirdness, just executing JavaScript in the browser. Save to a .html file, open in a browser, add console.log statements in between etc. and see the output in the browser's console.

vnijs commented

Thanks for the suggestion @ines but I keep getting errors even-though I can connect to the running container from Atom and VSCode. I'll close for now and perhaps re-open once I learn more about the jupyterlab api and TS :)

image

ines commented

@vnijs If you don't mind, I'd be happy to keep this open – maybe someone else who comes across it has an idea.

I also did some digging to try and find how the VSCode extensions conntect to a notebook server and it all looks pretty similar.

A few more ideas:

  • Try adding a wsUrl (websocket) to your settings – so in your case, wsUrl: 'ws://localhost:8989'.
  • Can you double-check that CORS is enabled in your notebook server? jupyter/jupyter#79 (comment)
vnijs commented

Thanks @ines. Made some progress it seems. I can connect with a docker container with or without CORS using node connect where connect.ts is the typescript file. Note the addition of serverSettings in the connectTo call

import { Kernel, ServerConnection } from '@jupyterlab/services'

const settings = {
    // using vnijs/rsm-jupyterhub
    // baseUrl: 'http://localhost:8888',
    // token: '98b2840637c62c9c815ea346fdc27f1d25894363f7012163',
    // using vnijs/rsm-msba
    baseUrl: 'http://localhost:8989',
    token: 'jupyter',
}  // your server settings
const kernelType = 'python3'
const serverSettings = ServerConnection.makeSettings(settings)

// show available kernels and connect to one
Kernel.listRunning(serverSettings).then(kernelModels => {
    console.log(kernelModels);
    if (kernelModels.length > 0) {
        const kernel = Kernel.connectTo(kernelModels[0], serverSettings);
        console.log(kernel.name);
    };
});
// starts a new kernel
const kernel = Kernel.startNew({
    name: kernelType,
    serverSettings
}).then(kernel => {
    console.log('ready!')
    return kernel
});

I acitivated CORS in vnijs/rsm-jupyterhub and that now works with the HTML page. However vnijs/rsm-msba which doesn't have CORS does not work from the HTML page.

<!DOCTYPE html>
<html>

<body>
    <script src="https://unpkg.com/@jupyterlab/services@3.2.1/dist/dist/index.js"></script>
    <script>
        const { Kernel, ServerConnection } = window['@jupyterlab/services']
        const settings = {
            // using vnijs/rsm-jupyterhub
            baseUrl: 'http://localhost:8888',
            token: '98b2840637c62c9c815ea346fdc27f1d25894363f7012163',
            // using vnijs/rsm-msba (doesn't work due so CORS issues)
            // baseUrl: 'http://localhost:8989',
            // token: 'jupyter',
        } // your server settings
        const kernelType = 'python3'
        const serverSettings = ServerConnection.makeSettings(settings)
        Kernel.listRunning(serverSettings).then(kernelModels => {
            console.log(kernelModels);
            if (kernelModels.length > 0) {
                const kernel = Kernel.connectTo(kernelModels[0], serverSettings);
                console.log(kernel.name);
            }
        });
        const kernel = Kernel.startNew({
            name: kernelType,
            serverSettings
        }).then(kernel => {
            console.log('ready!')
            return kernel
        });
    </script>
</body>

</html>

Will try the course next.

vnijs commented

Feature request: Might be nice to have an icon in jupyterlab that a user can click to start one (or more) courses after they start a docker container that has the relevant materials. Alternatively, just a container with the key resources that starts on an exported port. Could be setup behind jupyterhub as well I assume.

Question: How are answers and scores stored?

Below are the settings I had to change to juniper.ts. It does require a jupyter container with CORS enabled, as you mentioned so you need to add the below to jupyter_notebook_config.py. I'm not sure why exactly CORS is required really as I can connect directly to these container from Atom and VSCode without that setting and node connect also works. However, from a browser, CORS seems necessary.

c.NotebookApp.allow_origin = "*

The container I was testing on does not actually contain your course-starter code (vnijs/rsm-jupyterlab on DockerHub) so I tried mounting the git directory as the home directory to use for the container and that (mostly) worked. Sometimes, however, I get the error below when clicking the Submit button.

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-4e7060bb2b5b> in <module>
----> 1 from wasabi import Printer
      2 __msg__ = Printer()
      3 __solution__ = """import json
      4 
      5 # This code will run relative to the root of the repo, so we can load files

ModuleNotFoundError: No module named 'wasabi'

Below the required changes to source/components/juniper.ts

    static defaultProps = {
        children: '',
        branch: 'master',
        url: 'http://localhost:8888',
        serverSettings: {
            baseUrl: 'http://localhost:8888',
            token: 'a17667e97f83a71c9d276f0e088a2cd5da09d2819fd0e9fe'
        },
        kernelType: 'python3',
        lang: 'python',
        theme: 'default',
        isolateCells: true,
        useBinder: false,
        ...
ines commented

Cool, glad it worked!

And it looks like all you're missing are the requirements: https://github.com/ines/course-starter-python/blob/master/binder/requirements.txt

In my framework, I use my little utility helper library wasabi for easy formatted command-line output (like, green with a check mark for success messages etc.) So if you want to use that in your tests and the testTemplate, you need to make sure it's installed.

Question: How are answers and scores stored?

If you mark an exercise as completed, that information is stored in the browser's local storage.

vnijs commented

I did install the npm module called wasabi before realizing it was a python module :) All working now. Thanks for your help @ines! Not sure if you still want to keep this open or not ...

Question: Do you have any plans to add an option to store scores?

ttimbers commented

@vnijs - am I reading correctly that you got this course framework working on a JupyterHub? If so, this is exactly what we'd like to do! Can we ping you for help? If you have any docs/notes, we'd gladly take them in any shape/format.

vnijs commented

@ttimbers I didn't get that far unfortunately. I do have a bunch of additional "services" running in jupyter so I think this should be doable. See example file below.

https://github.com/radiant-rstats/docker/blob/master/rsm-jupyterhub/jupyter_notebook_config.py