Feature discussion: Interactive network graph
Closed this issue · 13 comments
Opening a discussion here for figuring out the implementation of the interactive network. Preliminary work has come to the point where we can generate an interactive network using
- a python networkx backend (loads a txt file with defined edges)
- a D3.JS block that loads a JSON graph file exported by networkx
- a combination of D3.JS and HTML to load an interactive network
The advantages of using D3.JS here are its scalability (e.g. 350 nodes runs fine, see notebook), its modularity (should be reasonably straightforward to implement interactive features) and relatively simple insertion into a django environment (of which we can discuss the details at a later stage).
You can play with the interactive graph by checking out the branch, creating the conda env (modsim_env.yml) and running the .ipynb locally, but here is a screengrab:
At some point each node will be represented as a 2D mol structure of course (this is usually done by making a ./tmp/ populated with .png files).
It would be a good idea to discuss features that we want the network to have; I had some ideas (ordered in degree of importance):
- create new edge
- delete edge
- delete node
- undo
- easy zoom in/out
Not sure if we would want any of these:
- allocate more/less sampling?
- make an edge single direction?
- Chem. properties on hover node, difference in prop on hover edge
- preset # of edges (potential can of worms because this requires edge priorities)
Feel free to suggest any other features of course.
Hi @JenkeScheen,
A cool way to test python stuff using GPU's and even TPU's is google-colab.
https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d
I ran your jupyter script there:
https://colab.research.google.com/drive/1mZFX8w97x_RoDFhHGFpLF1ThE_3Q09VI?usp=sharing
and, though it worked, mostly, I couldn't get the neat interactive graph you show here.
Any special additional tips on how to run it?
Hi @esguerra,
I think that issue might be because google colab is bit different from jupyter notebooks under the hood.. Are all cells executing without complaining, but then there is no network appearing (under the HTML block)? This post seems to suggest an issue with loading D3JS in google colab:
https://livingwithmachines.ac.uk/d3-javascript-visualisation-in-a-python-jupyter-notebook/
I think the easiest way to reproduce the interactive network might be to clone, create the conda env and launching the notebook in a local jupyter session.
Regardless, this method of writing JS+HTML constructions within notebooks is a bit clunky as for instance if there is a bug in the JS code the error (ESlint/JSlint) won't show up. I think I'll have to move away from the warm comfort of jupyter notebooks in order to get this thing to work properly anyway.
I've playing around with a new example I found that is already quite close to what we'll probably want. Conveniently, you can play with that one here:
https://bl.ocks.org/cjrd/6863459
Once I have a reasonable tool based on this one I'll push to this branch and we can discuss further, but I think it could still be valuable to discuss the above points here.
@esguerra I found a way to share javascript web objects more easily:
If you visit this JSFiddle, you should be able to load jquery (left-hand side, click resources, search for jquery, click top result, then click the '+' icon to load the lib (takes a second)). Then a control/cmd + return should load the network on the right-hand side panel. You'll be able to play around with the utilities of network :) I haven't figured out how to load images into it from github yet, unfortunately (from local path works fine but I can't upload files to jsfiddle, it seems)
actually, seems broken again now.. will try to resolve asap
should be fixed now! if you see a '$ is not defined' error in the bottom right terminal, it means you haven't loaded jquery yet.
@esguerra I found a way to share javascript web objects more easily:
If you visit this JSFiddle, you should be able to load jquery (left-hand side, click resources, search for jquery, click top result, then click the '+' icon to load the lib (takes a second)). Then a control/cmd + return should load the network on the right-hand side panel. You'll be able to play around with the utilities of network :) I haven't figured out how to load images into it from github yet, unfortunately (from local path works fine but I can't upload files to jsfiddle, it seems)
@JenkeScheen Cool!
Will check it out a bit later. As you've noticed I added a few things to get the JuPyTeR notebook to be deployed automagically at binder (which creates a docker container with all that's needed in the "background"). So that also managed to address the problem of getting the interactive js up and running to play with.
yes, that looks great, it's interesting to see how far ipynbs can be pushed! I've moved away from D3.JS though and instead am now using visjs - if no one objects I'd rather move on to a dedicated JS IDE for developing this module as I've grown a bit more confident with it.
molecule images are loaded into the network now as well. Still need to resolve the images fitting in the nodes - this should be straightforward to do once I get around to it (need to write circular PNGs with python, then load images and set node shape as image shape in visjs)
pushed new code now that loads the CDK2 dataset FEP outputs, generates molecule imagery using rdkit, writes a graph.json using networkx and generates a network using networkgen.html. Check the JSFiddle to see how it looks!
Majority of the code can be found in helper_functions.py
I still have to take care of some molecule images being cut off (e.g. ligands 17 and 1h1r) - this is happening mostly because molecule images are saved as square images but nodes are circular - if people would prefer squeare nodes let me know, that would save me a ton of work.
SEMs shown as +-.. was mostly just to see if it's possible at the moment, I don't really like the look of it so might remove later. I can also show number of crashes in some way of course. At the bottom of the code in the fiddle you'll find some more of my todos. Let me know if you can think of any features I've not thoguht of!
cool work! I was just toying around a bit. I don't mind square or circular nodes so if it saves time I'd say let's go for the square ones.
I noticed, by accident, it's possible to draw an edge to the node itself:
this is something we might want to avoid happening.
I remember you were also mentioning an undo button at some point, but I couldn't find it (might be I am looking in the wrong direction)?
Nice things that could be added (might be good to have a selection menu as this would overcrowd the edges if all shown at once):
- MFP similarity
- Experimental DDGs if available
- Some information on convergence quality maybe
- Cycle closure errors?
Next step would be to have another window in table format that is linked to the network, which would include a per edge and per node comparison tab so you can easily compare DDGs and DGs.
I'll let you know if I think of more :)
I noticed, by accident, it's possible to draw an edge to the node itself:
Yes! this is something native to the API for whatever reason. I haven't found an option yet to disable this unfortunately (even though I'm sure it'll possible on low level); it seems the only high level option available is to have a pop-up which asks for confirmation of connecting to self (an easy fix would thus be to make the confirmation say "not yet implemented, edge will be ignored" even though this is quite sloppy). Might have to raise an issue on their github for this at some point.
I remember you were also mentioning an undo button at some point,
yes, I'd started work on this with my previous prototype but haven't picked that up for this one. Quick google suggests it's possible, should be easy to implement.
MFP similarity
Easy enough to pass to the JSON file. We might have to discuss what type of FP would be preferable. Could display on hover node, e.g.?
Experimental DDGs if available
Again, easy to display, but might overcrowd the network, so might have to think about how we display it.
Some information on convergence quality maybe
Depending on how you want to define this could be easy to display again. Could potentially introduce a colour gradient to edges.
Cycle closure errors?
This might be a complicated feature to develop but not impossible. Again, this might be worth discussing in further detail before attempting, but I definitely agree this would be a valuable inclusion. I suspect I'll have to hack around in python to get to this in the easiest way and then somehow pass it to JS.
Next step would be to have another window in table format that is linked to the network, which would include a per edge and per node comparison tab so you can easily compare DDGs and DGs.
Yep, should be easy to do in pure HTML or simply with Django + pandas
--> if I can find time this week I'll have a shot at a few in range options here, others we can discuss later in more detail. Thanks @jesperswillem!
added a new version of the network generator to my branch + my fiddle. This version includes:
- adjusted physics to be more repulsive; decreases occasional edge/node overlap on load
- now physics stops after populating the graph coordinates, preventing jiggle during user drag
- nodes are now rectangular shapes fitting the input images (added ligand names as node titles under each node)
- disabled functionality of adding a node edge to self
- included an undo+redo button: can be optimised visually through css at point of integration, but works for now. I noticed undoing deletion of a node reinserts the node disconnected from the graph - will fix this some other time. Edit: clicking undo again reinserts the edges. Might be enough to just prompt the user to do this some way.
Closing this for now as the base code for the network generator has been wrapped up. In case we want to further enhance the generator after successful inclusion into the UI we can reuse this thread, potentially.