html too large
bettinardi opened this issue · 3 comments
When doing a side by side comparison of two ABM runs, the html visualizer gets to be too large and internet browsers can no longer open and manage it effectively. As best as i can tell this is do to the link assignment comparison, which isn't that important. I imagine that currently all link points are loaded in, which is what slows it down and blows up the size. What I suggest is that we filter all the links when doing a ABM side by side run so that only links with a volume of above say, 100 by period and a 1000 by daily show up in the comparison. There would have to be a little extra logic, because the code will need to build a filter link set for each scenario and then find all the values for the total link references across both scenarios that met the criteria. So a link in one scenario might be 200, but 95 in the reference, we would still want that link information even though it was below the 100 threshold in the reference.
Related, on the welcome page is a large shapefile. Currently the compare to OHAS is a 20mb html (hard to open) and the ABM to ABM html is 40MB. I believe a lot of that 20MB in the OHAS comparison is the welcome shapefile. We should do a test with the shapefile aspect commented out of the html build and compare the file size. If the size and speed of the html is greatly improved than we should consider either keeping the shapefile out (to a later point when we actually develop some functionality with the shapefile), or have a greatly simplified html, with something like district shapes as opposed to MAZs.
The size appears to be the embedded data in the top of the html file. Some ideas discussed:
- Trim link table to some minimum value (as discussed above).
- Check and see if there are extra fields / information in the tables that can be dropped
- Look at the flex dashboard for more efficient ways to store data
- Split tables / data into separate html files, or split the data in some way
The self_contained argument of the flex_dashboard() function is true by default. This argument does the following:
Produce a standalone HTML file with no external dependencies, using data:
URIs to incorporate the contents of linked scripts, stylesheets, images, and
videos. Note that even for self contained documents MathJax is still loaded
externally (this is necessary because of it’s size)
These data entries appear to contain the bulk of the 40MB. Let's try setting this to false and see what is produced.
The large HTML size is an issue for the ABM vs ABM comparisons since it involves creating link volume scatter plots for all links. The Survey vs ABM HTML creates scatter only for links with a count location and the size is manageable (11.5 MB). The size issue for ABM vs ABM HTML was resolved by creating lightweight PNGs for link volume scatter plots. This reduces the size of ABM vs ABM HTMLs from 40 MB to 9.5 MB. The visualizer switches between PNG and regular scatter plot depending on whether the base is survey or ABM.