htm-community/nupic.visualizations

Performance profiling of the Graph

Opened this issue · 4 comments

breznak commented

Related to problem in #41

  1. on hotgym_full , garbage collection takes 49% time, can we reduce it?
  2. with strokeBorderWidth(=0.1) the function Dygraph.stroke._drawSeries() rampages from about 10sec/graph to 280sec/graph!
  3. highlightSeriesOpts also causing slowdown
  4. large graph loading too slow -- what are the steps, are we using chunking for that? may need to revert to a lower MAX_FILE_SIZE
breznak commented

ad 4) I fugured with "hide all" option, the graph renders much faster. Could be show the loading screen, keep loading (if not streaming) and Show All when reached end of file?

jefffohl commented

I think that 60MB might be too large. That said, we can profile the different parts of the process.

breznak commented

Ok, we can try how reducing the size solves the issue.
Currently we resort to "Windowing" on large files, which is great for streaming data, but not ok if you want to work with the whole file, as in the hotgym_full.
What do you think about adding a toggle: Whole file (slow), Sliding window, Compressed, that would appear only if file>MAX_SIZE? Whole file & Windowing are implemented, Compressed would be idea of #43.

jefffohl commented

That might be the solution. This ties into my thoughts about a revamped UX, where we perhaps have a step after the file has been checked, to ask the user how they would like the file to be processed.