================================= Large-Scale Test Bed Visualiation ================================= This repository provides the visualization capabilities for the LTB simulator Andes. This serves as the successor to the LTBWeb visualization application with a more direct and simple implementation. Components ---------- There are 2 main components to this visualization: 1. wsdime: an interface between Dime and web browsers, using websockets to communicate. 2. ltbvis: the main JavaScript code base that handles all the rendering and control of the visualization. In addition, the visualization uses the following other components: 1. andes: the simulator that provides data to the frontend via Dime. 2. dime: a message broker that fascilitates communicating with the simulator and other modules. 3. http.server: a simple http server that provides static assets to the browser, including JavaScript code and images. Running the Visualization ------------------------- To run the visualization requires installing some packages and tools. The first tool required is Andes [andes] with the Dime module [andes_addon]. First installed andes_addon with: $ cd andes_addon $ python3.7 -m pip install . Then install Andes: $ cd andes $ python3.7 -m pip install . Next Dime [dime] needs to be installed to run the server: $ cd dime $ python2.7 -m pip install . In addition, there are some packages that need to be installed: $ python3.7 -m pip install websockets With those installed, there's a series of 5 things that need to happen to see the visualization: 1. Start the static file server. $ python3.7 server.py --port 8810 2. Start the Dime server. $ dime tcp://127.0.0.1:8819 3. Start the Dime WebSockets interface. $ python3.7 wsdime.py --port 8811 --dport 8819 4. Open the web interface. $ chrome http://localhost:8810 5. Start Andes. $ andes --routine=tds cases/curent/WECC_WIND0.dm --dime=tcp://127.0.0.1:8819 Now in Chrome (or another browser), the simulator data should be received and start displaying on-screen. To see another visualization, only steps 4 and 5 need to be repeated. So, refresh the web page and then re-run Andes. [andes]: https://github.com/cuihantao/andes [andes_addon]: https://bitbucket.org/curent/andes_addon [dime]: https://bitbucket.org/curent/dime Docker ------ To simplify the process of setting things up, there is a Dockerfile that all dependencies can be installed into. To do this, first download zip files of the dependencies from the websites above. Specific versions need to be installed, and those versions can be checked in the Dockerfile (for instance, Dime expects the code for commit b3d58ba70df to be saved in dime-b43d58ba70df.zip). Then build the Docker image using the provided helper script: $ ./go.sh build To help with running everything, you can use tmux: $ tmux (inside tmux)$ ./go.sh dev Afterwards, 4 windows will be created with some commands pre-typed in (with comments at the beginning, to prevent accidentally running everything). Everything still needs to be run in the same order as described above. Project Layout -------------- The main code for the project is in the static/index.html file and all static/js/*.js files. Briefly, these are: 1. index.html: The main entrypoint of the visualization. Responsible for loading JavaScript files, creating the Leaflet map and layers, interacting with Dime, and ensuring the layers have the most up-to-date values from Dime. 2. NDArray.js: A wrapper around JavaScript TypedArrays that enables n-dimensional access to them. 3. DimeClient.js: A wrapper around the WebSocket Dime interface that enables a familiar sync and send_var interface to the Dime server. 4. CanvasLayer.js: A custom Leaflet layer that enables rendering to a canvas atop the Leaflet map. This serves as the basis for all other LTBVis layers. 5. TopologyLayer.js: A custom Leaflet layer that draws the topology (buses, synchronous generators, lines, etc) of the simulation. In general, this shouldn't update very often because most of the data is considered to be static. 6. ContourLayer.js: A custom Leaflet layer that draws the contour map (e.g. voltage values) across the entire simulation. In general, this layer updates every single frame of the simulation. There are also a few libraries used throughout the code base. These are: 1. Leaflet: This handles the map, including loading image tiles, zooming, and panning of the map. [leaflet-docs] 2. base64-arraybuffer: This converts between a base64 string (like Dime uses for sending/receiving arrays) to a byte buffer. [base64-arraybuffer] 3. twgl: This simplifies many of the interactions with the WebGL API by greatly reducing the amount of function calls needed to manage WebGL state [twgl-docs]. 4. d3-delaunay: This computes a delaunay triangulation of buses in the simulation for rendering the contour map. [leaflet-docs]: https://leafletjs.com/reference-1.5.0.html [base64-arraybuffer]: https://www.npmjs.com/package/base64-arraybuffer [twgl-docs]: https://twgljs.org/docs/index.html JavaScript Idioms ----------------- There are several idioms used in the JavaScript code base that might need some explanation. Because these are used in multiple files, there's not one good place to document it in comments. Hence, this is the best place for this discussion. Caching in rendering layers: There are several things that should be cached between renders so that the performance doesn't get degraded. For example, projecting Bus locations from lat/lng to pixel coordinates takes a long time, and doing that every frame would be wasteful. Another example is compiling WebGL shaders. One way around this would be to only compute things once and then save them somewhere. While that would work in general, it's not very flexible and makes it harder when things do need to update (the projection example does need to be recomputed every time the user finishes panning the map). Another way (which is used in the code) is to keep track of whether the data used has actually changed. To make this efficient, the code only checks whether the identity of that data has changed, rather than recursively checking if any fields have changed. This is done via the use of a WeakMap ("Weak" meaning that objects can still be garbage collected if needed). It has get and set functions that work like a normal Python dictionary. Because the WeakMap only supports one value per key, and some things depend on the same input value (say, anything derived from Buses and Lines all depend on the value of SysParam), then there is a secondary level of indirection used. The idiom is as follows: function render(context) { // this refers to the Layer // this._cache is a WeakMap instance // context is the current set of variables const SysParam = context.SysParam; let paramCache = this._cache.get(SysParam); // here, paramCache will be undefined if SysParam has changed; otherwise, it // will refer to a normal JavaScript object if (!paramCache) { // SysParam was not found in the WeakMap, so we need to define it here. // paramCache is a simple JavaScript object paramCache = {}; // update the field in the cache so that next time we'll use this object this._cache.set(SysParam, paramCache); } // This uses a JavaScript shortcut added in recent versions called "object // destructuring." It's identical to: // let busToIndexLookup = paramCache.busToIndexLookup; // but you only need to give the name once. let { busToIndexLookup } = paramCache; // if paramCache is newly created or otherwise busToIndexLookup hasn't been // defined, then we need to define it if (!busToIndexLookup) { busToIndexLookup = paramCache.busToIndexLookup = new Map(); // populate busToIndexLookup } // now code can just refer to the variable busToIndexLookup and it will be // using the latest version, that is possibly cached if it hasn't changed. } Object destructuring: It was touched on in the code above, but it's pretty important so here it is again. New versions of JavaScript support something called "object destructuring," which is syntactic sugar for accessing properties of an object, defining variables, and also for creating new objects. Destructuring looks like: let { foo } = obj; // same as: let foo = obj.foo; const { foo, bar } = obj; // same as: const foo = obj.foo, bar = obj.bar; Creating new objects looks like: const foo = 3.14, bar = 2.718; const obj = { foo }; // same as: const obj = { foo: foo }; const obj = { foo, bar }; // same as: const obj = { foo: foo, bar: bar }; let obj = { foo, bing: 'baz' }; // same as: const obj = { foo: foo, bing: 'baz' }; Let/Const: New versions of JavaScript have also added a better keyword than "var" called "let" and "const." The main distinction is in how let/const are scoped (now it matches C) and in whether you can modify the variable reference. const foo = 3.14; foo = 2.718; // error: foo is defined as a constant variable let bar = 3.14; bar = 2.718; // allowed; "let" allows bar to be modified const foo = 3.14; if (true) { const foo = 2.718; // allowed; this "foo" is in a different scope } // foo in this scope is still 3.14 let bar = 3.14; if (true) { bar = 2.718; // allowed; "bar" is mutable; here bar refers to the outer scope. }