node-heap-dump

Overview

This is a collection of programs for creating and viewing Node heap dumps using V8 snapshots. This was an experiment to see what run-time structures could be viewed post-mortem using heap snapshots.

There are the following components:

pmd.cc: add-on that provides one function, takeSnapshot, that takes a new heapsnapshot and serializes it as JSON to stdout (using the built-in V8 mechanism for this).
sample.js: simple script that uses the "pmd" add-on to generate a heap snapshot.
heap-dump.js: a class called HeapDump that parses serialized heap snapshots. This is probably the component most likely to be useful for other projects because it parses the heap snapshot pretty generically.
hdb.js: a program that takes a heap dump generated by the "pmd" add-on, parses it using heap-dump.js, and lets you summarize or explore the results.

The heap snapshot parser is pretty basic: it resolves references to strings, type names, etc. but does not resolve references to other nodes. This would be easy to add as a second pass.

Note that the heap snapshot is a relatively compact representation; the in-memory representation generated by this parser is substantially larger.

Setup: creating a snapshot

# install dependencies
npm install sprintf

# build the add-on
node-waf configure
node-waf build
node-waf install

# generate a heap snapshot
node sample.js > heapsnapshot

Example 1: summarize a snapshot

node hdb.js heapsnapshot text > heapsummary.txt

The result is a text file that resembles the snapshot but with various string and type references resolved and presented in more human-readable form.

Example 2: explore a snapshot

You can use "explore" to explore a snapshot. It opens up a REPL with the following functions:

node(num): return the node with index num.
findstr(str): return the node corresponding to the string str.
findrefs(num): return nodes referencing the node with index num.
parents(num): returns array of nodes referencing the specified node
dump(depth): dumps out information from the root
tree(num, depth): prints a tree of nodes
root(num): prints the shortest path from the specified node to the root

For example:

node hdb.js heapsnapshot explore
> findstr('compost')
{ index: 503438,
  type: 'string',
  name: 'compost',
  children: [] }

> findrefs(503438)
[ { node: '40615',
    node_nchildren: 16385,
    node_name: '',
    type: 'hidden',
    name_or_index: 7722,
    to_node: 503438 },
  { node: '165335',
    node_nchildren: 6,
    node_name: 'Array',
    type: 'element',
    name_or_index: 0,
    to_node: 503438 },
  { node: '226953',
    node_nchildren: 6,
    node_name: 'Array',
    type: 'element',
    name_or_index: 0,
    to_node: 503438 },
  { node: '294910',
    node_nchildren: 3,
    node_name: '',
    type: 'hidden',
    name_or_index: 2,
    to_node: 503438 } ]

> node(165335)
{ index: 165335,
  type: 'object',
  name: 'Array',
  children: 
   [ { type: 'element', name_or_index: 0, to_node: 503438 },
     { type: 'element', name_or_index: 1, to_node: 566310 },
     { type: 'property',
       name_or_index: '__proto__',
       to_node: 40475 },
     { type: 'hidden', name_or_index: 1, to_node: 248392 },
     { type: 'hidden', name_or_index: 2, to_node: 570782 },
     { type: 'hidden', name_or_index: 3, to_node: 294910 } ] }

Though it's not obvious if you haven't spent time looking at V8 heap dumps, we've established that node 165335 is an array whose first element is the string 'compost'.

Resources

The snapshot format is documented in:

node/deps/v8/include/v8-profiler.h

This discussion contains additional useful information: