Memory leak when using longjohn and mongodb with replica sets

[jwalton]

I'm trying to use longjohn with (mongodb)[https://github.com/mongodb/node-mongodb-native], but it seems that if you have replica sets configured, and you use the "ping strategy", then node.js will very slowly leak memory if you have longjohn enabled.

I used (memwatch)[https://github.com/lloyd/node-memwatch] to print out the top 10 object types that were growing every 60 seconds. Rather consistently, I was seeing results like this:

15327 - Interval - Top 10 growing heap objects:
  Interval -   Array - 157.3 kb - 842
  Interval -   Closure - 55.97 kb - 796
  Interval -   Object - 6.73 kb - 205
  Interval -   Error - 3.09 kb - 132
  Interval -   Number - 1.02 kb - 65
  Interval -   String - 2.66 kb - 63
  Interval -   Date - 5.16 kb - 55
  Interval -   exports.DbCommand - 4.73 kb - 55
  Interval -   Code - 9.25 kb - 37
  Interval -   PingStrategy._pingServer.pingFunction - 528 bytes - 22
  Interval -   Timer - 288 bytes - 9

DbCommand and PingStrategy are both from mongodb. Using (node-webkit-agent)[https://github.com/c4milo/node-webkit-agent] to poke about in the heap, it seems the pingFunction functions are all referenced, indirectly, from Error objects. The Error object in question (and most of the tens of thousands of Error objects sticking around in memory) all have a "location" member, which points to longjohn created Errors.

If I remove longjohn from my configuration, then my memory leak seems to go away.

If you want to look at the code responsible on the mongodb side, (this is the file)[https://github.com/mongodb/node-mongodb-native/blob/master/lib/mongodb/connection/strategies/ping_strategy.js] that's causing all the trouble.

-Jason

[jwalton]

Here is a minimalist example which demonstrates the problem:

// To run:
//   npm install longjohn
//   npm install memwatch

var longjohn = require('longjohn');
var memwatch = require('memwatch');

/*
 * This is the meat of the problem
 */

var a = function() {
  setTimeout(b, 2);
}

var b = function() {
  a();
}

// Kick things off
a();


/*
 * Log heap growth
 */
var heapDiff = new memwatch.HeapDiff();

function HeapLogger() {
  var interval;
  heapDiff = new memwatch.HeapDiff();
  setInterval(this.printObjectGrowth, 1 * 1000);
}

HeapLogger.prototype.printObjectGrowth = function() {
  var delta, detail, details, diff, growing, index;

  growing = [];

  if (heapDiff) {
    diff = heapDiff.end();
    details = diff.change.details;
    details.sort(function(a, b) {return (b["+"] - b["-"]) - (a["+"] - a["-"]);});
    for (index = 0; index < details.length; index++) {
      detail = details[index];
      delta = detail["+"] - detail["-"];
      if ((index > 10) || (delta <= 0)) {
        break;
      }
      growing.push("  " + detail.what + " - size: " + detail.size + ", count: " + delta);
    }
    diff = null;
  }

  if (growing.length > 0) {
    console.log("Top 10 growing heap objects:");
    for (i = 0; i < growing.length; i++) {
      console.log(growing[i]);
    }
    console.log();
  }
  growing = null;

  heapDiff = new memwatch.HeapDiff();
};

logger = new HeapLogger();

[jwalton]

Closing, since this is now tracked by Pull Request #8.