numenta/nupic.core-legacy

On Py_True and Py_False we call DECREF but not INCREF

mrcslws opened this issue · 0 comments

The py::Bool class destructor calls Py_XDECREF via py::Ptr: https://github.com/numenta/nupic.core/blob/5bd0bac33d900da6dd0b964e40db3fafecf8d4b4/src/nupic/py_support/PyHelpers.cpp#L143

But it never calls Py_XINCREF. When I created py::Bool (over a year ago) I copied the code for other scalars, which also never call Py_XINCREF. They don't do this because they rely on APIs like PyInt_FromLong which automatically increment the reference count. But with bools, we don't call any Python APIs. So we have to manually increment the count.

If I add this code to Bool in PyHelpers.cpp:

  Bool::~Bool()
  {
    if (p_ == Py_True)
    {
      NTA_INFO << "Py_True refcount " << p_->ob_refcnt;
    }

    if (p_ == Py_False)
    {
      NTA_INFO << "Py_False refcount " << p_->ob_refcnt;
    }
  }

you can see the refcounts continuously drop.

INFO:   Py_True refcount 2354
INFO:   Py_True refcount 2353
INFO:   Py_True refcount 2352
INFO:   Py_True refcount 2351
INFO:   Py_True refcount 2350
INFO:   Py_True refcount 2349
INFO:   Py_True refcount 2348
INFO:   Py_True refcount 2347
INFO:   Py_True refcount 2346
INFO:   Py_True refcount 2345
INFO:   Py_True refcount 2344
INFO:   Py_True refcount 2343
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_False refcount 3952
INFO:   Py_True refcount 2330
INFO:   Py_True refcount 2329
INFO:   Py_True refcount 2328
INFO:   Py_True refcount 2327
INFO:   Py_True refcount 2326
INFO:   Py_True refcount 2325
INFO:   Py_True refcount 2324
INFO:   Py_True refcount 2323
INFO:   Py_True refcount 2322
INFO:   Py_True refcount 2321
INFO:   Py_True refcount 2320
INFO:   Py_True refcount 2319
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928
INFO:   Py_False refcount 3928

(I'm not sure why Py_False is decrementing all at once)

This is a ticking time bomb. When it hits 0, we get an impossible-to-debug segfault. For example: https://github.com/numenta/htmresearch/blob/ff9374316463d2aec1f44e57cead8f81656005e9/projects/l2_pooling/infer_hand_crafted_objects.py#L109