On Py_True and Py_False we call DECREF but not INCREF
mrcslws opened this issue · 0 comments
The py::Bool
class destructor calls Py_XDECREF
via py::Ptr
: https://github.com/numenta/nupic.core/blob/5bd0bac33d900da6dd0b964e40db3fafecf8d4b4/src/nupic/py_support/PyHelpers.cpp#L143
But it never calls Py_XINCREF
. When I created py::Bool
(over a year ago) I copied the code for other scalars, which also never call Py_XINCREF
. They don't do this because they rely on APIs like PyInt_FromLong
which automatically increment the reference count. But with bools, we don't call any Python APIs. So we have to manually increment the count.
If I add this code to Bool
in PyHelpers.cpp:
Bool::~Bool()
{
if (p_ == Py_True)
{
NTA_INFO << "Py_True refcount " << p_->ob_refcnt;
}
if (p_ == Py_False)
{
NTA_INFO << "Py_False refcount " << p_->ob_refcnt;
}
}
you can see the refcounts continuously drop.
INFO: Py_True refcount 2354
INFO: Py_True refcount 2353
INFO: Py_True refcount 2352
INFO: Py_True refcount 2351
INFO: Py_True refcount 2350
INFO: Py_True refcount 2349
INFO: Py_True refcount 2348
INFO: Py_True refcount 2347
INFO: Py_True refcount 2346
INFO: Py_True refcount 2345
INFO: Py_True refcount 2344
INFO: Py_True refcount 2343
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_False refcount 3952
INFO: Py_True refcount 2330
INFO: Py_True refcount 2329
INFO: Py_True refcount 2328
INFO: Py_True refcount 2327
INFO: Py_True refcount 2326
INFO: Py_True refcount 2325
INFO: Py_True refcount 2324
INFO: Py_True refcount 2323
INFO: Py_True refcount 2322
INFO: Py_True refcount 2321
INFO: Py_True refcount 2320
INFO: Py_True refcount 2319
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
INFO: Py_False refcount 3928
(I'm not sure why Py_False
is decrementing all at once)
This is a ticking time bomb. When it hits 0, we get an impossible-to-debug segfault. For example: https://github.com/numenta/htmresearch/blob/ff9374316463d2aec1f44e57cead8f81656005e9/projects/l2_pooling/infer_hand_crafted_objects.py#L109