numenta/cortipy

handle very sparse fingerprints

BoltzmannBrain opened this issue · 0 comments

The REST API sometimes returns very sparse fingerprints -- for very uncommon terms, but also some more frequent terms, e.g. "after". Theses sparsities are ~0.1%, where the target for the API is 1.0%. Some viable solutions to increase the sparsity are:

  1. randomly generate a fingerprint, as we do when the API gives a null response
  2. preserving the few returned bits, fill out the sparsity w/ random generation
  3. preserving the few returned bits, fill out the sparsity by returning the bits in closest (spatial) proximity to the returned bits

@subutai @scottpurdy what do you think?

Note: the previous HTTP API client, pycept, would scrap the returned bits and generate a new, random SDR at the target sparsity.