MarginaliaSearch/MarginaliaSearch

(task) Update Stale Language Models

vlofgren opened this issue · 0 comments

The language ngram and term frequency models are very old and questionable in how they were constructed.

  • NGramBloomFilter -- how was this even created? It's used in query construction. May not be necessary.
  • TermFrequencyDict -- construction logic needs patching to run, can then be generated on prod data probably.