RustPython/Parser

Investigate alternate PHF algorithms for detecting keywords

DimitrisJim opened this issue · 0 comments

This relates to a discussion on discord a while back.

The basic idea is that we can create a custom hash function for our keywords that's based on their structure (see related video in discussion) and possibly gain a good perf boost by replacing rust_phf. It still needs some investigating and bench-marking to actually evaluate if this would result in any significant improvement.

Afaik, this usually results in a bigger table being generated (for a keyword set of 30 odd functions, we might need a table of 1024 or 2048 elements mostly filled with None's) but that's a small price to pay considering this table should only be generated for the keywords and nothing else.