ezyang/pytorch-unattached

More structured symbol type

Opened this issue · 1 comments

Just because we've gone all dynamically typed on the IR, doesn't mean we can't have our symbols have a little more structure than just "strings". What we have found extremely useful in GHC is to be able to allocate one level of subnamespaces to our symbols. To keep things as strings, we achieve this by reserving the first letter of the symbol as the namespace signifier.

Looking at our list of built-in symbols, here are the namespaces I'd like to see:

  • A Toffee Op namespace 'o' (oMul, oNegate)
  • A Toffee Attribute namespace 'a' (aepsilon, amomentum)
  • A "known key-attribute" namespace 'k' (kOffset, kInPlaceOutputs)

And you can always add more as necessary. The namespace signifier can be easily removed to get a raw string as necessary.

CC @zdevito

Seems like a good idea to me.