awslabs/slapo

[Tracer] PyTorch tracer does not rename SelfAttention module

chhzh123 opened this issue · 2 comments

HuggingFace uses a very terrible variable name, self.self, for the SelfAttention module in Transformer-based models, which overwrites the Python self keyword and causes problems for the traced torch.fx module. (See this line in BERT)

Currently, we automatically rename the SelfAttention module as self_m in the HF tracer, but preserves the PyTorch tracer as the same. Users may encounter issues when they try to access the original SelfAttention module. We may want to unify both tracers and figure out a better way to handle these terrible naming issues.

Low priority issue, just a note for future reference.

It seems there is no need to rename the module at this time. So I'll close this issue.