Custom implementation of hierarchical segment-wise self-attention network (HHSA) from Dialog-Post: Multi-Level Self-Supervised Objectives and Hierarchical Model for Dialogue Post-Training (Zhang et al., ACL 2023). Main difference is that each segment is separate utterance. Built upon huffing face MPNet. You can load weights using from_pretrained() method. Example usage is provided here.
