sequence-parallelism

There are 2 repositories under sequence-parallelism topic.

  • xrsrke/pipegoose

    Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

    Language:Python7743217

  • AlibabaPAI/FlashModels

    Fast and easy distributed model training examples.

    Language:Python61003