TODO: Fix pipeline parallelism bugs
hyunwoongko opened this issue · 1 comments
hyunwoongko commented
Describe a TODO feature
- Currently, when pipeline parallelization is run on a large model, an issue arises that gradient values are different. This issue should be addressed.
Assignees
hyunwoongko commented
Fixed