MeshTF + pipeline parallelism?
eric-haibin-lin opened this issue · 0 comments
eric-haibin-lin commented
Hi, is it possible to run mesh-tf and apply ideas like GPipe to do model-parallel training inside a layer, and pipeline parallelism cross layers? What are the caveats?