Could flexgen be used for training?

[leiwen83]

Hi,

As I understand, current flexgen only serve as infer llm in small gpu, which would not happen if without flexgen.
As llm training is also resource monster process, could flexgen be applied in the training process? So that with limited cpu memory and gpu could train big model?