The generated videos are shaking and unstable.

[nankepan]

I trained an opensora model with webvid. The generated videos are shaking and unstable. Can you give me some reasons or suggestions？Is there some training tricks?

sample_0.2.mp4

sample_1.2.mp4

sample_2.2.mp4

[Edwardmark]

I test the open-sourced checkpoint the repo provided is unstable and shaking too. Is there anything missing?
I run the demo using:
torchrun --standalone --nproc_per_node=8 scripts/opensora/sample_opensora.py
--model_time_scale 1
--model_space_scale 1
--image_size 512 512
--num_frames 80
--fps 8
--dtype fp16
--sequence_parallel_size 8
--enable_flashattn
--enable_layernorm_kernel
--text_speedup
--model_pretrained_path ./ckpt/OpenSora-v1-HQ-16x512x512.pth

[oahzxl]

close due to inactive