Jitter problem.

Question

Jitter problem.

zhanglonghao1992 opened this issue 2 years ago · 5 comments

zhanglonghao1992 commented 2 years ago

Hi～
When I tested it on videos, I found that there was obvious inter frame jitter in the result, as shown below.

shape.mp4

Why does it cause such obvious jitter? Is there any way to optimize it.

Answer 1 · 2022-08-03T18:55:54.000Z

Do you use DECA camera or a camera from the mobile-net?

Answer 2 · 2022-08-05T06:29:06.000Z

@khakhulin I use the default settings in infer.py.

Answer 3 · 2022-08-29T13:09:59.000Z

So, it should be ok, with the natural jitter from the deca (see video on the site).

Answer 4 · 2022-09-14T00:59:49.000Z

@khakhulin Is there a way to reduce this jitter? What is the 'camera from the mobile-net' that you refer to?

Answer 5 · 2022-09-18T22:05:59.000Z

@nlml I just suggest using the original camera from the DECA, not the distiller version. I think if the jitter is a real problem. then you need to optimize the sequence of the keypoints itself. Since original DECA doesn't guarantee temporal consistency if you run it with per-frame expression transfer.