Performance in dequant.
Closed this issue · 5 comments
Would it be faster to perform the dequant as each coefficient is decoded? Right now the whole TU is dequantized afterwards, and for typical video that means lots of operations on 0 coefficients.
Doing dequant on the fly would result in way less operations, although it means no opportunity to use SSE2 routines....
There is a rounding that you have to apply even on zero coefficients.
Mickael
Sent by my iPhone
Le 9 sept. 2013 à 20:23, pieter3d notifications@github.com a écrit :
Would it be faster to perform the dequant as each coefficient is decoded? Right now the whole TU is dequantized afterwards, and for typical video that means lots of operations on 0 coefficients.
Doing dequant on the fly would result in way less operations, although it means no opportunity to use SSE2 routines....
—
Reply to this email directly or view it on GitHub.
Are you sure? I'm pretty sure that if a decoded coefficient is 0, then the dequant coefficient is always 0 too. This is how we have implemented a HW decoder and it does not have issues with that.
pretty sure if x == 0 then you have to add "add" in the following code.
#define SCALE(dst, x) (dst) = av_clip_int16(((x) + add) >> shift)
__
Mickaël
Le 9 sept. 2013 à 20:48, pieter3d notifications@github.com a écrit :
Are you sure? I'm pretty sure that if a decoded coefficient is 0, then the dequant coefficient is always 0 too. This is how we have implemented a HW decoder and it does not have issues with that.
—
Reply to this email directly or view it on GitHub.
Yes, but in that case the downshift will always obliterate whatever add was. "add" does not have any bit position set higher than shift
you are right.
__
Mickaël
Le 9 sept. 2013 à 21:07, pieter3d notifications@github.com a écrit :
Yes, but in that case the downshift will always obliterate whatever add was. "add" does not have any bit position set higher than shift
—
Reply to this email directly or view it on GitHub.