uploadcare/pillow-simd

Resize generate black band on image

Lucuz91 opened this issue · 6 comments

Lucuz91 commented

What did you do?

Just resize a image with a particular size

What did you expect to happen?

Don't add black band on image

What actually happened?

Add a black band on image

What are your OS, Python and Pillow versions?

  • OS: ubuntu 18.04
  • Python: 2.7 and 3.7
  • Pillow-simd: 5.3.0.post0

This code work well with pillow, but with pillow-simd the black band appear.
Also if i change size works well, i don't understand why only with this image and this size.
I tried to replicate the thumbnail function without usate draft function, and works well, i don't know if this help to investigate.

from PIL import Image
img = Image.open('original.jpg')
img.thumbnail((170,170), 1)
img.save('pillow-simd-converted.jpg')

Original image
original

Pillow-simd converted
pillow-simd-converted

homm commented

I can confirm the error. It appears during resizing with Lanczos filter when the width of the source image is 341 px and destination width is 170 px on Pillow-SIMD both complied with SSE4 and AVX2.
Minimal test case:

im = Image.new('RGB', (341, 60), 'pink')
im.resize((170, 60), Image.LANCZOS).save('_out.png')

_out

170 is related to 341 as 2x+1, but the bug is not appear with other combinations of widths (for example, 321 and 160). Needs further investigation.

I'm sorry you're facing this weird thing.

homm commented

I found the bug, it was in the convolution coefficients normalisation to UINT16 type:

2eb555f#diff-704f59b07cceb3ff1645f2127d7e6d8cR298

The former implementation assumed a maximal coefficient value as a double on each step and compared it with int type range:
maxkk < (1 << (MAX_COEFS_PRECISION-1)

While in reality, the final value is calculated with rounding:
(int) (0.5 + prekk[x] * (1 << coefs_precision))

It so happened that given combination of parameters (341→170 resize with Lanczos filter) has lead to overflow (the maximum coefficient was outside the range of UINT16) in coefficients in two central pixels.

I've changed the condition and now it perfectly match following normalisation with rounding.

I'll tag the fix tomorrow. Thank you for finding this and sorry again.

  • 👍1
Lucuz91 commented

No, thanks to you, you have solved this strange problem very quickly. Thanks again for the support.

homm commented

I've done some research about the frequency of the overflow issue. I've tested all resize operations for the source size from 1 to 10'000 pixels and the destination size also from 1 to 10'000 using five available interpolation filters (500 million operations in the total). In theory, the overflow should happens in every 32'768th case, when limited factor is capacity of UINT16 type (more than 99% operations from given ranges). There are several conclusions:

  • Overflow never happens for upscaling and downscaling in less than two times;
  • There are 1906 overflows but the frequency is different for different interpolations:
    • Overflow never happens for BOX interpolation (rarely used);
    • For LINEAR interpolation overflow appears 103 times, i.e. every 970'873th operation;
    • HAMMING: 609 times, every 164'203th operation;
    • BICUBIC: 374 times, every 267'379th operation;
    • LANCZOS: 820 times, every 121'951th operation;
  • With LINEAR and BICUBIC interpolations black bands appears only on the edges of an image.
  • With LANCZOS interpolation black bands can appear at any part of an image. For example, here is a resizing from 2387x682px to 1190x340px using LANCZOS interpolation:

1190x340-1

The main reason why this error was undetected for so long time is that we are using a BICUBIC interpolation in production. By a Fluke, overflow for this interpolation has medium frequency and not very annoying appearance.

  • 👍1
Lucuz91 commented

Congratulations on the investigation, very accurate.
We were lucky / unlucky to find this bug, since the probability of having it is really low.

But thanks to you now everything is solved, thanks again :)