Color palette not working with the latest version of Pillow

[valerie-vallet]

What did you do?

I am trying to color a 3D map with a color palette

What did you expect to happen?

Results are perfect with Pillow 9.4.0, but comes out with wrong colors with 10.1.0

What actually happened?

What are your OS, Python and Pillow versions?

• OS: Mac
• Python: 3.11.3
• Pillow: 10.1.0

import sys
import os.path
import skimage.io
import numpy as np
import matplotlib.pyplot as plt
from IPython.display import FileLink
from PIL import Image as PImage
import plotly.graph_objects as go
from plotly import tools
import plotly.offline

# Input parameters
if (len(sys.argv) != 7):
    print("""\
This plotting script requires 4 arguments!

Usage:  <image.jpg> <zfile> <wima> <hima> <z_min> <z_max>
""")
    sys.exit(1)

inimag = sys.argv[1]
inzval = sys.argv[2]
wima = int(sys.argv[3])
hima = int(sys.argv[4])
z_min = float(sys.argv[5])
z_max = float(sys.argv[6])

basename = os.path.splitext(inimag)[0]
print(inimag)
print(basename)

# Read image
img = skimage.io.imread(inimag)

# Read z-values
fin = open(inzval, "rb")
dr = np.fromfile(fin, dtype=np.short)
d = np.reshape(dr, (hima, wima))

#z_min=np.min(d)
#z_max=np.max(d)


fig, ax = plt.subplots(1,2, figsize=(20,10))
ax[0].text(50, 100, 'original image', fontsize=16, bbox={'facecolor': 'white', 'pad': 6})
ax[0].imshow(img)

ax[1].text(50, 100, 'depth map', fontsize=16, bbox={'facecolor': 'white', 'pad': 6})
ax[1].imshow(d)

d = np.flipud(d)
img = np.flipud(img)

def create_rgb_surface(rgb_img, depth_img, depth_cutoff=0, **kwargs):
    rgb_img = rgb_img.swapaxes(0, 1)[:, ::-1]
    depth_img = depth_img.swapaxes(0, 1)[:, ::-1]
    eight_bit_img = PImage.fromarray(rgb_img).convert('P', palette='WEB', colors=256, dither=None)
    idx_to_color = np.array(eight_bit_img.getpalette()).reshape((-1, 3))
    colorscale=[[i/255.0, "rgb({}, {}, {})".format(*rgb)] for i, rgb in enumerate(idx_to_color)]
    depth_map = depth_img.copy().astype('float')
    print(depth_map)
    depth_map[depth_map<depth_cutoff] = np.nan
    return go.Surface(
        z=depth_map,
        surfacecolor=np.array(eight_bit_img),
        cmin=0, 
        cmax=255,
        colorscale=colorscale,
        showscale=False
    )

fig = go.Figure(
    data=[create_rgb_surface(img, d, 0 )],
    layout_title_text="3D Surface"
)

#fig.update_traces(contours_z=dict(show=True, usecolormap=True, project_z=True, size=5, start=np.min(d), end=np.max(d)))

fig.update_layout(title=basename, autosize=True, width=1000,
        scene = dict(zaxis = dict(range=[z_min, z_max]), aspectmode='manual', aspectratio=dict(x=1, y=1, z=0.5)),
        scene_camera_eye=dict(x=1.20, y=1.20, z=0.50),
        margin=dict( l=20, r=20, b=40, t=40, pad=1),
)

outfile = (basename + "-rgbd.html")
plotly.offline.plot(fig, filename = outfile, auto_open=False)
FileLink(outfile)

#outfile = (basename + "-rgbd.png")
#fig.write_image(outfile, height=2048, width=2048, engine="kaleido")

[valerie-vallet]

[radarhere]

Could you provide the inputs for your code so that we can test it exactly? I gather there are 7 arguments, including an image and a file. What are the arguments? Could you attach those files here?

[valerie-vallet]

data.zip
I attach the files and the RUN command to execute the python code.
Hope this helps you @radarhere to take a look at the problem.
Thanks a lot!

[radarhere]

#7289 set an image's C palette to be empty by default. In your case, this has the effect of eight_bit_img.getpalette() only returning 226 entries, instead of the full 256.

If you replace

colorscale=[[i/255.0, "rgb({}, {}, {})".format(*rgb)] for i, rgb in enumerate(idx_to_color)]

with

colorscale = []
for i in range(256):
    rgb = idx_to_color[i] if i < len(idx_to_color) else (0, 0, 0)
    colorscale.append([i/255.0, "rgb({}, {}, {})".format(*rgb)])

I think the problem should be resolved.

[theCapypara]

Hello! We are also running into the same or very similiar issues with palettes, for us resulting in IndexErrors.

Failing test pipeline:
https://github.com/SkyTemple/skytemple-files/actions/runs/6536495712/job/17748820497?pr=438

It's the first failing test, the second can be disregarded, this is just due to the new default font.

This is an editing library for an old video game which uses paletted images.
After doing some more investigation I can provide more compact examples based on our (rather complex) code structure here, if that's needed and helpful, but the issue is probably clear already, reading the previous comments.

Should this not be considered a bug, could this change be documented in the changelog? I'm not even 100% sure what exactly changed, is it that Pillow now no longer fills the palettes of PNGs?

Edit: Reading your previous comment I guess it's any indexed images, no matter how it was created.

[radarhere]

I don't think of this as a bug. The palette isn't missing useful information - it's no longer returning redundant information.

I don't think it was ever documented that getpalette() should return 256 entries. It started returning fewer entries with #6060, back in Pillow 9.1.0, listed in the changelog as 'Consider palette size when converting and in getpalette()'. This actually helped clarify things - a user in #6046 was confused about the purpose of the extra entries that were filling up the palette to 256 entries.

So the Pillow 10.1.0 change in behaviour is just a continuation of that.

You might find this odd, but given that your package works with a ROM, I'm unsure about the legal status of your package, and correspondingly whether I want to assist you in that.

[valerie-vallet]

The fix you propose @radarhere works! Thanks a lot for your help!

Valérie