Compress NES Linearity better

[pinobatch]

Right now the biggest test by far is Linearity because it encodes four distinct images that don't compress well using PB53 map. Their total compressed data size is 9617 bytes.

• Need at least two, as pixel aspect ratio differs on NTSC and PAL.
• Linearity and Linearity with grid are separate pictures that share no data, unlike on GBC and GBA where the grid is added at runtime.
• Linearity with grid in NTSC doesn't have a lot of horizontal runs because its background is a 56x8-pixel (7x1-tile) pattern.

The plan is

• In the map, encode tiles seen for the first time more efficiently, as Linearity uses essentially no horizontal repeats of nonblank tiles.
• Encode tiles seen before with fewer bits, especially early in the map data.
• Encode the same tile multiple times if it appears in positions with different phases against the repeating grid. This allows adding the grid in post-processing.

The steps are

• Make a unique tiles finder that respects grid phase
• Make a bit packer in Python
• Pack using 0 for a blank tile, 10 for a new tile, or 11xxxxx to repeat an existing nonblank tile (variable count of X's based on log2 of number of nonblank tiles so far)
• 6502 decoder for nametables in this format
• Create grid pattern table from nongrid pattern table using nametable readback to determine phase for each tile number
• Integrate this into the main executable

Because of the lack of space in the Suite's fixed bank, I may have to prototype new Linearity in a separate folder. But I fully expect to save 5K with this technique, or more if I can apply it to the other SB53 maps in the suite.

[pinobatch]

This will need a tool called sav2iu53 that parallels sav2sb53. So what all will the file need to include?

• Left, top, width, height -- though linearity and sharpness fill the screen, crosstalk, gus_portrait, and lag_clock_face do not
• How many palettes are included? (0-4)
• Is an attribute table included?
• How many tiles need to be decompressed? (unpb53_tiles_left during unpb53_tiles)
• Where in CHR RAM do they start? (PPUADDR during unpb53_tiles and uniu_first_nonblank during uniu)
• How many tile numbers are reserved for nonsequential tiles that appear in the image? (uniu_seen_tiles)