Craftax pixels square size in code (7) vs. in paper (10)

[matomatical]

The paper explains that Craftax Pixels-based observations comprised of 10x10 squares (emphasis added):

PixelsThe pixel-based observations for Craftax-Classic take the same form as those in Crafter, with each 16x16 square downscaled to 7x7. For Craftax, we downscale only to 10x10. This is because we deal with numbers greater than 9, meaning that the digits had to be rendered in a smaller font size. At 7x7 downscaling many of these digits were indistinguishable. Pixel observations from Crafter, Craftax-Classic and Craftax are shown in Figure 25.

But the source code craftax/craftax/constants.py sets BLOCK_PIXEL_SIZE_AGENT = 7 on line 17.

And I confirmed that the following code produces observations where certain digits are indistinguishable (1 from 5, 2 from 3, 0 from 8 from 9).

import jax
from craftax.envs.craftax_pixels_env import CraftaxPixelsEnv
import matplotlib.pyplot as plt

env = CraftaxPixelsEnv()

rng = jax.random.PRNGKey(seed=0)
obs, state = env.reset(rng)

for i in range(100):
    print(f"rendering {i}...")
    custom_state = state.replace(
        inventory=state.inventory.replace(
            wood=i,
        ),
    )
    rgb = env.get_obs(custom_state)
    print(rgb.shape)
    plt.imshow(rgb)
    plt.show()

I think this means that with the default configuration as-released, Craftax does not conform to the description in the paper, in terms of observation size or digit distinguishability.

[MichaelTMatthews]

Thanks for finding this, yep that's absolutely a bug in the code.
Feel free to make a PR for this and I'll approve it or I can just do it

[MichaelTMatthews]

Fixed, thanks again for finding this!

[matomatical]

Thank you. I am curious if this issue also affected the experiments in the Craftax paper. Were the results (performance, wall clock time etc.) reported in the paper based on observations with block pixel size 7 or block pixel size 10?

[MichaelTMatthews]

All our experiments in the paper were done with symbolic observations, so no this wouldn't affect the results