knulli-cfw/distribution

"Clean Gamelists & Remove Unused Media" removes new media

Opened this issue · 3 comments

martin-braun commented

Knulli build version

40-dev-d3b520f949 2024/07/21

Your architecture

RG35XX

Issue description

If I scrape data for a new game and run "Clean Gamelists & Remove Unused Media" afterwards, the images and videos from that game get deleted again.

Only if I restart the system, those data is persisted. This however shouldn't be necessary.

Detailed reproduction steps

  • Install a new game
  • Scrape data for that game (ScreenScraper)
  • Go Settings / System Settings / Frontend Developer Settings / Clean Gamelists & Remove Unused Media
  • Go back and see how the media that just got scraped is lost, other metadata like region settings are persisted though

Details of any attempts to fix this yourself

Restarting after scraping anything really helps to not have this happening, but it's sub-optimal.

Details of any modifications you have made to Knulli.

I really just setup SSH and Syncthing. I can also see how the files are created after scraping, but then get deleted again when running "Clean Gamelists & Remove Unused Media".

Logs and data

No response

HoroTW commented

For what its worth it should also not happen when you do the following:

  • Install a new game
  • Scrape
  • Main Menu->Game Settings->Update Gamelists
  • Settings / System Settings / Frontend Developer Settings / Clean Gamelists & Remove Unused Media

But I see that it would be good to trigger a meta data update after the scrape.

martin-braun commented

@HoroTW Ah so "Update Gamelist" writes the gamelist.xml files from the data in memory and "Clean Gamelists & Remove Unused Media" probably parses the gamelist.xml and some scraped data might not be serialized in that spot when "Update Gamelist" is not called.

That's so hard to understand, intuitively. I'm a software engineer and it took me a while to understand what is happening in the background. Also the fact, that I cannot edit gamelist.xml files on another device without disabling the saving of metadata when shutting off the system, because it would overwrite my gamelist.xml from memory when I restart the device, which would be necessary to load any changes I made from another device I sync to.

My suggestion

  • If not done (I couldn't find it), explain the options in the Wiki properly. What is really happening in the background, how is gamelist.xml / game data serialized. Explain that metadata is only held in memory and how "Update Gamelist" will write it to storage for persistence
  • Let "Create Gamelist & Remove Unused Media" call "Update Gamelist" first, so that it does not accidentally remove data that hasn't been serialized to gamelist.xml files already
HoroTW commented

I agree and can see how this is not intuitive!
I also only inferred it because of the following settings/behaviors:

  • Displaying "save metadata" on shutdown
  • The "Save metadata on exit" option
  • The "Fast Shutdown System" button, which differs from the normal "Shutdown System" in that it reads "does not save metadata")

I think a PR to the wiki adding a description of this behavior/operation principle would be welcome 😉
Your second point would probably also be a nice addition, I'd argue that this could be expected behavior, so yeah, I'm for it! 😊