Add command to selectively download games and their files
Opened this issue · 7 comments
I understand that gogcli acts more like a mirror program but it would be nice if there was a command to download only selected game(s) and possibly only selected files within those games.
For example I may only want to download the extras of a game which I have already downloaded previously, or only download the installer with required patches instead of all patches.
The tool may or may not provide what you need here depending on what your exact needs are.
- If you just need to download some game files that you will throwaway afterwards
Once you have generated your full manifest, you can use the "gogcli manifest search" command to generate a sub-manifest with only the games and game files you need.
From there, you just need to specify a filesystem storage in a temporary folder and you'll be able to download what you need
- If you just want to update your backup what GOG.com indicates has changed
You can run "gogcli update generate" (fetches the list of updated games) + "gogcli storage download manifest" (fetches the manifest from your storage) + "gogcli manifest update --update=" (updates the manifest with the content of your update file) + "gogcli storage apply" (applies your modified manifest on your storage)
3, If you just want to update/add a specific game in your manifest
You can use the same list of commands as above, except that in the "gogcli manifest update" command, you'll use the following flags instead "gogcli manifest update --id=<id of first game you want to update/add> --id=<id of the second game you want to update/add> ..."
You can also mix it and use both an update file and an additional list of games to update/add.
- If you need to update specific files of specific games in your backup
That, I do not currently handle. I'd have to add additional flags for this in the "gogcli manifest update" command
btw, if memory serves, with the api endpoints that I use, the extras are easy to sort out (they are marked as extras with an indicator of what kind of extra it is). However, as far as I recall, the installers are a bit of a mess. Installers files are jumbled together with no clear indication of what are installer files and what are patches. That would be a tad more complicated to sort out as GOG.com didn't provide a deterministic convention to figure this out.
In the above case, should I go that far with the tool, I'd rather leave it as input to a human user or script as to how to sort out installers from patches (using patterns in the names for example) rather than hardcode in the tool a loose convention that may have exceptions (ex: maybe you want to say that all files that include the name "patch" are patches, but what happens that day some dev releases a game with the title "Patchman" ?).
Thanks for the cool mini-guide! I was mostly thinking about the first scenario (without the throwaway part), I don't really have the space to download ALL of my library, so I download games as I play them and archive them into my local storage.
It would be nice if there was a way to do that without having to use a sub-manifest... perhaps something basic could be implemented which will download a single game's installers and optionally its extras?
The command line tool was always meant to provide more of a basic building-block composable kind of workflow so it is pretty low-level (conveniently, that decision does make it incredibly easy to troubleshoot many components of the tool in isolation).
I've tried to make it as un-opinionated as I can, though there has to be constraints there to keep the complexity of the implementation manageable (I've stuck to a "manifest as source of truth for your storage" declarative model), although applying the synchronization process between the manifest and storage in an incremental way is definitely feasible (technically, the "storage apply/resume" commands already allow you to apply your games to storage individually or in batches and it wouldn't be that far-fetched to extend the functionality to also partially apply individual games, selecting only certain files).
I've always meant for higher-level functionalities to be implemented on top of the lower-level commands using things like scripts, although I've also got a long term plans to implement a command for the tool that launches a server serving a RESTful api and provide a more user-friendly web dashboard (that would call the RESTful api) that allows you to manage your games.
For your particular use-case of only downloading one game at a time (the first time ever you download each game), it is already supported in the "storage apply/resume" commands though (just, not to the granularity where it will only download individual files in the game... if you want to only download certain kinds of files, this is currently only supported when you initially generate the manifest).
If you only want to download a certain game, you run the following the first time ever that you apply your manifest "gogcli storage apply --maximum=1 --preferred-ids=< id of your game > --concurrency=1 ...".
Subsequent times, you run "gogcli storage resume --maximum=1 --preferred-ids=< id of your game > --concurrency=1 ..."
If your partially applied manifest gets dated because gog updates stuff, I'll refer you to those instructions: https://github.com/Magnitus-/gogcli#updating-your-storage-with-gogcom-when-you-have-pending-actions
I guess the documentation could benefit from a bit of an architectural overhead view to explain how the components fit together (GOG.com, storages, manifests, action files and update files). I'll add that to the doc.
I think most users will end up using a tool on top of this client and I already have the design in my head for my upcoming web dashboard user interfacer, but I guess DIYers who want to script something could use the clarification.
@TheDcoder Finally got around to writing a bit of architecture documentation: https://github.com/Magnitus-/gogcli/tree/main/architecture-documentation
It will not add absent functionality, but hopefully, it will assist in better leveraging the functionality that is there.
I am using Playnite to manage my games library and want to integrate gogcli commands as in-app-actions for the following situations:
- add a single game to the backup
- remove a single game from the backup
- update backups of currently backuped games
1 and 3 should work somehow. But how can I remove a single game completely from the backup?
As I understand it, the trim funktion is only for patches. Or does it also work with the installer files?
If there is no solution to 2. at the moment, a function to delete a single game from the manifest/storage by stating its id would be great.
I believe there is indeed no direct command currently to remove a game. You'd have to:
- Run the command to get the manifest
- Remove a game from the manifest
- Apply the manifest
I guess it is simple enough to implement and if it facilitates integration with another tool, it seems like good value for low effort.