Enable assets manager to run on read-only filesystem when assets do not need to be downloaded
ldeflandre opened this issue · 3 comments
Current state
During asset creation development process, or if you want to embed assets in a container at build time,
Modelkit can be configured to use StorageProvider with LocalStorageDriver and a AssetsDir pointing to the same location of the LocalStorage.
This allow to only use local file and do not duplicate them on local file system.
However, modelkit still requires to have write permission on the filesystem because the management of lock files is not disabled in this condition.
Fix
Idea is to disable the lock files management in this configuration.
It is fine to do that since we are guaranteed that we will not download anything in this configuration
Note that if you already have the assets, you do not need to use a LocalStorageDriver. It should be enough to have only the assets_dir set, and the remote set to none.
As for you fix I think it makes sense, we should just set this behavior with an environment variable.
if you want to embed assets in a container at build time,
BTW, we have a CLI for that modelkit download-assets ...
Note that if you already have the assets, you do not need to use a LocalStorageDriver. It should be enough to have only the assets_dir set, and the remote set to none.
Ah nice indeed.
The lock management is still getting in the way but it is exactly what I wanted to do mid-term - cool.
So we should probably prevent the configuration of LocalStorageDriver and AssetManager to point to the same location, what do you think ?
As for you fix I think it makes sense, we should just set this behavior with an environment variable.
Just disabling lock mechanism when storageProvider is None seems backward compatible. So no need to add a new env var, right ?