Unify model + api versions in config
schuderer opened this issue · 2 comments
In the following, please keep in mind that we are talking about versioning a product built with mllaunchpad, and not the versioning of mllaunchpad itself.
I don't think that is is sensible from a user perspective to have different versions for the model and api.
While one can argue that models and apis might change independently, when regarded from a semver perspective, the impact of minor/major changes should be the same.
Starting from the basics, there are two "kinds" of versions, one describing the code's life cycle, and one describing de ML model's life cycle.
- Code life cycle: Standard semver practices apply. When something changes so that existing usage of the interface (that is, the API and/or the datasources/sinks) might break, the major version is increased. Minor versions and patches are dealt with the standard semver way.
- Model life cycle: It's not too certain what would constitute a breaking change that would affect the API, but not the model... Maybe if the model stops working (not really serious)... One could try to draw the distinction at making the model responsible for the datasources, and the API only for the API, but IMO that seems like splitting hairs. So, essentially, when forcing everything else to keep the same, the model life cycle in isolation can only ever add (hidden) functionality (bumps minor version), or patch stuff (bumps patch version).
What re-training does to the model version is another question:
- One answer could be to just bump the patch version (after all, re-training with up-to-date data could be regarded as being analogous to project maintenance like updating dependencies or adding compatibility with the newest version of python).
- But when the model+api is actually in a deployed state, it would be very strange (and potentially cause chaos) if the version of the deployed product changes when retraining! It would be like having a "hidden deployment" with no artifacts to show for it. Hmm, maybe we still need some separate model version after all, but... sigh, see above.
Maybe the retrained model(s) should be identified by their date, then there can be only one version string for the whole of the product? Re-training by developers could still bump the patch version (there will be another deployment if this happens, so this would make sense).
As this will be a breaking change, we should deprecate the separate version numbers ASAP and try to get this into the next major release (1.0.0).