uber/neuropod

Save platform version that were used to train model and generate "binary" data

vkuzmin-uber opened this issue · 2 comments

Feature

Add metadata attribute "native platform version" as indirect specification for model's data format and platform that was used to generate data.

Tensorflow and Torchscript models has binary data of saved model, Proto and Pt respectively. We have no metadata to identify compatibility if there are many backends available (NEUROPOD_BASE_DIR path has many versions) and model has no pinned version on platform_version_semver.

Note, even platform_version_semver could be used as identity, in fact it isn't designed as identifier and "*" is default for most of use cases.

ModelConfig{name, platform, platform_version_semver, inputs, outputs, custom_ops, input_tensor_device});

:param  platform_version_semver: 
     The versions of the platform (e.g. Torch, TensorFlow, etc)
     that this model is compatible with specified as semver range.
     See https://semver.org/, https://docs.npmjs.com/misc/semver#ranges
     or https://docs.npmjs.com/misc/semver#advanced-range-syntax for
     examples and more info. Default is `*` (any version is okay).

     When this model is loaded, Neuropod will load it with a backend
     that is compatible with the specified versions ranges or throw an
     error if no compatible backends are installed. This can be used to
     ensure a model always runs with a particular version of a framework.

     !!! note ""
         ***Example***:
         `1.13.1` or `> 1.13.1` or `1.4.0 - 1.6.0`

Is your feature request related to a problem? Please describe.

If platform breaks backward compatibility in future, say TF3 won't support V1, Neuropod Core may maintain "compatibility table" but it should be able to identify model's format first that isn't possible now.

Describe the solution you'd like

Add new attribute to config.json, string, required "data_platform_version" or other name that makes it clear. We should try to set it automatically during "model.save" or extract from data if available during create_*_neuropod call.

Describe alternatives you've considered

  • Use platform_version_semver
  • Use metadata from data (if this info available, in header or so?)

Additional context

This also can be useful for bugreport, debug and inventory/history.

I think we should use platform_version_semver as that was its intended purpose.

I agree that the current default value of * isn't ideal. platform_version_semver is new in v0.3.0-rc1 so we can change the default to something more reasonable without it being a breaking change

Maybe we should set the default to the packaging-time framework version and print out an info message during packaging if the user doesn't explicitly pick a required version range.

For example, if the user is running the packaging code with TF 1.15.0, we'll use 1.15.0 as the default value for platform_version_semver and let the user know that they can override it if they want to support a wider range of backends.

For example, if the user is running the packaging code with TF 1.15.0, we'll use 1.15.0 as the default value for platform_version_semver and let the user know that they can override it if they want to support a wider range of backends.

In this case, what would be the behavior if the packaging stack uses version 1.13.0, but on serving, a later backend version is installed, say 1.15.0? Would Neuropod be expected to respect semver semantics here and use version 1.15.0 on serving?