Standardize PyTorch 2.x features config in model-config.yaml
agunapal opened this issue ยท 1 comments
agunapal commented
๐ The feature
Define the following structure for a PyTorch 2.x feature config in model-config.yaml
pt2:
<API name>:
enable: True
option1: value1
optionN: valueN
Example:
For torch.compile
, we would specify the following options
pt2:
compile:
enable: True
backend: "inductor"
mode: "max-autotune"
Motivation, pitch
When torch.compile
was integrated with TorchServe, there was no torch.export
and torch.ao
We had only the following config in TorchServe for torch.compile
pt2: "inductor"
This was later modified to support other torch.compile
options
pt2 : {backend: inductor, mode: reduce-overhead}
With torch.export
, torch.ao
being included in TorchServe, there is a need to clearly define the interface to make the user experience intuitive for customers. There is also a need to make the design consistent for any new/future PyTorch 2.x feature.
Alternatives
No response
Additional context
No response
agunapal commented
Done