Include new Video Model
janbl opened this issue · 1 comments
janbl commented
Jason has trained a new model for Videos. Can we get the new Model as well?
Maybe you can also throw some light on the conversion process from the pytorch pth file to your model file and why you have chosen to put some of the file content into code.
GlebSBrykin commented
DeOldify.NET it does not support video processing, so there are no models for video in the list. The conversion of pth to custom files takes place in several stages:
- spectral_form and weight_form are removed from the model in PyTorch (in fact, they are not just removed, but the weights and biases of convolutional layers are calculated).
- All weights from pth are sequentially stored in a binary file without separators. At the same time, C# code is generated to read weights from this file. Since the data in the binary file is not structured, the weights must be read in the same sequence in which they were written. The generated C# code allows you to get a dictionary with named weights (like state_dict in PyTorch).