256x256 FFHQ StyleGAN-V2
eladrich opened this issue · 25 comments
Hi,
I was wondering if you're aware of a pretrained 256x256 FFHQ StyleGANv2 model available? I think it would be really be helpful for tasks where the full model is too big to start with.
A while back I tried to train one starting from an adapted version of the 1024x model, but the transferring process seemed to cripple it a bit, so gave up.
The only one I've seen is in the pytorch stylegan2 repo: https://github.com/rosinality/stylegan2-pytorch But this was trained without the learnable noise parameters, so is a little different to "official" StyleGAN2.
I might try training a 256x256 model on ffhq, but this might take me a little while as my computing resources are limited. If anyone has one already it would be great to know about it!
Interesting, wonder what can make that happen. it seems like the last two scales, and their style vectors, make only minor changes in the output image so wouldn't expect them to affect the training process so much 🤔.
I've actually tried the one in the rosinality repo but it has a worse FID score, maybe the missing noise you mentioned, or some other small variant in the training process.
Hopefully someone will find the time and resources to release a tuned model :-)
I got you. Here's a 256px model (StyleGAN2 config-f) that I derived from the weights for FFHQ released by @NVlabs. All I did was cut off the top few layers of G and D, then finetune the 256x256 layers (especially the toRGB layer). Then I finetuned the whole network for just a few kimg, which helped get a tiny bit of improvement for the perceptual quality of samples.
Download link. Let me know if that works for you, or if you encounter any issues. I haven't tested it out thoroughly or computed FID, and it is possible there are problems with the performance of D.
@aydao could you please explain a bit more, how did you manage to finetune only select (toRGB) layers in F config?
I restricted the trainable variables by modifying the list of trainables for G and D, by filtering using this line here. Only the variables named in self.trainables for a dnnlib network will be trained (e.g., in the tensorflow computation graph, only those vars will be updated during backprop), all others will be "frozen" and remain the same.
For example, I think I used self.trainables = OrderedDict((name, var) for name, var in self.vars.items() if var.trainable and '256x256' in name) or something close to it.
thanks, will give it one more go. i tried modifying this line to freeze D at finetuning [for this] but didn't succeed (can't recall now what was wrong)
Just checked the fid of the model trained above and it's 19.2520
@aydao tried freezing vars again, recalled the question - how did you replace G/D network, loaded from the source pkl, with the modified one (loaded from the code)? did you save the source weights as standard TF checkpoint, or..?
UPD: pardon for stupidity, have been missing resume_with_new_nets param all this time.. fixed now.
OK so after training for a while the best I get for a 256x256 config-e trained from scratch is a FID of
11.2
I'll add it to the repo soon, but for now here is a Google Drive link: https://drive.google.com/open?id=1BUL-RIzXC7Bpnz2cn230CbA4eT7_Etp0
If anyone ever trains a version to a better FID (or a config-f) or tries out the above model for transfer learning then I'd love to hear about it.
For now I'll close this issue.
@aydao Could you explain how you cut off the top few layers of G and D? You mean create a new model in 256x256 resolusion, then load the weights? I can not find a way to load the weights to a new model in lower resolusion in stylegan2. Could you explain how you did this? Thank you in advance.
Just find the solution. Use
Gs_target.copy_vars_from(Gs_source)
@aydao Would you mind elaborating about how you train the ffhq 256 or sharing the code? I can not reproduce the results. You mention that finetune the 256x256 layers (especially the toRGB layer). What do you mean by especially the toRGB layer, you mean first finetune the 256x256 toRGB layers of G and the 256x256 fromRGB layers of D, then finetune the whole layers of 256x256 of both G and D? Did you change the learning rate?
Since you highlight the toRGB layers, I believe you treat them differently compared to regular feature layers.
Thank you for your help.
@betterze I honestly do not remember exactly what I did, however I'm hoping it should be straightforward to figure out. Here's how I would do it again.
Use the scripts in my repo here https://github.com/aydao/stylegan2-surgery specifically create_initial_network_pkl.py to create a freshly initialized StyleGAN2 at 256px resolution, then copy the weights from the source stylegan pkl into that new target pkl using copy_weights_portable.py. At that point, you can simply train on FFHQ and it should learn quickly and stably, no need to freeze or anything.
Since I wanted to keep the pkl I shared above as close to the original weights as possible, I decided to freeze all layers except the 256x256 toRGB and 256x256 fromRGB. I trained (synonymously 'finetuned' / 'transferred') on FFHQ for a few steps. Because the perceptual quality of samples degraded somewhat at that point, I then also trained (finetuned) the whole network (all layers, no freezing) for a few more kimgs, just enough to bring back some perceptual quality.
@aydao Thank you for your detailed reply. It helps a lot. I will try the way you described.
Hi,
I was wondering if you're aware of a pretrained 256x256 FFHQ StyleGANv2 model trained using Tensorflow 2.0?
I was wondering if you're aware of a pretrained 256x256 FFHQ StyleGANv2 model trained using Tensorflow 2.0?
What about the official pre-trained networks?
TensorFlow:
PyTorch:
Isn't this model using TF1?
Isn't this model using TF1?
My bad! You must be right!
@Akila-Ayanthi tf2? is it compatibility mode? Tensorflow has been abandoned by Nvidia labs / no plans to go to tf2
https://github.com/rosasalberto/StyleGAN2-TensorFlow-2.x
@Akila-Ayanthi tf2? is it compatibility mode? Tensorflow has been abandoned by Nvidia labs / no plans to go to tf2
https://github.com/rosasalberto/StyleGAN2-TensorFlow-2.x
Here there is a 1024x1024 model. I want a 256x256 FFHQ because of the limited resources I have.
post issue in that repo.
Or does anyone know of a way to convert StyleGAN2 TF1 256x256 weights to TF2?
I was wondering if you're aware of a pretrained 256x256 FFHQ StyleGANv2 model trained using Tensorflow 2.0?
What about the official pre-trained networks?
TensorFlow:
PyTorch:
Can you tell me where this pytorch weight corresponds to yes code? Thank you.