Quality Improvement
TDS4874 opened this issue · 6 comments
Using DDIM, I found that aligning the alphas_cumprod tensor with the original Animatediff repository improves the quality. I haven't changed the way alphas_cumprod is calculated itself, so a proper implementation requires expertise. This is beyond my current capabilities. Could someone knowledgeable please correct it? Thank you.
Before and after the improvement. The generation conditions are identical for both.
It would be extremely helpful if you could provide a skeleton code and I can look into how to hack A1111.
new_schedule.txt
new_schedule of alphas_cumprod when you run original Animatediff is outputed by changing scheduling_ddim.py in diffusers.
And I changed make_schedule method of DDIMSampler class in repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddim.py of webui repo like below.
def make_schedule(self, ddim_num_steps, ddim_discretize="uniform", ddim_eta=0., verbose=True):
self.ddim_timesteps = make_ddim_timesteps(ddim_discr_method=ddim_discretize, num_ddim_timesteps=ddim_num_steps,
num_ddpm_timesteps=self.ddpm_num_timesteps,verbose=verbose)
alphas_cumprod = self.model.alphas_cumprod
#insert to change schedule
with open('new_schedule.json', 'r') as f:
new_schedule_list = json.load(f)
new_schedule_tensor = torch.tensor(new_schedule_list)
if new_schedule_tensor.shape == alphas_cumprod.shape:
new_schedule_converted = new_schedule_tensor.to(dtype=alphas_cumprod.dtype, device=alphas_cumprod.device)
alphas_cumprod = new_schedule_converted
#insert to change schedule
assert alphas_cumprod.shape[0] == self.ddpm_num_timesteps, 'alphas have to be defined for each timestep'
to_torch = lambda x: x.clone().detach().to(torch.float32).to(self.model.device)
import torch
beta_start = 0.00085
beta_end = 0.012
# beta_schedule = "linear"
num_train_timesteps = 1000 # default
betas = torch.linspace(beta_start, beta_end, num_train_timesteps, dtype=torch.float32)
alphas = 1.0 - betas
alphas_cumprod = torch.cumprod(alphas, dim=0)
Thank you very much for your contribution. I have merged the gist into v1.3.0
牛的牛的,只剩下75负面提示词了