question about diffusion

[llllly26]

Hi, i am interested in this paper and focus on the diffusion application of protein structure prediction. Now, i have a confusion about diffusion model on side chain packing. the diffusion model can reconstruct the correct angles, sampling noisy angles from prior distribution. if i sampling different noise for the same angles and the same sequence, such as \chi_2， the noise is different, but the final angles for this sequence should be fixed, different noise can restore the same angles for the sequence? maybe my thought was wrong, i don't understand, could u please answer to me? thanks!

[Oxer11]

Hi, thanks for your interest in our work! Your understanding is correct. For different initial noise, we will generate different angles. Then, we use a confidence model to select the best prediction. For the advantages of generative models over regression models, you can refer to the DiffDock paper.

[llllly26]

oh thanks! the truth is, the generated angles will be different when sampling different noise for the same sequence, this is the same as what i considerd diffusion model. and after this, there is a step that select from these results to get a best prediction. emmm, i have a new question that,

• in this way, for a sequence, it needs several times to use diffusion model to generate different angles, and use a confidence model to select the best one ? instead of just using one time to generate angles? i guess the inference result of one time is: [ generated angles, a confidence score], right?
• where is the detail of the confidence model, i am sorry that i don't notice it in paper, maybe i ignore it.
  thanks!

[Oxer11]

1. Yes, your understanding is correct. In principle, the inference result should be [generated angles, a confidence score]. In practice, there may be some differences in the implementation.
2. The confidence model also uses GearNet-Edge as the encoder and an MLP to predict the RMSD. The details of implementation can be found here

   DiffPack/diffpack/task.py

   Lines 224 to 250 in 81ee516

   	class ConfidencePrediction(TorsionalDiffusion):
   	eps = 1e-10
   	_option_members = {"task", "criterion", "metric"}
   	def __init__(self, sigma_embedding: nn.Module,
   	model: nn.Module,
   	confidence_model: nn.Module,
   	torsion_mlp_hidden_dims: list,
   	schedule_1pi_periodic: SO2VESchedule,
   	schedule_2pi_periodic: SO2VESchedule,
   	num_sample: int = 5,
   	num_mlp_layer: int = 1,
   	graph_construction_model: Optional[Any] = None,
   	verbose: int = 0,
   	train_chi_id: Optional[Any] = None):
   	super().__init__(sigma_embedding,
   	model,
   	torsion_mlp_hidden_dims,
   	schedule_1pi_periodic,
   	schedule_2pi_periodic,
   	graph_construction_model,
   	verbose,
   	train_chi_id)
   	self.confidence_model = confidence_model
   	self.num_sample = num_sample
   	self.mlp = layers.MLP(self.confidence_model.output_dim,
   	[self.confidence_model.output_dim] * num_mlp_layer + [1])

   
   The corresponding config is listed here
   

   DiffPack/config/inference_confidence.yaml

   Lines 32 to 42 in 81ee516

   	confidence_model:
   	class: GearNet
   	input_dim: 39
   	hidden_dims: [ 128, 128, 128, 128, 128, 128 ]
   	batch_norm: True
   	concat_hidden: True
   	short_cut: True
   	readout: 'sum'
   	num_relation: 6
   	edge_input_dim: 58
   	num_angle_bin: 8

[llllly26]

1. Yes, your understanding is correct. In principle, the inference result should be [generated angles, a confidence score]. In practice, there may be some differences in the implementation.

2. The confidence model also uses GearNet-Edge as the encoder and an MLP to predict the RMSD. The details of implementation can be found here

   DiffPack/diffpack/task.py

   Lines 224 to 250 in 81ee516

   	class ConfidencePrediction(TorsionalDiffusion):
   	eps = 1e-10
   	_option_members = {"task", "criterion", "metric"}
   	def __init__(self, sigma_embedding: nn.Module,
   	model: nn.Module,
   	confidence_model: nn.Module,
   	torsion_mlp_hidden_dims: list,
   	schedule_1pi_periodic: SO2VESchedule,
   	schedule_2pi_periodic: SO2VESchedule,
   	num_sample: int = 5,
   	num_mlp_layer: int = 1,
   	graph_construction_model: Optional[Any] = None,
   	verbose: int = 0,
   	train_chi_id: Optional[Any] = None):
   	super().__init__(sigma_embedding,
   	model,
   	torsion_mlp_hidden_dims,
   	schedule_1pi_periodic,
   	schedule_2pi_periodic,
   	graph_construction_model,
   	verbose,
   	train_chi_id)
   	self.confidence_model = confidence_model
   	self.num_sample = num_sample
   	self.mlp = layers.MLP(self.confidence_model.output_dim,
   	[self.confidence_model.output_dim] * num_mlp_layer + [1])

   The corresponding config is listed here
   

   DiffPack/config/inference_confidence.yaml

   Lines 32 to 42 in 81ee516

   	confidence_model:
   	class: GearNet
   	input_dim: 39
   	hidden_dims: [ 128, 128, 128, 128, 128, 128 ]
   	batch_norm: True
   	concat_hidden: True
   	short_cut: True
   	readout: 'sum'
   	num_relation: 6
   	edge_input_dim: 58
   	num_angle_bin: 8

thanks! I will check the code to further enhance my understanding.