Questions about gated query transformation

[fzb408]

Thank you for writing such an excellent paper. Where in the code is the operation of the query gate mentioned in your paper？

[xinlong-yang]

Thanks for your interest. The relevant part is in 'BertLayer' class in med.py. Since you get 'layer_output' (crossattention output) and 'residual_text' (scale&shift selfattention output), you can use the following code to perform gated query transformation:

g = torch.softmax(torch.sum(torch.matmul(layer_output,residual_text.permute(0, 2, 1)),dim=-1),dim=1).unsqueeze_(2)
layer_output = layer_output * g + (1 - g) * residual_text

[fzb408]

Thank you very much for your help！

…

------------------ 原始邮件 ------------------ 发件人: "WillDreamer/Aurora" ***@***.***>; 发送时间: 2023年11月17日(星期五) 晚上8:01 ***@***.***>; ***@***.***>;"State ***@***.***>; 主题: Re: [WillDreamer/Aurora] Questions about gated query transformation (Issue #4) Thanks for your interest. The relevant part is in 'BertLayer' class in med.py. Since you get 'layer_output' (crossattention output) and 'residual_text' (scale&shift selfattention output), you can use the following code to perform gated query transformation: g = torch.softmax(torch.sum(torch.matmul(layer_output,residual_text.permute(0, 2, 1)),dim=-1),dim=1).unsqueeze_(2) layer_output = layer_output * g + (1 - g) * residual_text — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[Arsiuuu]

@fzb408 Hello~ have you reproduced successfully?

[fzb408]

你好~你复制成功了吗？

您好，我没有复制成功

[Arsiuuu]

你好~你复制成功了吗？

您好，我没有复制成功

没有复现出结果吗，是图像还是视频

[fzb408]

新一期

我想要把门限查询应用到另一个代码中没有成功，报出了loss.backward的错误