attention model
HaminyG opened this issue · 3 comments
HaminyG commented
Excellent job!! Cause I am a fresh bird in this area, could you please tell me what is l and g which are as inputs for linearattentionblock?
0aqz0 commented
I add the attention blocks and refer to this paper: Learn to Pay Attention(ICLR2018). And this figure shows the architecture in the paper.
To my understanding, l represents features extracted from different layers and g represents features extracted from the final layer.
You can check out this paper. In my job, I try to add similar attention blocks to 3D ResNet. But the result now is a bit worse than 3D ResNet without attention :) and I am still tuning the model.
HaminyG commented
Thanks so much. This figure makes more sense.
发自我的iPhone
…------------------ Original ------------------
From: Haodong Zhang <notifications@github.com>
Date: Mon,Mar 23,2020 7:57 PM
To: 0aqz0/SLR <SLR@noreply.github.com>
Cc: HaminyG <992532036@qq.com>, Author <author@noreply.github.com>
Subject: Re: [0aqz0/SLR] attention model (#1)
I add the attention blocks and refer to this paper: Learn to Pay Attention(ICLR2018). And this figure shows the architecture in the paper.
To my understanding, l represents features extracted from different layers and g represents features extracted from the final layer.
You can check out this paper. In my job, I try to add similar attention blocks to 3D ResNet. But the result now is a bit worse than 3D ResNet without attention :) and I am still tuning the model.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
0aqz0 commented
You're welcome.