protein embedding is odd

Question

protein embedding is odd

jxzly opened this issue 4 years ago · 3 comments

the dim=359 of output is so large, mean~7

Answer 1 · 2021-04-14T15:51:45.000Z

Sorry for taking so long to get to this. Not sure I understand the question though. Which dimension of the output is this? What are you taking a mean over?

Answer 2 · 2021-05-12T07:30:20.000Z

For single protein sequence, the embedding is L*D
In dim 359 for D, the distribution of embedding is so large

Answer 3 · 2021-05-12T17:28:30.000Z

I'm having a hard time understanding what you're asking, or what the issue is. I suppose the 359th dimension has high mean? No idea why - training is a random process, and so may result in some odd results occasionally. Not sure the layer norms are optimally placed for this model.

If you have an actual question, let me know...