feature_dim in muti_head_attention

[zjzzzz]

I wonder if 'feature_dim' could be assigned by human. In your code, given the input, 'feature_dim' is fixed, so that the shape of 'Wq','Wk','Wv' is fixed.

def build(self, input_shape): if isinstance(input_shape, list): q, k, v = input_shape else: q = k = v = input_shape feature_dim = int(v[-1])
self.Wq = self.add_weight( shape=(int(q[-1]), feature_dim), self.Wk = self.add_weight( shape=(int(k[-1]), feature_dim), self.Wv = self.add_weight( shape=(int(v[-1]), feature_dim),

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.