some questions about scaling

[miganchuanbo]

DALLE2-pytorch/dalle2_pytorch/dalle2_pytorch.py

Line 869 in 00e07b7

def forward(self, x, mask = None, attn_bias = None):

It seems we need to scale up Q and K when using cosine sim. But what is the reason for scaling Q before applying rotary emb?