wbhu/DnCNN-tensorflow

the initialization of weights ?

lizhengwei1992 opened this issue · 1 comments

hi, it is a good job for the work. I want to know the meaning of initialization about ''get_conv_weights''''get_bn_weights", like this:
`def get_conv_weights(weight_shape, sess, name="get_conv_weights"):
return math.sqrt(2 / (9.0 * 64)) * sess.run(tf.truncated_normal(weight_shape))

def get_bn_weights(weight_shape, clip_b, sess, name="get_bn_weights"):
weights = get_conv_weights(weight_shape, sess)
return clipping(weights, clip_b)

def clipping(A, clip_b, name="clipping"):
h, w = A.shape
for i in xrange(h):
for j in xrange(w):
if A[i,j] >= 0 and A[i,j] < clip_b:
A[i,j] = clip_b
elif A[i,j] > -clip_b and A[i,j] < 0:
A[i,j] = -clip_b
return A
`

math.sqrt(2 / (9.0 * 64)) refers to the He initialization as shown in http://arxiv.org/abs/1502.01852. 3^2*64 is the number of parameters of 64 filters with 3x3 dimension.
Don't know anything about clipping though