kyle-dorman/bayesian-neural-network-blogpost

Unknown entry in loss dictionary: "logits_variance".

leedrake5 opened this issue · 1 comments

model.compile(optimizer=optimizers.Adam(lr=1e-3, decay=0.001), loss={'logits_variance': bayesian_categorical_crossentropy(100, 10), 'softmax_output': 'categorical_crossentropy'}, metrics={'softmax_output': metrics.categorical_accuracy}, loss_weights={'logits_variance': .2, 'softmax_output': 1.})

returns the "logits_variance" issue in the title, or the following:

TypeError: Expected float32, got <tile.Value ZerosLike FLOAT32(<tile.Value Ceiling UINT64()>, -7)> of type 'Value' instead.
...after many file path dumps in the error.

I've tried using the code here, but I keep running into this. Scanning the code, I'm unclear of where this is defined. Any advice would be helpful. I've tried defining my own model, I've also tried generating a model as follows:

model=create_bayesian_model('resnet50', (400, 400, 3), 2)

Been working on this for a couple days, I think I've resolved part of the error by defining terms. But I keep running into this once an epoch opens for a bnn:

ValueError: Error when checking input: expected input_8 to have 2 dimensions, but got array with shape (32, 400, 400, 3)

I think the problem is in the header loading from resnet50 in the original data used. Which is likely due to an argument or default changing in tensorflow/keras versions since this project was last updated 2 years ago. Will report here if I find a solution.

Update - this is the case. Just using the resnet50() with input dimensions as the only argument throws this warning:

UserWarning: The output shape of ResNet50(include_top=False) has been changed since Keras 2.2.0.
warnings.warn('The output shape of ResNet50(include_top=False)

The code is as follows:

def resnet50(input_shape):
	input_tensor = Input(shape=input_shape)
	base_model = ResNet50(include_top=False, weights='imagenet', input_tensor=input_tensor)
	# freeze encoder layers to prevent over fitting
	for layer in base_model.layers:
		layer.trainable = False
	output_tensor = Flatten()(base_model.output)
	return Model(inputs=input_tensor, outputs=output_tensor)

So include_top is what needs to be updated.