Conflicting values for Best model; or how to interpet the output
deprekate opened this issue · 1 comments
deprekate commented
Describe the bug
Getting None for some Best Values
Below is the output for the best model, it says that 4 conv_layers is the best, but then has 'None' values for the size and kernal of the layer ("filters_3" and "kernel_3")
Search: Running Trial #16
Value |Best Value So Far |Hyperparameter
4 |4 |conv_layers
88 |80 |filters_0
128 |96 |filters_1
96 |72 |filters_2
120 |None |filters_3
5 |8 |kernels_0
6 |10 |kernels_1
6 |10 |kernels_2
11 |None |kernels_3
1 |3 |dense_layers
0.09 |0.09 |dropout
128 |120 |neurons_0
72 |104 |neurons_1
112 |88 |neurons_2
1 |1 |tuner/epochs
0 |0 |tuner/bracket
0 |0 |tuner/initial_epoch
0 |0 |tuner/round
To Reproduce
This is the code for the HyperModel
class HyperRegressor(kt.HyperModel):
def build(self, hp):
inputs = tf.keras.layers.Input(shape=(99,), dtype=tf.int32)
x = tf.keras.layers.Lambda(lambda x: tf.one_hot(x,depth=6), name='one_hot')(inputs)
for i in range(hp.Int("conv_layers", 2, 6, default=3)):
x = tf.keras.layers.Conv1D(
filters = hp.Int(f"filters_{i}", 72, 128, step=8, default=96),
kernel_size = hp.Int(f"kernels_{i}", 3, 12, step=1, default= 7),
activation = "relu",
padding = "same",
)(x)
x = tf.keras.layers.Flatten()(x)
d = hp.Float("dropout", 0.00, 0.10, step=0.01, default=0.05)
for i in range(hp.Int("dense_layers", 1, 3, default=3)):
x = tf.keras.layers.Dense(
units=hp.Int(f"neurons_{i}", min_value=72, max_value=128, step=8),
activation='relu'
)(x)
x = tf.keras.layers.Dropout(
rate=d
)(x)
outputs = tf.keras.layers.Dense(3, activation='softmax')(x)
model = tf.keras.models.Model(inputs, outputs)
model.compile(loss='categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])
return model
Expected behavior
That None should only show up as a Best Value when that value is not used.
Additional context
Would you like to help us fix it?
Yes
deprekate commented
I figured I can avoid the error with None values by making the default value the highest value:
for i in range(hp.Int("conv_layers", 2, 6, default=6):