Merge issue when using stack size of 1
MBSMGW opened this issue · 2 comments
When using a stack size of 1, is there an issue with also using skip connections? It just throws an error. I am not sure if it needs to be a bug report or a feature request for automatically deffering to no skip connections for a stack size of 1? anyway, here is the code to reproduce the error:
`def model_create(nb_filters, kernel_size, dropout_rate, lr, dilation):
#if K.backend() == 'tensorflow':
#K.clear_session()
with mirrored_strategy.scope():
model=Sequential()
model = compiled_tcn(return_sequences=return_sequences,
num_feat=num_feat,
num_classes=num_classes,
nb_filters=nb_filters,
kernel_size=kernel_size,
dilations=[2 ** i for i in range(dilation)],
nb_stacks=1,
max_len=segment_length,
output_len=1, # type: int
padding='causal', # type: str
use_skip_connections=use_skip_connections, # type: bool
regression=False, # type: bool
dropout_rate=dropout_rate, # type: float
name=name, # type: str,
kernel_initializer=kernel_initializer, # type: str,
activation=activation, # type:str,
opt=opt,
lr=lr,
use_batch_norm=use_batch_norm,
use_layer_norm=use_layer_norm,
use_weight_norm=use_weight_norm)
return model
model_grid = KerasClassifier(build_fn=model_create, epochs=15, verbose=0)
search_space['nb_filters'] = 12, 14, 16, 18, 20
search_space['kernel_size'] = 3, 5, 16, 64, 128
search_space['dropout_rate'] = 0.01, 0.03, 0.05, 0.10, 0.15
search_space['lr'] = 0.001, 0.005, 0.010, 0.015
search_space['dilation'] = 1, 3, 7, 9
param_grid = dict(nb_filters = search_space['nb_filters'],
kernel_size = search_space['kernel_size'],
dropout_rate = search_space['dropout_rate'],
lr = search_space['lr'],
dilation = search_space['dilation'])
cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=2, random_state=1)
search = HalvingGridSearchCV(estimator = model_grid, param_grid = search_space,n_jobs=1, min_resources="exhaust", factor=3, cv = cv)
result = search.fit(xy_t[0][0], xy_t[0][1], batch_size=512)`
my data is 2400 sequence points long univariate time sequences of vibration data. and the following is the error received when running:
`/usr/local/share/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:614: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details:
Traceback (most recent call last):
File "/usr/local/share/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 593, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File "/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/wrappers/scikit_learn.py", line 223, in fit
return super(KerasClassifier, self).fit(x, y, **kwargs)
File "/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/wrappers/scikit_learn.py", line 157, in fit
self.model = self.build_fn(**self.filter_sk_params(self.build_fn))
File "", line 27, in model_create
use_weight_norm=use_weight_norm)
File "/home/ubuntu/tcn_ed.py", line 422, in compiled_tcn
use_weight_norm, name=name)(input_layer)
File "/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 926, in call
input_list)
File "/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1117, in _functional_construction_call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 258, in wrapper
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
/home/ubuntu/tcn_ed.py:332 call *
x = layers.add(self.skip_connections)
/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/layers/merge.py:771 add **
return Add(**kwargs)(inputs)
/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py:926 __call__
input_list)
/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py:1098 _functional_construction_call
self._maybe_build(inputs)
/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py:2643 _maybe_build
self.build(input_shapes) # pylint:disable=not-callable
/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py:323 wrapper
output_shape = fn(instance, input_shape)
/usr/local/share/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/layers/merge.py:97 build
'Got ' + str(len(input_shape)) + ' inputs.')
ValueError: A merge layer should be called on a list of at least 2 inputs. Got 1 inputs.`
@MBSMGW a stack size of 1 with skip_connections should definitely work! By default the stack_size is 1. Try to use the TCN object instead of using compiled_tcn.
Here is an example that works with compiled_tcn (your case):
keras-tcn/tasks/copy_memory/main.py
Line 23 in 1b2c1a6
@MBSMGW a stack size of 1 with skip_connections should definitely work! By default the stack_size is 1. Try to use the TCN object instead of using compiled_tcn.
Here is an example that works with compiled_tcn (your case):
keras-tcn/tasks/copy_memory/main.py
Line 23 in 1b2c1a6
Yeah great suggestion, really should have embraced it at the start of thesis instead of trying to wrangle the compiled_tcn function (Which is great, especially starting..)
Thanks!