ZFTurbo/Keras-inference-time-optimizer

Does not work in Callback Function

MaximumProgrammer opened this issue · 2 comments

This code does not work in Callback Functions

File "Pipe_Classificator_Train.py", line 1512, in
main_train(P_NUM_CLASSES, P_IMG_SIZE, P_BATCH_SIZE, P_EPOCHS, P_VALIDATION_SPLIT, P_RELOAD, P_MAX_ELEMENTS_EACH_LABEL, P_NORMALIZE, P_USE_LIB, P_INTERPOLATION_METHOD, P_INPUT_DIRECTORIES, P_PRE_PROCESS_METHOD)
File "Pipe_Classificator_Train.py", line 1017, in main_train
class_weight=class_weights
File "/home/development/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1042, in fit
validation_steps=validation_steps)
File "/home/development/.local/lib/python2.7/site-packages/keras/engine/training_arrays.py", line 219, in fit_loop
callbacks.on_epoch_end(epoch, epoch_logs)
File "/home/development/.local/lib/python2.7/site-packages/keras/callbacks.py", line 77, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "Pipe_Classificator_Train.py", line 212, in on_epoch_end
keras_model_reduced = reduce_keras_model(keras_model)
File "/usr/local/lib/python2.7/dist-packages/kito/init.py", line 311, in reduce_keras_model
tmp_model = Model(inputs=input, outputs=input)
File "/home/development/.local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/development/.local/lib/python2.7/site-packages/keras/engine/network.py", line 91, in init
self._init_graph_network(*args, **kwargs)
File "/home/development/.local/lib/python2.7/site-packages/keras/engine/network.py", line 165, in _init_graph_network
layer, node_index, tensor_index = x._keras_history
UnboundLocalError: local variable 'x' referenced before assignment

The callback function

class My_Callback_CNTK_2(keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs={}):
if(mpi_rank == 0):
import cntk
temp_export_dir="./files_cntk/"

      keras_model = load_model("./files_keras/model_keras.h5")
      cntk.combine(keras_model.model.outputs).save(temp_export_dir+"model_cntk.dnn")
      
      #https://github.com/ZFTurbo/Keras-inference-time-optimizer
      keras_model_reduced = reduce_keras_model(keras_model)
      max_error = compare_two_models_results(keras_model, keras_model_reduced, test_number=10000, max_batch=128)
      print (max_error)
      cntk.combine(keras_model_reduced.model.outputs).save(temp_export_dir+"model_cntk_reduced.dnn")

It's hard to say what's wrong. I only tested code on Python 3.5 + Tensorflow backend. Are you sure the problem is with Callback? As I can see from Error mesage problem actually with model structure itself. Can you reduce this model outside of Callback?

Hmm it's true this was supposed to be used with CNTK not with Tensorflow, to improve the performance with Keras whereas the backend is CNTK