The keras backend support

[flydsc]

Hi, thank you for this amazing package!

It runs perfectly by only using keras itself

I met backend issue when I was trying to use tf.keras

here is my problem:

Describe the Bug

When I was try to run the demo code which provided in readme file

and I ran the code with environment variable TF_KERAS=1

I got the error as :

 from keras_radam import RAdam
  File "/home/flydsc/.local/lib/python3.6/site-packages/keras_radam/__init__.py", line 1, in <module>
    from .selection import *
  File "/home/flydsc/.local/lib/python3.6/site-packages/keras_radam/selection.py", line 7, in <module>
    from .optimizer_v2 import RAdam
  File "/home/flydsc/.local/lib/python3.6/site-packages/keras_radam/optimizer_v2.py", line 5, in <module>
    from tensorflow.keras import backend_config
ImportError: cannot import name 'backend_config'

Version Info

I was using Tensorflow version 1.13 and I found no backend_config under keras folder any more.

Instead, there exsists a file called back which I suppose is the similar function of back_config.
You may found it from link: https://github.com/tensorflow/tensorflow/blob/r1.13/tensorflow/python/keras/backend.py
In this case, if I modify the code(keras_radam/optimizer_v2.py) as

from tensorflow.python.keras import backend as backend_config

the demo code runs perfectly.

Minimal Codes To Reproduce

import tensorflow as tf
import numpy as np
from keras_radam import RAdam


model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(input_shape=(17,), units=3))
model.compile(RAdam(), loss='mse')


x = np.random.standard_normal((4096 * 30, 17))
w = np.random.standard_normal((17, 3))
y = np.dot(x, w)


model.fit(x, y, epochs=5)

[CyberZHG]

I think the newer releases are compatible with tf-1.13.

[flydsc]

Thank you for your updating! Worked fine with the demo code. Still I got the error when I was trying to replace Adam with RAdam() with following code:

        model.compile(
            loss='categorical_crossentropy',
            # optimizer=tf.keras.optimizers.Adam(lr=0.001),
            optimizer=RAdam(),
            metrics=['accuracy'],)
        model.fit(X, labels, batch_size=batch_size, epochs=epochs, callbacks=[early_stop,scheduler])

Please find the error message below:

Traceback (most recent call last):
  File "classifier.py", line 95, in <module>
    classfier.pre_train()
  File "classifier.py", line 79, in pre_train
    self.train_source_classifier(source_)
  File "classifier.py", line 74, in train_source_classifier
    model.fit(X, labels, batch_size=batch_size, epochs=epochs, callbacks=[early_stop,scheduler])
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 880, in fit
    validation_steps=validation_steps)
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 195, in model_iteration
    f = _make_execution_function(model, mode)
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 122, in _make_execution_function
    return model._make_execution_function(mode)
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1983, in _make_execution_function
    self._make_fit_function()
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1926, in _make_fit_function
    '_fit_function', [self.total_loss] + metrics_tensors)
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1895, in _make_train_function_helper
    params=self._collected_trainable_weights, loss=self.total_loss)
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 379, in get_updates
    return [self.apply_gradients(grads_and_vars)]
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 364, in apply_gradients
    update_ops.append(update_grad_to_var(grad, var))
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 351, in update_grad_to_var
    grad.values, var, grad.indices)
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 693, in _resource_apply_sparse_duplicate_indices
    return self._resource_apply_sparse(summed_grad, handle, unique_indices)
  File "/home/flydsc/.local/lib/python3.6/site-packages/keras_radam/optimizer_v2.py", line 180, in _resource_apply_sparse
    m_t = self._resource_scatter_add(m, indices, m_scaled_g_values)
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 415, in __getattribute__
    raise e
  File "/home/flydsc/.local/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 405, in __getattribute__
    return super(OptimizerV2, self).__getattribute__(name)
AttributeError: 'RAdam' object has no attribute '_resource_scatter_add'

maybe I should follow some rules to do the replacement?

Any thoughts or ideas are appraciated.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ferrydjaja]

I have the same issue here AttributeError: 'RAdam' object has no attribute '_resource_scatter_add'. how can I resolve it?