warmspringwinds/tf-image-segmentation

TF AdamOptimizer dimension size error.

ahundt opened this issue · 6 comments

ahundt commented

I'm implementing a couple new fcn networks for testing purposes, but for some reason I'm getting dimension errors when I call tf.train.AdamOptimizer in densenet_fcn.py on the model.

Here is the error:

 tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.7335
pciBusID 0000:02:00.0
Total memory: 7.92GiB
Free memory: 256.12MiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:02:00.0)
creating densenet model
(?, 384, 384, 21)
Running with tf training, initializing batches...
Traceback (most recent call last):
  File "densenet_fcn.py", line 132, in <module>
    train_step = tf.train.AdamOptimizer(learning_rate=0.0001).minimize(cross_entropy_sum)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 269, in minimize
    grad_loss=grad_loss)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 335, in compute_gradients
    colocate_gradients_with_ops=colocate_gradients_with_ops)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gradients_impl.py", line 482, in gradients
    in_grads = grad_fn(op, *out_grads)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_grad.py", line 348, in _ReshapeGrad
    return [array_ops.reshape(grad, array_ops.shape(op.inputs[0])), None]
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 2448, in reshape
    name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2242, in create_op
    set_shapes_for_outputs(ret)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1617, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1568, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
    debug_python_shape_fn, require_shape_fn)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Dimension size must be evenly divisible by 3096576 but is 147456 for 'adam_vars/gradients/Reshape_27_grad/Reshape' (op: 'Reshape') with input shapes: [384,384], [4].

have you seen anything like it or have advice on fixing the dimensions?

ahundt commented

Specifically 3096576/147456 = 21 which is the number of classes I have. Should I add something to remap the 21 classes into a single channel? Is there an existing function to do that?

ahundt commented

fixed it, wasn't using the result of get_valid_logits_and_labels

linus007 commented

i meet the the same question, how did u solve it?

B-Xi commented

I meet the the a question like that when I use MomentumOptimizer , how did u solve it?

whaoer commented

i meet the the same question：
ValueError: Dimension size must be evenly divisible by 200704 but is 1179648 for 'Reshape_1' (op: 'Reshape') with input shapes: [128,6,6,256], [2] and with input tensors computed as partial shapes: input[1] = [?,200704].
how did u solve it?

adelra commented

Can someone please explain how to solve this issue?

  • 👍3