custom training error
ssetty opened this issue · 5 comments
When I try to give images of an aged person -> response comes in mid-age range < 50 years should have been 70 years. I attempted to custom train 'inception' model with images of 87 years but fails with exception. I am not clear about "fold_dir", " data_dir". Kindly help
below are commands
python preproc.py --fold_dir
/home/setsat2011/software/agegender/AgeGenderDeepLearning/Folds/train_val_txt_files_per_fold/test_fold_is_0 --train_list lata.txt --valid_list lata.txt --data_dir /home/setsat2011/software/images --output_dir /home/setsat2011/age_test_fold_is_0
python train.py --train_dir /home/setsat2011/age_test_fold_is_0 --max_steps 15000 --model_type inception --batch_size 32 --eta 0.001 --dropout 0.5 --pre_model /home/setsat2011/software/inception/inception_v3.ckpt
Caused by op 'cross_entropy_per_example/cross_entropy_per_example', defined at:
File "train.py", line 192, in
tf.app.run()
File "/home/setsat2011/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "train.py", line 122, in main
total_loss = loss(logits, labels)
File "train.py", line 93, in loss
logits=logits, labels=labels, name='cross_entropy_per_example')
File "/home/setsat2011/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 1693, in spar
se_softmax_cross_entropy_with_logits
precise_logits, labels, name=name)
File "/home/setsat2011/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 2491, in
_sparse_softmax_cross_entropy_with_logits
features=features, labels=labels, name=name)
File "/home/setsat2011/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line
767, in apply_op
op_def=op_def)
File "/home/setsat2011/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2630, in c
reate_op
original_op=self.default_original_op, op_def=op_def)
File "/home/setsat2011/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1204, in _
init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Received a label value of 87 which is outside the valid range of [0
, 1). Label values: 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 87 8
7
[[Node: cross_entropy_per_example/cross_entropy_per_example = SparseSoftmaxCrossEntropyWithLogits[T=DT_FLO
AT, Tlabels=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](output/output, batch_processing/Reshape_1)]]
setsat2011@
Check your md.json file. It looks like preproc.py found only 2 classes, not 88
This probably means whatever training data you are using is messed up
Thanks dpressel.
Please send me instructions for custom training. I need to train aged people > 70 years.
Regards
Please help/ any hint or suggestion I can try. Thanks
I do not have enough information to help you with your specific problem. In general, you need 1) training data covering the labels you wish to classify, and 2) you need to update the last layer of this network to support the range of labels you want.
IMO there is no substitute for reading and understanding the code, its not particularly complicated. Once you get whats going on, its fairly simple to make this modification, assuming (again) you have training data to cover this.