Training on Own Dataset
Closed this issue · 6 comments
Hello,
thank you for the work. Is there an easy way to train the Models on my own Dataset, and some new Classes? I have tried it with 9 Classes(created _Classnum), but I get 2 Errors:
python3 finetune.py ucf9 rgb 1
/usr/local/lib/python3.5/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
2018-06-12 16:28:40.489353: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-06-12 16:28:40.576145: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-06-12 16:28:40.576407: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:
name: GeForce GTX 960M major: 5 minor: 0 memoryClockRate(GHz): 1.176
pciBusID: 0000:01:00.0
totalMemory: 3.95GiB freeMemory: 3.42GiB
2018-06-12 16:28:40.576421: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-06-12 16:28:41.162533: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-12 16:28:41.162570: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2018-06-12 16:28:41.162575: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2018-06-12 16:28:41.162758: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3151 MB memory) -> physical GPU (device: 0, name: GeForce GTX 960M, pci bus id: 0000:01:00.0, compute capability: 5.0)
INFO:tensorflow:Restoring parameters from ./data/checkpoints/rgb_imagenet/model.ckpt
----Here we start!----
Output wirtes to output/finetune-ucf9-rgb-1
2018-06-12 16:29:14.904305: W tensorflow/core/framework/op_kernel.cc:1261] Invalid argument: TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/script_ops.py", line 147, in call
ret = func(*args)
File "finetune.py", line 73, in process_video
clip_seq, label_seq = data.next_batch(1, _CLIP_SIZE)
File "/home/chris/Desktop/Action/I3D_Finetune/lib/action_dataset.py", line 33, in next_batch
self.videos[self.perm[i]].get_frames(frame_num, data_augment=data_augment))
File "/home/chris/Desktop/Action/I3D_Finetune/lib/video_3d.py", line 41, in get_frames
frames.extend(self.load_img((i-1)%self.total_frame_num+1))
TypeError: 'NoneType' object is not iterable
2018-06-12 16:29:14.904819: W tensorflow/core/framework/op_kernel.cc:1261] Invalid argument: TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/script_ops.py", line 147, in call
ret = func(*args)
File "finetune.py", line 73, in process_video
clip_seq, label_seq = data.next_batch(1, _CLIP_SIZE)
File "/home/chris/Desktop/Action/I3D_Finetune/lib/action_dataset.py", line 33, in next_batch
self.videos[self.perm[i]].get_frames(frame_num, data_augment=data_augment))
File "/home/chris/Desktop/Action/I3D_Finetune/lib/video_3d.py", line 41, in get_frames
frames.extend(self.load_img((i-1)%self.total_frame_num+1))
TypeError: 'NoneType' object is not iterable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "finetune.py", line 291, in
main(**vars(p.parse_args()))
File "finetune.py", line 230, in main
feed_dict={dropout_holder: _DROPOUT, is_train_holder: True})
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 905, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1140, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1321, in _do_run
run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/script_ops.py", line 147, in call
ret = func(*args)
File "finetune.py", line 73, in process_video
clip_seq, label_seq = data.next_batch(1, _CLIP_SIZE)
File "/home/chris/Desktop/Action/I3D_Finetune/lib/action_dataset.py", line 33, in next_batch
self.videos[self.perm[i]].get_frames(frame_num, data_augment=data_augment))
File "/home/chris/Desktop/Action/I3D_Finetune/lib/video_3d.py", line 41, in get_frames
frames.extend(self.load_img((i-1)%self.total_frame_num+1))
TypeError: 'NoneType' object is not iterable
[[Node: PyFunc = PyFunc[Tin=[DT_STRING, DT_STRING, DT_STRING], Tout=[DT_FLOAT, DT_INT64], token="pyfunc_0"](arg0, PyFunc/input_1, PyFunc/input_2)]]
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[<unknown>, <unknown>], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"](Iterator)]]
[[Node: IteratorGetNext/_467 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_141_IteratorGetNext", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
The First Error appears a dozen times. The second only two times.
I can't find whats the problem here. Maybe u can help me.
Hi @OgLocGreen ,
Thanks for your attention. I am reading your error outputs. Could you please try using python2? I don't know if this is an issue about python version but you can try first.
Hi @OgLocGreen ,
I figure out why the errors occur. In short, in function load_img at here, your image path is wrong so the function return None. Then in function get_frames at here, the frames.extend(None) raise the error "TypeError: 'NoneType' object is not iterable". You should check if you use parameters like ptyhon3 finetune.py dataset rgb 1 for python3 finetune.py dataset flow 1 . Note you must use rgb or flow in the commandline
thank u @vra for the fast respond. I start the code with the right prams python3 finetune.py ucf9 rgb 1. And my rgb.txt is build as described(_GolfSwing_g08_c06 /HDD/OgLocGreen/ucf9/Videos/_GolfSwing_g08_c06 267 0).
When I add a print("self.path",self.path) in I3D_Finetune/lib/video_3d.py:
class Video_3D:
def init(self, info_list, tag='rgb', img_format='frame{:06d}{}.jpg'):
'''
info_list: [name, path, total_frame, label]
tag: 'rgb'(default) or 'flow'
img_format: 'frame{:06d}{}.jpg'(default)
'''
#initialzie,to ensure the int is int
self.name = info_list[0]
self.path = info_list[1]
print("self.path",self.path)
I get lots of: self.path /HDD/OgLocGreen/ucf9/Videos/_GolfSwing_g08_c06 which seems fine.
Later when its getting the Error, it prints between :
self.path b'/HDD/OgLocGreen/ucf9/Videos/_GolfSwing_g08_c06'
Somewhere it adds the b and ' ' to the path but i don't know where and why. Thanks for the time 😊 I install python2 and try it.
Hi @OgLocGreen ,
according to here, the b'xxx' means the varibel has the type of "byte" rather than "string". So you can try to replace these two lines to lines below to check if self.path has the type of "string":
f1 = open(data_info, 'rt')
f2 = open(test_split, 'rt')Also you can try to print variables around here to check what variable is NoneType, since the error message is "TypeError: 'NoneType' object is not iterable".
Hope this helps.
Ty, i have tried with python2 and its working fine!
Hi,How is the training effect?