[KeyError] Pandas raises KeyError when reading landmak from all.csv for hand dataset
littlestudent886 opened this issue · 2 comments
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/pandas/core/indexes/base.py", line 3361, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 2131, in pandas._libs.hashtable.Int64HashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 2140, in pandas._libs.hashtable.Int64HashTable.get_item
KeyError: 4064
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "main.py", line 32, in
Runner(args).run()
File "/content/drive/MyDrive/YOLO_Universal_Anatomical_Landmark_Detection/universal_landmark_detection/model/runner.py", line 152, in run
self.train()
File "/content/drive/MyDrive/YOLO_Universal_Anatomical_Landmark_Detection/universal_landmark_detection/model/runner.py", line 257, in train
self.update_params(epoch, pbar)
File "/content/drive/MyDrive/YOLO_Universal_Anatomical_Landmark_Detection/universal_landmark_detection/model/runner.py", line 294, in update_params
for i, data_dic in enumerate(loader):
File "/content/drive/MyDrive/YOLO_Universal_Anatomical_Landmark_Detection/universal_landmark_detection/model/utils/mixIter.py", line 52, in next
return next(self.cur_iter_list[idx]), idx # todo
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 530, in next
data = self._next_data()
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 570, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/drive/MyDrive/YOLO_Universal_Anatomical_Landmark_Detection/universal_landmark_detection/model/datasets/hand.py", line 49, in getitem
points = self.readLandmark(name, origin_size)
File "/content/drive/MyDrive/YOLO_Universal_Anatomical_Landmark_Detection/universal_landmark_detection/model/datasets/hand.py", line 65, in readLandmark
li = list(self.labels.loc[int(name), :])
File "/usr/local/lib/python3.7/dist-packages/pandas/core/indexing.py", line 925, in getitem
return self._getitem_tuple(key)
File "/usr/local/lib/python3.7/dist-packages/pandas/core/indexing.py", line 1100, in _getitem_tuple
return self._getitem_lowerdim(tup)
File "/usr/local/lib/python3.7/dist-packages/pandas/core/indexing.py", line 838, in _getitem_lowerdim
section = self._getitem_axis(key, axis=i)
File "/usr/local/lib/python3.7/dist-packages/pandas/core/indexing.py", line 1164, in _getitem_axis
return self._get_label(key, axis=axis)
File "/usr/local/lib/python3.7/dist-packages/pandas/core/indexing.py", line 1113, in _get_label
return self.obj.xs(label, axis=axis)
File "/usr/local/lib/python3.7/dist-packages/pandas/core/generic.py", line 3776, in xs
loc = index.get_loc(key)
File "/usr/local/lib/python3.7/dist-packages/pandas/core/indexes/base.py", line 3363, in get_loc
raise KeyError(key) from err
KeyError: 4064
It's a bug that the labels in all.csv don't contain some images in the hand dataset. You can clone the newly updated codes or change Line 26 in file universal_landmark_detection/model/datasets/hand.py as
index_set = set(self.labels.index)
files = [i[:-4] for i in sorted(os.listdir(self.pth_Image))]
files = [i for i in files if int(i) in index_set]Thank you very much! The problem troubles me for a long time!