lyhue1991/torchkeras

list index out of range

luoclab opened this issue · 1 comments

在做外卖分类的时候可以正常训练,但是在验证的最后一般发现无法保存出现报错如下:---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[39], line 1
----> 1 keras_model.fit(train_data = dl_train,
2 val_data = dl_val,
3 epochs=2,
4 patience=3,
5 monitor='val_loss',
6 mode='min',
7 ckpt_path = ckpt_path,
8 mixed_precision='no',
9 gradient_accumulation_steps = 16
10 )

File ~/.conda/envs/Luobaseline/lib/python3.10/site-packages/torchkeras/kerasmodel.py:242, in KerasModel.fit(self, train_data, val_data, epochs, ckpt_path, patience, monitor, mode, callbacks, plot, wandb, quiet, mixed_precision, cpu, gradient_accumulation_steps)
239 self.history[name] = self.history.get(name, []) + [metric]
241 if self.accelerator.is_local_main_process:
--> 242 [cb.on_validation_epoch_end(model = self) for cb in self.callbacks
243 if hasattr(cb,'on_validation_epoch_end')]
245 # 3,early-stopping -------------------------------------------------
246 self.accelerator.wait_for_everyone()

File ~/.conda/envs/Luobaseline/lib/python3.10/site-packages/torchkeras/kerasmodel.py:242, in (.0)
239 self.history[name] = self.history.get(name, []) + [metric]
241 if self.accelerator.is_local_main_process:
--> 242 [cb.on_validation_epoch_end(model = self) for cb in self.callbacks
243 if hasattr(cb,'on_validation_epoch_end')]
245 # 3,early-stopping -------------------------------------------------
246 self.accelerator.wait_for_everyone()

File ~/.conda/envs/Luobaseline/lib/python3.10/site-packages/torchkeras/kerascallbacks.py:187, in VisMetric.on_validation_epoch_end(self, model)
185 n = len(dfhistory)
186 x_bounds = [dfhistory['epoch'].min(), min(10+(n//10)*10,model.epochs)]
--> 187 title = self.get_title(model)
188 self.update_graph(model, title = title,x_bounds = x_bounds)

File ~/.conda/envs/Luobaseline/lib/python3.10/site-packages/torchkeras/kerascallbacks.py:204, in VisMetric.get_title(self, model)
203 def get_title(self, model:'KerasModel'):
--> 204 best_epoch,best_score = self.get_best_score(model)
205 title = f'best {model.monitor}={best_score:.4f} (@epoch {best_epoch})'
206 return title

File ~/.conda/envs/Luobaseline/lib/python3.10/site-packages/torchkeras/kerascallbacks.py:200, in VisMetric.get_best_score(self, model)
198 arr_scores = dfhistory[model.monitor]
199 best_score = np.max(arr_scores) if model.mode=="max" else np.min(arr_scores)
--> 200 best_epoch = dfhistory.loc[arr_scores==best_score,'epoch'].tolist()[0]
201 return (best_epoch, best_score)

IndexError: list index out of range