tobiasvanderwerff/full-page-handwriting-recognition

RuntimeError: expected scalar type Long but found Int

Closed this issue · 10 comments

when i run the main file with the command:

python src/main.py --data_dir D:\IAMdatabase --data_format form --max_epochs 3 --synthetic_augmentation_proba 0.4 --precision 16 --use_aachen_splits

i get the error from the title os this issue: RuntimeError: expected scalar type Long but found Int

full terminal output is:

Global seed set to 1337
D:\Anaconda\envs\ocrproject\Lib\site-packages\torchvision\models_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
D:\Anaconda\envs\ocrproject\Lib\site-packages\torchvision\models_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None.
warnings.warn(msg)
Using 16bit native Automatic Mixed Precision (AMP)
Trainer already configured with model summary callbacks: [<class 'pytorch_lightning.callbacks.model_summary.ModelSummary'>]. Skipping setting a default ModelSummary callback.
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

| Name | Type | Params

0 | model | FullPageHTREncoderDecoder | 17.8 M
1 | model.encoder | FullPageHTREncoder | 11.3 M
2 | model.decoder | FullPageHTRDecoder | 6.5 M
3 | model.cer_metric | CharacterErrorRate | 0
4 | model.wer_metric | WordErrorRate | 0
5 | model.loss_fn | CrossEntropyLoss | 0

17.8 M Trainable params
0 Non-trainable params
17.8 M Total params
35.632 Total estimated model params size (MB)
Validation sanity check: 0it [00:00, ?it/s]D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\data_loading.py:110: UserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argument(try 8 which is the number of cpus on this machine) in theDataLoader` init to improve performance.
rank_zero_warn(
Validation sanity check: 0%| | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
File "D:\download\full-page-handwriting-recognition-master\src\main.py", line 329, in
main(args)
File "D:\download\full-page-handwriting-recognition-master\src\main.py", line 283, in main
trainer.fit(model, dl_train, dl_val)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 735, in fit
self._call_and_handle_interrupt(
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 682, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 770, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1193, in _run
self._dispatch()
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1272, in _dispatch
self.training_type_plugin.start_training(self)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 202, in start_training
self._results = trainer.run_stage()
^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1282, in run_stage
return self._run_train()
^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1304, in _run_train
self._run_sanity_check(self.lightning_module)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1368, in _run_sanity_check
self._evaluation_loop.run()
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\base.py", line 145, in run
self.advance(*args, **kwargs)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\dataloader\evaluation_loop.py", line 109, in advance
dl_outputs = self.epoch_loop.run(dataloader, dataloader_idx, dl_max_batches, self.num_dataloaders)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\base.py", line 145, in run
self.advance(*args, **kwargs)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\epoch\evaluation_epoch_loop.py", line 123, in advance
output = self._evaluation_step(batch, batch_idx, dataloader_idx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\epoch\evaluation_epoch_loop.py", line 215, in _evaluation_step
output = self.trainer.accelerator.validation_step(step_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\accelerators\accelerator.py", line 236, in validation_step
return self.training_type_plugin.validation_step(*step_kwargs.values())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 219, in validation_step
return self.model.validation_step(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\download\full-page-handwriting-recognition-master\src\lit_models.py", line 93, in validation_step
return self.val_or_test_step(batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\download\full-page-handwriting-recognition-master\src\lit_models.py", line 100, in val_or_test_step
logits, _, loss = self(imgs, targets)
^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\download\full-page-handwriting-recognition-master\src\lit_models.py", line 84, in forward
return self.model(imgs, targets)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\download\full-page-handwriting-recognition-master\src\models.py", line 398, in forward
loss = self.loss_fn(
^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\modules\loss.py", line 1174, in forward
return F.cross_entropy(input, target, weight=self.weight,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\functional.py", line 3029, in cross_entropy
return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expected scalar type Long but found Int

It seems like either input or target value is int when its supposed to be float but i couldnt find a way to correct this issue any solution or insight would be appreciated.

P.S: I'm not super experienced in deeplearning.

I suspect an explicit cast to Long is necessary for the targets passed to the cross-entropy loss. Could you try adding targets = targets.long() right before you call the model and see if that resolves the issue? You could also change it in src.data.IAMDataset.collate_fn:160, where the target tensor is first created.

Hello thank you for the quick reply. I tried your targets = targets.long() suggestion on two places:

  1. in src.main

else: # train a model
targets = targets.long()
trainer.fit(model, dl_train, dl_val)
...

  1. in src.data.collate_fn

else:
imgs, targets = zip(*batch)
targets = targets.long()
img_sizes = [im.shape for im in imgs]
...

in both i get the same error: AttributeError: 'tuple' object has no attribute 'long'
I think the zip function returns a tuple for the targets.

I also tried casting the tuple to tensor with targets = torch.tensor(targets) but i got the error: expected sequence of length 48 at dim 1 (got 95)

I don't know what i did wrong.

Can you try adding targets_padded = targets_padded.long() after line 160 of data.py? Line 160 is where the targets are created as a tensor. After that you can convert it to a Long. I.e. after imgs, targets_padded = torch.tensor(imgs), torch.tensor(targets_padded).

I tried that and it seems to have solved that issue but i think i have a different problem now i get the fallowing output and error:

Global seed set to 1337
Missing logger folder: D:\download\full-page-handwriting-recognition-master\lightning_logs
D:\Anaconda\envs\ocrproject\Lib\site-packages\torchvision\models_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
D:\Anaconda\envs\ocrproject\Lib\site-packages\torchvision\models_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None.
warnings.warn(msg)
Using 16bit native Automatic Mixed Precision (AMP)
Trainer already configured with model summary callbacks: [<class 'pytorch_lightning.callbacks.model_summary.ModelSummary'>]. Skipping setting a default ModelSummary callback.
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

| Name | Type | Params

0 | model | FullPageHTREncoderDecoder | 17.8 M
1 | model.encoder | FullPageHTREncoder | 11.3 M
2 | model.decoder | FullPageHTRDecoder | 6.5 M
3 | model.cer_metric | CharacterErrorRate | 0
4 | model.wer_metric | WordErrorRate | 0
5 | model.loss_fn | CrossEntropyLoss | 0

17.8 M Trainable params
0 Non-trainable params
17.8 M Total params
35.632 Total estimated model params size (MB)
Validation sanity check: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [02:27<00:00, 64.86s/it]D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\modules\activation.py:1144: UserWarning: Converting mask without torch.bool dtype to bool; this will negatively affect performance. Prefer to use a boolean mask directly. (Triggered internally at ..\aten\src\ATen\native\transformers\attention.cpp:152.)
return torch._native_multi_head_attention(
Global seed set to 1337
Epoch 0: 0%| | 0/805 [00:00<?, ?it/s]Traceback (most recent call last):
File "D:\download\full-page-handwriting-recognition-master\src\main.py", line 326, in
main(args)
File "D:\download\full-page-handwriting-recognition-master\src\main.py", line 281, in main
trainer.fit(model, dl_train, dl_val)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 735, in fit
self._call_and_handle_interrupt(
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 682, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 770, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1193, in _run
self._dispatch()
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1272, in _dispatch
self.training_type_plugin.start_training(self)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 202, in start_training
Traceback (most recent call last):
File "", line 1, in
File "D:\Anaconda\envs\ocrproject\Lib\multiprocessing\spawn.py", line 120, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\multiprocessing\spawn.py", line 130, in _main
self = reduction.pickle.load(from_parent)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
EOFError: Ran out of input
self._results = trainer.run_stage()
^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1282, in run_stage
return self._run_train()
^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1312, in _run_train
self.fit_loop.run()
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\base.py", line 145, in run
self.advance(*args, **kwargs)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\fit_loop.py", line 234, in advance
self.epoch_loop.run(data_fetcher)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\base.py", line 140, in run
self.on_run_start(*args, **kwargs)
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\epoch\training_epoch_loop.py", line 141, in on_run_start
self._dataloader_iter = _update_dataloader_iter(data_fetcher, self.batch_idx + 1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\loops\utilities.py", line 121, in _update_dataloader_iter
dataloader_iter = enumerate(data_fetcher, batch_idx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\utilities\fetching.py", line 198, in iter
self._apply_patch()
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\utilities\fetching.py", line 133, in _apply_patch
apply_to_collections(self.loaders, self.loader_iters, (Iterator, DataLoader), _apply_patch_fn)
^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\utilities\fetching.py", line 181, in loader_iters
loader_iters = self.dataloader_iter.loader_iters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\supporters.py", line 523, in loader_iters
self._loader_iters = self.create_loader_iters(self.loaders)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\supporters.py", line 563, in create_loader_iters
return apply_to_collection(loaders, Iterable, iter, wrong_dtype=(Sequence, Mapping))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\utilities\apply_func.py", line 92, in apply_to_collection
return function(data, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\utils\data\dataloader.py", line 442, in iter
return self._get_iterator()
^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\utils\data\dataloader.py", line 388, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\utils\data\dataloader.py", line 1043, in init
w.start()
File "D:\Anaconda\envs\ocrproject\Lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\multiprocessing\context.py", line 336, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "D:\Anaconda\envs\ocrproject\Lib\multiprocessing\popen_spawn_win32.py", line 94, in init
reduction.dump(process_obj, to_child)
File "D:\Anaconda\envs\ocrproject\Lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'IAMSyntheticDataGenerator.get_worker_init_fn..worker_init_fn'. Did you mean: '_loader_iters'?

This is a bit more mysterious to me. It seems to have something to do with multiprocessing. Are you setting --num_workers? If so, can you try disabling multiprocessing by using --num_workers 0?

Yes i used 8 workers since it said:

UserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argument(try 8 which is the number of cpus on this machine) in theDataLoader` init to improve performance.

but i used 0 as you said and now it seems to work well. It is only taking a long time at the stage where it says:
Running train inference on best model...

It has been a few hour and its still at that stage which i assume is normal?

edit: its been a few more hours its still stuck something seems to be wrong but im not sure.

The "running inference on best model" line is due to the LogWorstPredictions callback, which is activated at the end of training. I'm not sure why it may be so slow, but if it continues to cause problems for you then you can safely comment it out in main.py, i.e. comment out the part where the LogWorstPredictions callback is added to the list of callbacks.

As to why multiprocessing is failing: it may perhaps be a Windows-specific issue, because I have not seen it before. The warning about not using enough workers is generated by Pytorch Lightning. Basically, training can become faster if you set num_workers to > 0 because it will spawn multiple processes instead of just one, but this is only useful in the sense that it will make training somewhat faster. So if it only works for you if num_workers=0, I would continue to use that.

Hi, i've commented out the LogWorstPredictions part and now im getting a different error unfortunately.
The full console output is here:

Global seed set to 1337
D:\Anaconda\envs\ocrproject\Lib\site-packages\torchvision\models_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
D:\Anaconda\envs\ocrproject\Lib\site-packages\torchvision\models_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None.
warnings.warn(msg)
Using 16bit native Automatic Mixed Precision (AMP)
Trainer already configured with model summary callbacks: [<class 'pytorch_lightning.callbacks.model_summary.ModelSummary'>]. Skipping setting a default ModelSummary callback.
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

| Name | Type | Params

0 | model | FullPageHTREncoderDecoder | 17.8 M
1 | model.encoder | FullPageHTREncoder | 11.3 M
2 | model.decoder | FullPageHTRDecoder | 6.5 M
3 | model.cer_metric | CharacterErrorRate | 0
4 | model.wer_metric | WordErrorRate | 0
5 | model.loss_fn | CrossEntropyLoss | 0

17.8 M Trainable params
0 Non-trainable params
17.8 M Total params
35.632 Total estimated model params size (MB)
Validation sanity check: 0it [00:00, ?it/s]D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\data_loading.py:110: UserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argument(try 8 which is the number of cpus on this machine) in theDataLoaderinit to improve performance. rank_zero_warn( Validation sanity check: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [03:13<00:00, 93.72s/it]D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\modules\activation.py:1144: UserWarning: Converting mask without torch.bool dtype to bool; this will negatively affect performance. Prefer to use a boolean mask directly. (Triggered internally at ..\aten\src\ATen\native\transformers\attention.cpp:152.) return torch._native_multi_head_attention( Global seed set to 1337 D:\Anaconda\envs\ocrproject\Lib\site-packages\pytorch_lightning\trainer\data_loading.py:110: UserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of thenum_workers argument (try 8 which is the number of cpus on this machine) in the DataLoader init to improve performance.
rank_zero_warn(
Epoch 0: 0%| | 0/202 [00:00<?, ?it/s]Generated form height exceeds maximum height. Generating a new form.
D:\Anaconda\envs\ocrproject\Lib\site-packages\torch\nn\functional.py:4999: UserWarning: Support for mismatched key_padding_mask and attn_mask is deprecated. Use same type for both instead.
warnings.warn(
Epoch 4: 26%|████████████████████████▋ | 53/202 [01:18<03:40, 1.48s/it, loss=2.53, char_error_rate=2.120, word_error_rate=3.000, val_loss=4.100]WARNING (randomly_displace_and_pad): cropping input image before padding because it exceeds the size of the frame.
Epoch 4: 95%|███████████████████████████████████████████████████████████████████████████████████████▉ | 191/202 [08:54<00:30, 2.80s/it, loss=2.48, char_error_rate=2.120, word_error_rate=3.000, val_loss=4.100]Exception ignored in: <function Image.del at 0x000001EF39CB4CC0> | 3/15 [04:01<16:08, 80.73s/it]
Traceback (most recent call last):
File "D:\Anaconda\envs\ocrproject\Lib\tkinter_init_.py", line 4073, in del
self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.del at 0x000001EF3997E160>
Traceback (most recent call last):
File "D:\Anaconda\envs\ocrproject\Lib\tkinter_init_.py", line 410, in del
if self._tk.getboolean(self._tk.call("info", "exists", self.name)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.del at 0x000001EF3997E160>
Traceback (most recent call last):
File "D:\Anaconda\envs\ocrproject\Lib\tkinter_init
.py", line 410, in del
if self._tk.getboolean(self._tk.call("info", "exists", self.name)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.del at 0x000001EF3997E160>
Traceback (most recent call last):
File "D:\Anaconda\envs\ocrproject\Lib\tkinter_init
.py", line 410, in del
if self._tk.getboolean(self._tk.call("info", "exists", self.name)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.del at 0x000001EF3997E160>
Traceback (most recent call last):
File "D:\Anaconda\envs\ocrproject\Lib\tkinter_init
.py", line 410, in del
if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Tcl_AsyncDelete: async handler deleted by the wrong thread

well i tried again a few times and the training works well. thank you for the help. closing the issue now.