[Train Error] tensorboud writer.add_histogram reported an error
Closed this issue · 1 comments
teble commented
numpy 1.24+ 对数据类型更加严格导致train报错
Traceback (most recent call last):
File "/root/Mortal/mortal/train.py", line 424, in <module>
main()
File "/root/Mortal/mortal/train.py", line 402, in main
train()
File "/root/Mortal/mortal/train.py", line 383, in train
train_epoch()
File "/root/Mortal/mortal/train.py", line 264, in train_epoch
writer.add_histogram('q_predicted', all_q_1d, steps)
File "/root/miniconda3/envs/mortal/lib/python3.10/site-packages/torch/utils/tensorboard/writer.py", line 485, in add_histogram
histogram(tag, values, bins, max_bins=max_bins), global_step, walltime
File "/root/miniconda3/envs/mortal/lib/python3.10/site-packages/torch/utils/tensorboard/summary.py", line 358, in histogram
hist = make_histogram(values.astype(float), bins, max_bins)
File "/root/miniconda3/envs/mortal/lib/python3.10/site-packages/torch/utils/tensorboard/summary.py", line 386, in make_histogram
cum_counts = np.cumsum(np.greater(counts, 0, dtype=np.int32))
TypeError: No loop matching the specified signature and casting was found for ufunc greater
降级1.23临时解决 pip install numpy==1.23
Equim-chan commented
这是 torch 的 tensorboard 的用法问题。我看那边已经有人提过 pytorch/pytorch#93017 pytorch/pytorch#91516。