cuda error: device-side accert triggred
ops-admin-chengdu opened this issue · 0 comments
Description
处理部分字段为空或者含有特殊字符的csv文件时出现异常。
Reproduce
python3 1.py
1.py 的内容
import os
import csv
from opendlp.sensitive_analyze import table_analyzer
import tracemalloc
#tracemalloc.start()
# 定义目标目录
output_directory = 'log-new/result'
# 打开 csv_paths.txt 文件以逐行扫描
with open('csv_paths.txt', 'r') as file:
for line in file:
csv_table_path = line.strip() # 去除行尾的换行符和空白
print(csv_table_path)
# 检查文件是否存在
if os.path.exists(csv_table_path):
result = table_analyzer.analyze(csv_table_path)
# 创建一个新的输出文件路径
output_file_name = os.path.join(output_directory, f"output_analyze_{os.path.basename(csv_table_path)}.txt")
# 确保目标目录存在,如果不存在则创建
os.makedirs(os.path.dirname(output_file_name), exist_ok=True)
with open(output_file_name, 'w', newline='') as output_csvfile:
writer = csv.writer(output_csvfile, delimiter=',')
for k, v in result.items():
writer.writerow([k, v])
else:
print(f"File does not exist: {csv_table_path}")
Expected behavior
Context
- Operating System and version: ubuntu16.04
- Browser and version(if necessary): none
- Which version are you using:
python3.6.10
tldextract==3.1.2
Ipy==1.1
pandas==1.1.5
torch==1.10.0
hanlp==2.1.0a64
pytest==7.0.1
GPU: tesla P4
Error message
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:699: indexSelectLargeIndex: block: [32,0,0], thread: [94,0,0] Assertion `srcIndex < srcSelectDimSize` failed. /pytorch/aten/src/ATen/native/cuda/Indexing.cu:699: indexSelectLargeIndex: block: [32,0,0], thread: [95,0,0] Assertion `srcIndex < srcSelectDimSize` failed. return self._apply(convert) File "/root/anaconda3/envs/python36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 570, in _apply module._apply(fn) File "/root/anaconda3/envs/python36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 593, in _apply param_applied = fn(param) File "/root/anaconda3/envs/python36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 897, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) RuntimeError: CUDA error: device-side assert triggered
Configuration
csv文件内容举例: # cat 1.csv (1.csv部分内容为空时,抛出如上错误信息) id,sample_flow_id,order_id,sale_assistant_id,opt_manager_id,opt_director_id,area_opt_holder_id,audit_person_id,audit_time,audit_valid,invalid_reason,reason_attachment,area_holder_id,sample_priority,sample_total 1300668,58,2048434,,10149824,,,,,,,,,27,147 1300666,58,2048432,,10149824,,,,,,,,,89,147 1300664,58,2048422,,10149610,,,,,,,,,3,5 1300662,58,2048420,,10149824,,,,,,,,,46,147 1300660,58,2048406,,10045256,,,,,,,,,4,4 1300658,58,2048404,,10149824,,,,,,,,,10,147 1300656,58,2048402,,10149824,,,,,,,,,3,147 1300654,58,2048382,,10184292,,,,,,,,,0,1 1300652,58,2048372,,10149610,,,,,,,,,2,2 1300650,58,2048364,,10045256,,,,,,,,,5,11#cat 2.csv (含有中文)
id,name,vehicle_photo_type_id,terminal_photo_type_id,is_active,create_at,update_at,version
54,其他,9,5,1,2020-11-13 02:46:15,2020-11-13 02:46:15,0
52,照片模糊,9,5,1,2020-11-13 02:46:15,2020-11-13 02:46:15,0
50,其他,9,4,1,2020-11-13 02:46:15,2020-11-13 02:46:15,0
48,照片模糊,9,4,1,2020-11-13 02:46:15,2020-11-13 02:46:15,0
46,设备号/卡号与照片不一致,9,1,1,2020-08-03 23:09:53,2020-08-03 23:09:53,0
44,设备号/卡号照片模糊,9,1,1,2020-08-03 23:09:53,2020-08-03 23:09:53,0
42,安装位置与照片不一致,9,3,1,2020-08-03 23:09:53,2020-08-03 23:09:53,0
40,安装位置照片模糊,9,3,1,2020-08-03 23:09:53,2020-08-03 23:09:53,0
38,车架号与照片不一致,4,,1,2020-08-03 23:09:53,2020-08-03 23:09:53,0
36,车架号照片模糊,4,,1,2020-08-03 23:09:53,2020-08-03 23:09:53,0
Additional context
Add any other context about the problem here.