CSV error
abdelaziz-benasser opened this issue · 2 comments
Hello,
Environment : JIRA Datacenter, jiraone 0.8.1
When trying to export a lot of issues ( about 5000) with issue_export function, I get this error message:
File "C:\Program Files\Python312\Lib\site-packages\jiraone\reporting.py", line 4696, in export_issues
merging_files() # loop through each file and attempt combination
^^^^^^^^^^^^^^^
File "C:\Program Files\Python312\Lib\site-packages\jiraone\reporting.py", line 4656, in merging_files
file_reader(
File "C:\Program Files\Python312\Lib\site-packages\jiraone\reporting.py", line 7932, in file_reader
load = [d for d in read]
^^^^^^^^^^^^^^^^^
_csv.Error: field larger than field limit (131072)
It seems that a field exceeded the limit. Is there a way to increase this limit ? Or is there a way to not merge the CSV temp files in a single file at the end of process?
Thank you.
It seems you have extensive data within a field in the data extraction (i.e. within a row), which is greater than the default value of the CSV parser. You can probably increase the limit using the code below
# previous jiraone import statement
import csv
import sys
csv.field_size_limit(sys.maxsize) # max_size within your system or probably
# set an arbitrary number in integers greater than the current default
# then call the issue_export function
To your other question, the answer is No. The whole process performs a merger by performing a calculation after the downloads to determine the fields within the file.
Subsequently, you can use the page
argument to get a few pages out of the 5K issues. However, I believe the same issue will arise if at any point you attempt a merge using the merge_files
argument.
Let me know if the above helps.
Hello @princenyeche ,
csv.field_size_limit(sys.maxsize) gave me overflow error
OverflowError: Python int too large to convert to C long
I tried with csv.field_size_limit(2147483647) as described in https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072 and it was successful.
Thank you!