CSV error

[abdelaziz-benasser]

Hello,

Environment : JIRA Datacenter, jiraone 0.8.1

When trying to export a lot of issues ( about 5000) with issue_export function, I get this error message:

File "C:\Program Files\Python312\Lib\site-packages\jiraone\reporting.py", line 4696, in export_issues
merging_files() # loop through each file and attempt combination
^^^^^^^^^^^^^^^
File "C:\Program Files\Python312\Lib\site-packages\jiraone\reporting.py", line 4656, in merging_files
file_reader(
File "C:\Program Files\Python312\Lib\site-packages\jiraone\reporting.py", line 7932, in file_reader
load = [d for d in read]
^^^^^^^^^^^^^^^^^
_csv.Error: field larger than field limit (131072)

It seems that a field exceeded the limit. Is there a way to increase this limit ? Or is there a way to not merge the CSV temp files in a single file at the end of process?

Thank you.

[princenyeche]

Hey @abdelaziz-benasser

It seems you have extensive data within a field in the data extraction (i.e. within a row), which is greater than the default value of the CSV parser. You can probably increase the limit using the code below

# previous jiraone import statement
import csv
import sys

csv.field_size_limit(sys.maxsize) # max_size within your system or probably 
# set an arbitrary number in integers greater than the current default
# then call the issue_export function

To your other question, the answer is No. The whole process performs a merger by performing a calculation after the downloads to determine the fields within the file.

Subsequently, you can use the page argument to get a few pages out of the 5K issues. However, I believe the same issue will arise if at any point you attempt a merge using the merge_files argument.

Let me know if the above helps.

[abdelaziz-benasser]

Hello @princenyeche ,

csv.field_size_limit(sys.maxsize) gave me overflow error

OverflowError: Python int too large to convert to C long

I tried with csv.field_size_limit(2147483647) as described in https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072 and it was successful.

Thank you!