UnicodeDecodeError: ascii codec can't decode
Closed this issue · 4 comments
Issue
I get the same error as reported in issue #38 when running bidskit on my server. I'm using the command line version:
$ bidskit --no-sessions
Bidskit throws the following UnicodeDecodeError on first pass conversion and does not create all files (e.g. Protocol_Translator.json is missing).
To debug this, I've run locale in python and got:
>>> import locale
>>> locale.getpreferredencoding(False)
'ANSI_X3.4-1968'
The above should probably 'UTF-8' but the default in my local python is some form of ASCII. Thus, python opens files in ASCII.
Any advice for fixing this? Would it be possible to add a line in bidskit to change the encoding to UTF-8, so that one doesn't need to poke the default encoding? For instance, I'm on a shared server and wouldn't want to break anything by changing the default python encoding.
Screenshots
[sbhesa@soc-emotion DecodingEmotions]$ bidskit --no-sessions
------------------------------------------------------------
BIDSKIT 2020.8.1
------------------------------------------------------------
dcm2niix version v1.0.20201102 detected
Initializing BIDS dataset directory tree in /home/sbhesa/projects/DecodingEmotions
Creating required file templates
Preserving previous dataset_description.json
Preserving previous participants.json
Source data directory : /home/sbhesa/projects/DecodingEmotions/sourcedata
Working Directory : /home/sbhesa/projects/DecodingEmotions/work
Use Session Directories : No
Overwrite Existing Files : No
Anonymize BIDS Output : Yes
Bind fieldmaps : No
------------------------------------------------------------
Pass 1 : DICOM to Nifti conversion and translator creation
------------------------------------------------------------
------------------------------------------------------------
Processing subject PILOT1
------------------------------------------------------------
Working subject directory : /home/sbhesa/projects/DecodingEmotions/work/sub-PILOT1
BIDS subject directory : /home/sbhesa/projects/DecodingEmotions/sub-PILOT1
Converting all DICOM images in /home/sbhesa/projects/DecodingEmotions/sourcedata/PILOT1/
Traceback (most recent call last):
File "/usr/local/bin/bidskit", line 11, in <module>
load_entry_point('bidskit==2020.8.1', 'console_scripts', 'bidskit')()
File "/usr/local/lib/python3.6/site-packages/bidskit-2020.8.1-py3.6.egg/bidskit/__main__.py", line 227, in main
args.clean_conv_dir, overwrite)
File "/usr/local/lib/python3.6/site-packages/bidskit-2020.8.1-py3.6.egg/bidskit/organize.py", line 64, in organize_series
nii_list, json_list, acq_times = btr.ordered_file_list(conv_dir)
File "/usr/local/lib/python3.6/site-packages/bidskit-2020.8.1-py3.6.egg/bidskit/translate.py", line 63, in ordered_file_list
acq_time = [get_acq_time(json_file) for json_file in json_list]
File "/usr/local/lib/python3.6/site-packages/bidskit-2020.8.1-py3.6.egg/bidskit/translate.py", line 63, in <listcomp>
acq_time = [get_acq_time(json_file) for json_file in json_list]
File "/usr/local/lib/python3.6/site-packages/bidskit-2020.8.1-py3.6.egg/bidskit/translate.py", line 83, in get_acq_time
info = read_json(json_file)
File "/usr/local/lib/python3.6/site-packages/bidskit-2020.8.1-py3.6.egg/bidskit/io.py", line 45, in read_json
json_dict = json.load(fd)
File "/usr/lib64/python3.6/json/__init__.py", line 296, in load
return loads(fp.read(),
File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 472: ordinal not in range(128)
Desktop:
- OS: Red Hat Enterprise Linux Server
- Version: 7.9 (Maipo)
@hpsaarimaki Hi Heini! This looks like something fixable. Could you send me an example JSON sidecar from your dataset to work with? There are probably locale encoding effects introduced during dcm2niix conversion that need to be handled by bidskit. Thanks for bringing this up!
@jmtyszka Hi Mike! Perfect, thanks! I've sent the JSON file by email. Let me know if you want me to add it here for future reference.
Did the jupyter notebook help debug this? Let me know if you've solved this issue.