"NoneType Object is not subscriptable" errors when reading dfs0 files

[ecomodeller]

Discussed in #545

^{Originally posted by jannik-el February 7, 2023}
Hello,

I'm not very familiar with dfs0 or mike (although I am from DHI), so this quite possibly has a simple answer, however I don't quite know what to do next so I thought I'd see if anyone else has seen this kind of problem.
I am converting a large amount of files from dfs0 into pandas DataFrames and then into .json format (not important for the problem, just a context).

Multiple files (more than 20) are throwing me errors when I read them using mikeio.read(inputfile).
The error is the following:

~\AppData\Local\Temp/ipykernel_16396/440985759.py in dfs0tojson(input_file, json_output, rst_indx)
      6     Returns: None
      7     """
----> 8     ds = mikeio.read(input_file)
      9     DF = ds.to_dataframe()
     10     if rst_indx:

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mikeio\__init__.py in read(filename, items, time, keepdims, **kwargs)
    119     dfs = open(filename)
    120 
--> 121     return dfs.read(items=items, time=time, keepdims=keepdims, **kwargs)
    122 
    123 

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mikeio\dfs0.py in read(self, items, time, keepdims)
    172 
    173         # read data from file
--> 174         fdata, ftime, fitems = self.__read(self._filename)
    175         self._source = self._dfs
    176         dfs = self._dfs
...
--> 226         matrix = raw_data[:, 1:]
    227         # matrix[matrix == self._deletevalue] = np.nan
    228         matrix[matrix == self._dfs.FileInfo.DeleteValueDouble] = np.nan  # cutil

TypeError: 'NoneType' object is not subscriptable

I've tried to figure out what is going on, I think python doesn't like indexing 'raw_data' because it cannot detect that it is an iterable, so maybe the dfs0 is empty? But that's just a hunch based on the error, python is more my expertise, not mike.

Here's a ftp to one of the files which is creating the error:
ftp://tea42@ftp.dhigroup.com/pub/JAEL/NO_TS_MO_FINO2WR_waves.dfs0

Cheers, Jannik

[ecomodeller]

Hi @jannik-el, could you please add the problematic file as an attachment here. You can drag and drop it in this window!

[jannik-el]

Apologies for the late answer, there has been a lot going on - it turns out the files which I was reading were actually corrupted. Would be maybe nice if you could implement some exception handling for those kind of cases? Although I understand that is quite difficult depending on the situation. One of the problematic files are still on the ftp if you want to test it. I can't add the problematic file as an attachment here, I can't publicly release client data. Devs from DHI can access on that ftp link however.
Cheers for the help anyways and apologies again about the delayed reply,
Jannik

[jsmariegaard]

@jannik-el , could you please send the file to Henrik by email?