primap-community/primap2

CSV reading with non-comma separators

JGuetschow opened this issue · 4 comments

Is your feature request related to a problem? Please describe.

When reading data from csv files, only files with comma as separator can be read using the read...csv_file_if functions as the sep parameter in pd.read_csv can not be specified

Describe the solution you'd like

Add the sep parameter to primap2 csv reading routines

Describe alternatives you've considered

Read csv using pandas function and convert in a second thep

Read csv using pandas function and convert in a second thep

Might actually be worth, the API surface of pd.read_csv is huge and next thing we probably want to read from database or something, which pandas can do, but we don't have a function. So, I'd say: just use pandas' reading functions, then convert_long_dataframe_if.

I do that all the time. At some point we might remove the csv reading functions as they can basically never be used

Yeah, I think reading the interchange format with a specific function has value, but in general so-called CSVs are super diverse. Probably it was not realistic to expect them to be sufficiently well-formed to read them with one function. /-:

I think we can close this as we decided not to implement it.