field type error
Closed this issue · 5 comments
Hi @Bugazelle ,
While injecting my csv with your Python module, it works really well if I drop the measurement first.
But now I've got the following problem: I' want to inject a lot more point and don't want to drop the measurement each time. If I drop_measurement=False, then I get the following error while re-injecting the data:
raise InfluxDBClientError(response.content, response.status_code)
influxdb.exceptions.InfluxDBClientError: 400: {"error":"partial write: field type conflict: input field \"my_column\" on measurement \"my_data\" is type float, already exists as type integer dropped=9"}
The weird thing is, my .csv is float only but it seems to be injected as integer? and then it gives me this error, that the type is different?
Is this a bug or is this intentional? How can I set the type while injecting? I really don't care about the type, I just want all fields as float.
Thank you in advance!
Hello @sonejostudios
It caused by 2 possible root causes.
1. Influx Bug:
When dropping the measurement, actually it is not totally "dropped", see more info here:
seems fixed in 1.7.8, but appears again in 1.7.9
influxdata/influxdb#10052
influxdata/influxdb#16947
How to fix:
Please confirm first that all the "my_column" data in your "my_data" measurement is float.
SHOW FIELD KEYS FROM my_data
If "my_column" data is really float, try to use the solution here:
influxdata/influxdb#10052 (comment)
Or simple directly:
drop the entire databases, then insert again
It is the easiest way, but not suggest. Because maybe your database has other measurements
If "my_column" data is integer, see: 2. It is a really type Issue
2. It is a really Type Issue:
So your previous "my_column" data is really integer type.
And influx does not allow you to insert the different type to the same filed.
For your "my_column", the export-csv-to-influx logic is:
Treat the data as int if all data is int,
Treat the data as float if all data is float, or part of float, part of int
Treat the data as string if above 2 rules not match
So, seems your previous data in "my_column" are all int, so the tool treats them as int and insert to influx.
Now your current data has the float, the tool treats them as float.
So the insert fails
How to fix:
To make sure match your request, you only want them as float type.
I have released a new version: v0.1.24 to help you.
Note: You still need to drop your measurement first, because the current "my_column" type is already integer
Use --force_float_columns my_column in the command to force insert the data in my_column as float type
Hope could be the help to you.
You are very welcome if you have any more questions
Hi @Bugazelle ,
thank you so much, I'll try asap!
All the best
Hello @Bugazelle, there might be another problem. If I have a float value, that ends with 0, like 123.0, then "export-csv-to-influx" is interpreting it as int.
Greets
UPDATE
I've just found out, that --force_float_columns will probably solve my problem. I overlooked the fact, that I can specify the columns, and not just force this to all of them.
Sorry to bother you. Thank you for this nice script!!
\UPDATE
I think, I've found the cause of this behavior. The function is_integer() returns true for numbers with only a zero behind the dot: https://python-reference.readthedocs.io/en/latest/docs/float/is_integer.html
Is this intended? I use your script in a bash.script for periodically inserting weatherdata from a .csv file. There are different data types included (string's, int's and float's) and sometimes, some of this values are ending with a 0... Here is a sample:
station;city;height_m;time;t_°C;dp_°C;rh_%;wd_°;ws_kmh;pwd_°;pws_kmh;rf_lm2;apr_hPa;aps_hPa; 12345;Vienna;184;24-03-2020 14:00;7.5;-8.3;32;342;14.0;345;44.6;0.0;1029.5;1007.0
Best regards