insert all columns or none
Closed this issue · 11 comments
Hi,
I would like to insert all columns of the csv as field_columns (except timestamp, of course) and no columns as tag_columns. How would be the syntax?
(My CSV has too many columns to write in the main command... and doesn't have tag columns)
I tried:
--field_columns *
--field_columns all
--field_columns columns
--tag_columns none
but nothing works.
Hello @sonejostudios
Sorry for the delay. These days are busy days for me.
For now, the tag_columns is mandatory set.
I will release a new release late to remove these mandatory settings.
I will keep you posted.
Sorry for the delay again,
Bugazelle
Hello @sonejostudios
In the new version v0.1.21, the --tag_columns is not mandatory any more.
Please try the new version on your side.
Notes:
The current version cannot support the: *, all
You need to pass the filed name after --field_columns, --tag_columns
Thank you, and have a nice day to you
Hello @Bugazelle ,
thank you very much for the new version!
Still any chance to implement a "all" or "*" to add all columns as fields?
my csv-files have 150+ columns... thus this would be really nice :)
otherwise I'll first need create a list of the columns. Can I just pass a python list as field_columns argument ?
Hello @sonejostudios
Now the new version 0.1.22 should support the *.
Would you please try it on your side?
And let us know if you encounter any bugs.
Nice day to you. ;)
Take a sample here.
1. We have the demo.csv
timestamp,url,response_time
2019-07-11 02:04:05,https://jmeter.apache.org/,1.434
2019-07-11 02:04:06,https://jmeter.apache.org/,2.434
2019-07-11 02:04:07,https://jmeter.apache.org/,1.200
2019-07-11 02:04:08,https://jmeter.apache.org/,1.675
2019-07-11 02:04:09,https://jmeter.apache.org/,2.265
2019-07-11 02:04:10,https://sample-demo.org/,1.430
2019-07-12 08:54:13,https://sample-show.org/,1.300
2019-07-12 14:06:00,https://sample-7.org/,1.289
2019-07-12 18:45:34,https://sample-8.org/,2.876
2. Use * to match all the columns to fields, and ignore tag
export_csv_to_influx \
--csv demo.csv \
--dbname demo \
--measurement demo \
--field_columns '*' \
--user admin \
--password admin \
--server 127.0.0.1:8086 \
--force_insert_even_csv_no_update True \
--drop_database=True
3. Validate data in influx
> use demo
Using database demo
> select * from demo
name: demo
time response_time timestamp url
---- ------------- --------- ---
1562810645000000000 1.434 2019-07-11 02:04:05 https://jmeter.apache.org/
1562810646000000000 2.434 2019-07-11 02:04:06 https://jmeter.apache.org/
1562810647000000000 1.2 2019-07-11 02:04:07 https://jmeter.apache.org/
1562810648000000000 1.675 2019-07-11 02:04:08 https://jmeter.apache.org/
1562810649000000000 2.265 2019-07-11 02:04:09 https://jmeter.apache.org/
1562810650000000000 1.43 2019-07-11 02:04:10 https://sample-demo.org/
1562921653000000000 1.3 2019-07-12 08:54:13 https://sample-show.org/
1562940360000000000 1.289 2019-07-12 14:06:00 https://sample-7.org/
1562957134000000000 2.876 2019-07-12 18:45:34 https://sample-8.org/
>
And you could also use the python to run the ExportCsvToInflux programmatically
from ExportCsvToInflux import ExporterObject
exporter = ExporterObject()
exporter.export_csv_to_influx(field_columns='*', ...)
# You could get the export_csv_to_influx parameter details by:
print(exporter.export_csv_to_influx.__doc__)
Hi Bugazelle,
thank you very much!
I wanted to try it and upgraded it with pip:
Successfully installed ExportCsvToInflux-0.1.22
But I get a very weird behavior:
The export_csv_to_influx command is still the old one (linux).
vince@ISGLAP10:~$ export_csv_to_influx -v
0.1.20
Is this intentional?
Otherwise I tested the new verswion 0.1.22 programmally and it works just fine! :)
All the best,
Vince
Hello @sonejostudios
That's werid...
Would you please try to: sudo pip uninstall ExportCsvToInflux?
Try uninstall several times to make sure no history version
Then install again: sudo pip install ExportCsvToInflux
Hope it works
Hi Bugazelle,
I found the problem. I've got a separated anaconda venv and anaconda just took over my $PATH on .bashrc
... and I had the 0.1.20 version installed on this venv.
now it is fixed! Thank you!
Update:
now it works fine via the command line with all fields columns and no tag column.
But I still get this warning, is it intentional?
Warning: The input --tag_columns does not expected or leaves None. Please check the fields are in csv headers or not. Continue checking...
Hello @sonejostudios
Forget about this warning.
Here just notice the users the tag columns is None.
Hope the tool helps you.
Have a nice day to you! ;)
Hi @Bugazelle ,
of course, it really helps! thank you so much. :)
All the best!
@sonejostudios my pleasure！
All the best to you!!! 😄