InfluxDB new-stream missing fields

[sanel]

Hi,

I've noticed :new-streams option was added and following http://riemann.io/howto.html#forward-to-influxdb and #898, Influx would not receive some fields, like host for example.

Looking at https://github.com/riemann/riemann/blob/0.3.1/src/riemann/influxdb.clj#L18 I'm curious, why these fields aren't propagated to :new-stream code?

Debugging data sent to Influx, here are differences. Without new-stream:

Listening on localhost:8086

----- Request Start ----->

/write?u=root&db=monitoring&precision=n&consistency=one
Content-Type: text/plain; charset=utf-8^M
Content-Length: 19398^M
Host: localhost:8086^M
Connection: Keep-Alive^M
Accept-Encoding: gzip^M
User-Agent: okhttp/3.9.1^M

postgresql-metabase/pg_blks-heap_read-public-data_migrations,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="heap_read-public-data_migrations",value=0.0 1551099842000000000
postgresql-metabase/pg_blks-tidx_hit-public-computation_job,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="tidx_hit-public-computation_job",value=0.0 1551099842000000000
postgresql-metabase/pg_blks-toast_read-public-qrtz_cron_triggers,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="toast_read-public-qrtz_cron_triggers",value=0.0 1551099842000000000
postgresql-metabase/pg_blks-toast_hit-public-qrtz_cron_triggers,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="toast_hit-public-qrtz_cron_triggers",value=0.0 1551099842000000000
postgresql-metabase/pg_blks-tidx_read-public-qrtz_cron_triggers,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="tidx_read-public-qrtz_cron_triggers",value=0.0 1551099842000000000
postgresql-metabase/pg_blks-idx_hit-public-qrtz_cron_triggers,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="idx_hit-public-qrtz_cron_triggers",value=0.0 1551099842000000000
postgresql-metabase/pg_blks-idx_hit-public-task_history,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="idx_hit-public-task_history",value=0.0 1551099842000000000
postgresql-metabase/pg_db_size,host=monitor ds_index="0",ds_name="value",ds_type="gauge",plugin="postgresql",plugin_instance="metabase",type="pg_db_size",value=22803943.0 1551099842000000000
postgresql-metabase/bytes-dbsize-postgres,host=monitor ds_index="0",ds_name="value",ds_type="gauge",plugin="postgresql",plugin_instance="metabase",type="bytes",type_instance="dbsize-postgres",value=7796199.0 1551099842000000000
postgresql-metabase/bytes-dbsize-template1,host=monitor ds_index="0",ds_name="value",ds_type="gauge",plugin="postgresql",plugin_instance="metabase",type="bytes",type_instance="dbsize-template1",value=7660035.0 1551099842000000000
postgresql-metabase/pg_blks-idx_read-public-qrtz_cron_triggers,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="idx_read-public-qrtz_cron_triggers",value=0.0 1551099842000000000
postgresql-metabase/bytes-dbsize-metabase,host=monitor ds_index="0",ds_name="value",ds_type="gauge",plugin="postgresql",plugin_instance="metabase",type="bytes",type_instance="dbsize-metabase",value=22803943.0 1551099842000000000
postgresql-metabase/pg_blks-tidx_hit-public-qrtz_cron_triggers,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="tidx_hit-public-qrtz_cron_triggers",value=0.0 1551099842000000000
postgresql-metabase/pg_blks-toast_hit-public-computation_job,host=monitor ds_index="0",ds_name="value",ds_type="derive:rate",plugin="postgresql",plugin_instance="metabase",type="pg_blks",type_instance="toast_hit-public-computation_job",value=0.0 1551099842000000000
postgresql-metabase/bytes-dbsize-template0,host=monitor ds_index="0",ds_name="value",ds_type="gauge",plugin="postgresql",plugin_instance="metabase",type="bytes",type_instance="dbsize-template0",value=7660035.0 1551099842000000000
cpu-0/percent-system,host=monitor ds_index="0",ds_name="value",ds_type="gauge",plugin="cpu",plugin_instance="0",type="percent",type_instance="system",value=0.5010020040080161 1551099852000000000

With new-stream

Listening on localhost:8086

----- Request Start ----->

/write?u=root&db=monitoring&precision=n&consistency=one
Content-Type: text/plain; charset=utf-8^M
Content-Length: 6106^M
Host: localhost:8086^M
Connection: Keep-Alive^M
Accept-Encoding: gzip^M
User-Agent: okhttp/3.9.1^M

postgresql-metabase/pg_blks-idx_hit-public-dependency value=0.0 1551105972000000000
postgresql-metabase/pg_blks-tidx_hit-public-dimension value=0.0 1551105972000000000
postgresql-metabase/pg_blks-idx_read-public-dependency value=0.0 1551105972000000000
postgresql-metabase/pg_blks-heap_read-public-dependency value=0.0 1551105972000000000
cpu-0/percent-system value=1.805416248746239 1551105982000000000
cpu-0/percent-wait value=0.0 1551105982000000000
cpu-0/percent-nice value=0.0 1551105982000000000
cpu-0/percent-interrupt value=0.0 1551105982000000000
cpu-0/percent-softirq value=0.10030090270812438 1551105982000000000
cpu-0/percent-steal value=0.0 1551105982000000000
cpu-0/percent-idle value=61.283851554664 1551105982000000000

I user supposed to manually fill ::influxdb-fields map?

[jamtur01]

I'm afraid I don't know much about InfluxDB - @mcorbin might have some ideas on this?

[mcorbin]

I user supposed to manually fill ::influxdb-fields map?

Yes, you can use smap to set the influxdb tags or fields, as described in the documentation.

[jamtur01]

Thanks. Please re-open if you need.