mre/kafka-influxdb

error while trying to write data in to influxdb

mrafeeq opened this issue · 3 comments

mrafeeq commented

I get this error while trying to write to influx( i am using a custom encoder which produces an array of influx db expressions.

INFO:root:Shutdown. Flushing remaining messages from buffer.
DEBUG:root:Writing message: ['cpu_load_short,host=server02 value=0.67
cpu_load_short,host=server02,region=us-west value=0.55 1422568543702900257
cpu_load_short,direction=in,host=server01,region=us-west value=2.0 1422568543702900257]
DEBUG:urllib3.connectionpool:http://127.0.0.1:8001 "POST /write?db=metrics&precision=s&rp=autogen HTTP/1.1" 404 1287
WARNING:root:Cannot write data points: 404: {
  "paths": [
    "/api",
    "/api/v1",
    "/apis",
    "/apis/apps",
    "/apis/apps/v1beta1",
    "/apis/authentication.k8s.io",
    "/apis/authentication.k8s.io/v1",
    "/apis/authentication.k8s.io/v1beta1",
    "/apis/authorization.k8s.io",
    "/apis/authorization.k8s.io/v1",
    "/apis/authorization.k8s.io/v1beta1",
    "/apis/autoscaling",
    "/apis/autoscaling/v1",
    "/apis/autoscaling/v2alpha1",
    "/apis/batch",
    "/apis/batch/v1",
    "/apis/batch/v2alpha1",
    "/apis/certificates.k8s.io",
    "/apis/certificates.k8s.io/v1beta1",
    "/apis/extensions",
    "/apis/extensions/v1beta1",
    "/apis/policy",
    "/apis/policy/v1beta1",
    "/apis/rbac.authorization.k8s.io",
    "/apis/rbac.authorization.k8s.io/v1alpha1",
    "/apis/rbac.authorization.k8s.io/v1beta1",
    "/apis/settings.k8s.io",
    "/apis/settings.k8s.io/v1alpha1",
    "/apis/storage.k8s.io",
    "/apis/storage.k8s.io/v1",
    "/apis/storage.k8s.io/v1beta1",
    "/healthz",
    "/healthz/ping",
    "/healthz/poststarthook/bootstrap-controller",
    "/healthz/poststarthook/ca-registration",
    "/healthz/poststarthook/extensions/third-party-resources",
    "/healthz/poststarthook/rbac/bootstrap-roles",
    "/logs",
    "/metrics",
    "/swaggerapi/",
    "/ui/",
    "/version"
  ]
}
Flushing output buffer. 18.59 messages/s
mre commented

You get a 404 (file not found), which indicates that InfluxDB is not reachable.
Looking at the logs, it seems like it's trying to connect to http://127.0.0.1:8001 to reach InfluxDB.
I'm pretty sure InfluxDB isn't running there.

Maybe on your local machine, but as far as I can say from #64, you run InfluxDB on a separate container. You have to be able to reach InfluxDB from within the kafka-influxdb container.
(E.g. if you open a shell to kafka-influxdb, you should be able to ping InfluxDB).

I suggest you check out docker-compose, which will help you to start multiple containers at once. There are also tutorials on YouTube on how to get started with this.

Once you have all services (InfluxDB, Kafka and kafka-influxdb) running next to each other, you should be able to reach InfluxDB from kafka-influxdb with the default configuration.

mrafeeq commented

if the custome encoder that i have written if it returns an array of multiple infludb insert expressions, is that the right way ? to do like below

['cpu_load_short,host=server02 value=0.67
cpu_load_short,host=server02,region=us-west value=0.55 1422568543702900257
cpu_load_short,direction=in,host=server01,region=us-west value=2.0 1422568543702900257]
mre commented

Looks good to me, yes.