write_prometheus plugin: MHD_start_daemon() failed.
phanirajl opened this issue · 12 comments
[2020-08-15 02:48:11] write_prometheus plugin: Opening a listening socket failed.
[2020-08-15 02:48:11] write_prometheus plugin: MHD_start_daemon() failed.
[2020-08-15 02:48:11] Initialization of plugin write_prometheus' failed with status -1. Plugin will be unloaded. [2020-08-15 02:48:11] plugin_unregister_read: No such read function: write_prometheus [2020-08-15 02:48:11] cpufreq plugin: Found 4 CPUs [2020-08-15 02:48:11] Error: one or more plugin init callbacks failed. [2020-08-15 02:48:11] Initialization complete, entering read-loop. [2020-08-15 02:48:11] unixsock plugin: Successfully deleted socket file "/var/cassandra/tmp/ds-8711088068536560017.sock". [2020-08-15 02:48:11] tcpconns plugin: Reading from netlink succeeded. Will use the netlink method from now on. [2020-08-15 02:48:12] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0) [2020-08-15 02:48:12] read-function of plugin ntpd' failed. Will suspend it for 20.000 seconds.
[2020-08-15 02:48:32] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
[2020-08-15 02:48:32] read-function of plugin ntpd' failed. Will suspend it for 40.000 seconds. [2020-08-15 02:49:12] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0) [2020-08-15 02:49:12] read-function of plugin ntpd' failed. Will suspend it for 80.000 seconds.
[2020-08-15 02:50:32] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
[2020-08-15 02:50:32] read-function of plugin ntpd' failed. Will suspend it for 160.000 seconds. [2020-08-15 02:53:12] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0) [2020-08-15 02:53:12] read-function of plugin ntpd' failed. Will suspend it for 320.000 seconds.
[2020-08-15 02:58:32] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
[2020-08-15 02:58:32] read-function of plugin ntpd' failed. Will suspend it for 640.000 seconds. [2020-08-15 03:09:12] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0) [2020-08-15 03:09:12] read-function of plugin ntpd' failed. Will suspend it for 1280.000 seconds.
[2020-08-15 03:30:32] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
[2020-08-15 03:30:32] read-function of plugin ntpd' failed. Will suspend it for 2560.000 seconds. [2020-08-15 04:13:12] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0) [2020-08-15 04:13:12] read-function of plugin ntpd' failed. Will suspend it for 5120.000 seconds.
[2020-08-15 05:38:32] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
[2020-08-15 05:38:32] read-function of plugin ntpd' failed. Will suspend it for 10240.000 seconds. [2020-08-15 08:29:12] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0) [2020-08-15 08:29:12] read-function of plugin ntpd' failed. Will suspend it for 20480.000 seconds.
[2020-08-15 14:10:32] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
[2020-08-15 14:10:32] read-function of plugin ntpd' failed. Will suspend it for 40960.000 seconds. [2020-08-16 01:33:12] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0) [2020-08-16 01:33:12] read-function of plugin ntpd' failed. Will suspend it for 81920.000 seconds.
[2020-08-17 00:18:32] ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
[2020-08-17 00:18:32] read-function of plugin `ntpd' failed. Will suspend it for 86400.000 seconds.
Hi,
Could you tell me the version of C* you are on? and the version of the metrics collector?
Hi @tjake
Cassandra Version is :3.11.2
MCAC Version : datastax-mcac-agent-0.1.11
tried datastax-mcac-agent-0.1.11 & datastax-mcac-agent-0.1.10 ... same issue
[2020-08-17 21:29:56] plugin_load: plugin "write_scribe" successfully loaded. [2020-08-17 21:29:56] write_scribe plugin: Ignoring config option SocketFile'.
[2020-08-17 21:29:56] plugin_load: plugin "write_prometheus" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "cpu" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "df" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "vmem" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "numa" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "contextswitch" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "cpufreq" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "irq" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "protocols" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "ethstat" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "disk" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "interface" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "load" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "memory" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "swap" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "uptime" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "processes" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "tcpconns" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "match_regex" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "target_set" successfully loaded.
[2020-08-17 21:29:56] plugin_load: plugin "target_replace" successfully loaded.
[2020-08-17 21:29:56] write_prometheus plugin: Opening a listening socket failed.
[2020-08-17 21:29:56] write_prometheus plugin: MHD_start_daemon() failed.
[2020-08-17 21:29:56] Initialization of plugin write_prometheus' failed with status -1. Plugin will be unloaded. [2020-08-17 21:29:56] plugin_unregister_read: No such read function: write_prometheus [2020-08-17 21:29:56] cpufreq plugin: Found 4 CPUs [2020-08-17 21:29:56] Error: one or more plugin init callbacks failed. [2020-08-17 21:29:56] Initialization complete, entering read-loop. [2020-08-17 21:29:56] unixsock plugin: Successfully deleted socket file "/var/cassandra/tmp/ds-214353039211942266.sock"
What linux version / distro are you on?
Could there be another collectd installed on the machine?
[2020-08-17 21:37:30] plugin_dispatch_values: time = 1597700249.506; interval = 10.000; host = xx.xx.xx.xxx; plugin = interface; plugin_instance = eth0; type = if_errors; type_instance = ;
[2020-08-17 21:37:30] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[0] = 0.000000
[2020-08-17 21:37:30] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[1] = 0.000000
[2020-08-17 21:37:40] plugin_dispatch_values: time = 1597700259.505; interval = 10.000; host = xx.xx.xx.xxx; plugin = interface; plugin_instance = eth0; type = if_errors; type_instance = ;
[2020-08-17 21:37:40] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[0] = 0.000000
[2020-08-17 21:37:40] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[1] = 0.000000
[2020-08-17 21:37:50] plugin_dispatch_values: time = 1597700269.507; interval = 10.000; host = xx.xx.xx.xxx; plugin = interface; plugin_instance = eth0; type = if_errors; type_instance = ;
[2020-08-17 21:37:50] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[0] = 0.000000
[2020-08-17 21:37:50] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[1] = 0.000000
[2020-08-17 21:38:00] plugin_dispatch_values: time = 1597700279.507; interval = 10.000; host = xx.xx.xx.xxx; plugin = interface; plugin_instance = eth0; type = if_errors; type_instance = ;
[2020-08-17 21:38:00] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[0] = 0.000000
[2020-08-17 21:38:00] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[1] = 0.000000
[2020-08-17 21:38:10] plugin_dispatch_values: time = 1597700289.504; interval = 10.000; host = xx.xx.xx.xxx; plugin = interface; plugin_instance = eth0; type = if_errors; type_instance = ;
[2020-08-17 21:38:10] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[0] = 0.000000
[2020-08-17 21:38:10] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[1] = 0.000000
[2020-08-17 21:38:20] plugin_dispatch_values: time = 1597700299.505; interval = 10.000; host = xx.xx.xx.xxx; plugin = interface; plugin_instance = eth0; type = if_errors; type_instance = ;
[2020-08-17 21:38:20] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[0] = 0.000000
[2020-08-17 21:38:20] uc_update: xx.xx.xx.xxx/interface-eth0/if_errors: ds[1] = 0.000000
Looks like it might be a confusing log message
Opening a listening socket failed.
Could something be listening on port 9103 already?
rpm -qa |grep collectd
libcollectdclient-5.8.0-2.19.amzn1.x86_64
collectd-5.8.0-2.19.amzn1.x86_64
Amazon Linux AMI release 2018.03
netstat -alpn |grep 9103
tcp 0 0 127.0.0.1:48678 127.0.0.1:9103 ESTABLISHED 3628/miniprom
tcp 0 0 ::ffff:127.0.0.1:9103 :::* LISTEN 6539/java
tcp 0 0 ::ffff:127.0.0.1:9103 ::ffff:127.0.0.1:48678 ESTABLISHED 6539/java
Yes, I think something is running on port 9103. Just change the port