ricoberger/script_exporter

When the script exits with non-0 code, output is not provided to prometheus

mvalo opened this issue · 5 comments

mvalo commented

Hello,

Unless mistaking in the way to use the exporter, when the script exits with non-0 return code, its output it not provided to prometheus.
I believe it comes from here : https://github.com/ricoberger/script_exporter/blob/main/pkg/exporter/metrics.go#L67

Example OK :

[~]# cat script_exiting_0
#!/bin/bash
echo "EXIT 0"
exit 0
[~]# 
[~]# curl -s "http://0:9469/probe?script=script_exiting_0"
# HELP script_success Script exit status (0 = error, 1 = success).
# TYPE script_success gauge
script_success{script="script_exiting_0"} 1
# HELP script_duration_seconds Script execution time, in seconds.
# TYPE script_duration_seconds gauge
script_duration_seconds{script="script_exiting_0"} 0.011737
# HELP script_exit_code The exit code of the script.
# TYPE script_exit_code gauge
script_exit_code{script="script_exiting_0"} 0
EXIT 0

Example NOK :

[~]# cat script_exiting_1
#!/bin/bash
echo "EXIT 1"
exit 1
[~]# 
[~]# curl -s "http://0:9469/probe?script=script_exiting_1"
# HELP script_success Script exit status (0 = error, 1 = success).
# TYPE script_success gauge
script_success{script="script_exiting_1"} 0
# HELP script_duration_seconds Script execution time, in seconds.
# TYPE script_duration_seconds gauge
script_duration_seconds{script="script_exiting_1"} 0.002808
# HELP script_exit_code The exit code of the script.
# TYPE script_exit_code gauge
script_exit_code{script="script_exiting_1"} 1
mvalo commented

@ricoberger even if it's my first time "writing" GO code, I would like to propose a fix through a PR but I don't have the rights to push a branch on this repo.

Hi @mvalo, this is the expected behavior.

What you can do is to catch the exit code of your command within the script and export a custom metric my_command_exit_code <EXIT CODE>.

I'm a bit afraid of changing this behavior because a lot of our scripts would break when they leave the happy path, because the output will not be properly formatted anymore.

mvalo commented

Hello @ricoberger
if the script fails (even if error is caught), to me it's not natural to exit with 0.
I'm not sure to understand your relunctancy to change this.
Would it be possible to let me open a PR for you to see my changes ? then of course it's up to you, the script owner, to merge it or not :)

Hi @mvalo, you can fork the repository and create a PR from your fork. I will review it then 🙂

mvalo commented

@ricoberger here it is : #67