nginxinc/nginx-prometheus-exporter

json: cannot unmarshal number '*' into Go struct field

Closed this issue · 4 comments

ddimych commented

Describe the bug

2022/09/20 15:44:20 Error getting stats: failed to get stats: failed to get slabs: error unmarshaling response "{\"SSL\":{\"pages\":{\"used\":49,\"free\":2495},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"64\":{\"used\":1,\"free\":63,\"reqs\":1,\"fails\":0},\"128\":{\"used\":321,\"free\":191,\"reqs\":819,\"fails\":0},\"256\":{\"used\":320,\"free\":192,\"reqs\":818,\"fails\":0},\"512\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"jwk\":{\"pages\":{\"used\":3,\"free\":12},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":1,\"free\":126,\"reqs\":1,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":1,\"free\":15,\"reqs\":1,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"oidc_id_tokens\":{\"pages\":{\"used\":24,\"free\":2520},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":35,\"free\":92,\"reqs\":44,\"fails\":0},\"64\":{\"used\":1,\"free\":63,\"reqs\":1,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":36,\"free\":12,\"reqs\":45,\"fails\":0},\"512\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":35,\"free\":3,\"reqs\":44,\"fails\":0}}},\"refresh_tokens\":{\"pages\":{\"used\":240,\"free\":2304},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":222,\"free\":32,\"reqs\":230,\"fails\":0},\"64\":{\"used\":1,\"free\":63,\"reqs\":1,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":223,\"free\":17,\"reqs\":231,\"fails\":0},\"512\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"oidc_pkce\":{\"pages\":{\"used\":2,\"free\":29},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":1,\"free\":126,\"reqs\":1,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":1,\"free\":15,\"reqs\":1,\"fails\":0},\"512\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"wallarm_stat\":{\"pages\":{\"used\":0,\"free\":477},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":2395},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":4,\"free\":-4,\"reqs\":392011001,\"fails\":139927184851456},\"64\":{\"used\":44,\"free\":36,\"reqs\":549,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":782543507,\"free\":-782543506,\"reqs\":80,\"fails\":19},\"512\":{\"used\":0,\"free\":654,\"reqs\":0,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":1,\"fails\":605718211},\"2048\":{\"used\":100,\"free\":-20,\"reqs\":510,\"fails\":0}}},\"ams_cache\":{\"pages\":{\"used\":2,\"free\":1270},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":1,\"free\":126,\"reqs\":1,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"ams\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":2,\"free\":502,\"reqs\":2,\"fails\":0},\"16\":{\"used\":2,\"free\":252,\"reqs\":2,\"fails\":0},\"32\":{\"used\":2,\"free\":125,\"reqs\":2,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"calendar\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":2,\"free\":502,\"reqs\":2,\"fails\":0},\"16\":{\"used\":1,\"free\":253,\"reqs\":1,\"fails\":0},\"32\":{\"used\":3,\"free\":124,\"reqs\":3,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"confluence\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":1,\"free\":503,\"reqs\":1,\"fails\":0},\"16\":{\"used\":3,\"free\":251,\"reqs\":3,\"fails\":0},\"32\":{\"used\":2,\"free\":125,\"reqs\":2,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"jira\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":2,\"free\":502,\"reqs\":2,\"fails\":0},\"16\":{\"used\":2,\"free\":252,\"reqs\":2,\"fails\":0},\"32\":{\"used\":2,\"free\":125,\"reqs\":2,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"learning\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":2,\"free\":502,\"reqs\":2,\"fails\":0},\"16\":{\"used\":2,\"free\":252,\"reqs\":2,\"fails\":0},\"32\":{\"used\":2,\"free\":125,\"reqs\":2,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"request\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":2,\"free\":502,\"reqs\":2,\"fails\":0},\"16\":{\"used\":1,\"free\":253,\"reqs\":1,\"fails\":0},\"32\":{\"used\":3,\"free\":124,\"reqs\":3,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"tpwot_cache\":{\"pages\":{\"used\":24,\"free\":2520},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"64\":{\"used\":1,\"free\":63,\"reqs\":1,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":340,\"free\":12,\"reqs\":340,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"wgfitness\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":1,\"free\":503,\"reqs\":1,\"fails\":0},\"16\":{\"used\":3,\"free\":251,\"reqs\":3,\"fails\":0},\"32\":{\"used\":2,\"free\":125,\"reqs\":2,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"wgjira\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":2,\"free\":502,\"reqs\":2,\"fails\":0},\"16\":{\"used\":2,\"free\":252,\"reqs\":2,\"fails\":0},\"32\":{\"used\":2,\"free\":125,\"reqs\":2,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"wgra\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":2,\"free\":502,\"reqs\":2,\"fails\":0},\"16\":{\"used\":2,\"free\":252,\"reqs\":2,\"fails\":0},\"32\":{\"used\":2,\"free\":125,\"reqs\":2,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"wotcalendar\":{\"pages\":{\"used\":6,\"free\":9},\"slots\":{\"8\":{\"used\":1,\"free\":503,\"reqs\":1,\"fails\":0},\"16\":{\"used\":2,\"free\":252,\"reqs\":2,\"fails\":0},\"32\":{\"used\":3,\"free\":124,\"reqs\":3,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":4,\"free\":28,\"reqs\":4,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":1,\"free\":7,\"reqs\":1,\"fails\":0},\"1024\":{\"used\":1,\"free\":3,\"reqs\":1,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}},\"wallarm_worker\":{\"pages\":{\"used\":9,\"free\":6},\"slots\":{\"8\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"16\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"32\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"64\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"128\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"256\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"512\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"1024\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0},\"2048\":{\"used\":0,\"free\":0,\"reqs\":0,\"fails\":0}}}}": json: cannot unmarshal number -4 into Go struct field Slot.Slots.Free of type uint64

OS - Ubuntu Server 20.04
5.4.0-117-generic #132-Ubuntu SMP

To reproduce
Steps to reproduce the behavior:

  1. Start exporter with command
    nginx-prometheus-exporter -nginx.plus -nginx.scrape-uri=http://localhost/api
  2. Wait for requests
  3. See error above

Expected behavior
Exporter collect statistics data from Nginx+ API in a correct way

Your environment

  • Version of the Prometheus exporter - release version or a specific commit
    NGINX Prometheus Exporter version=0.11.0 commit=e4a6810d4f0b776f7fde37fea1d84e4c7284b72a date=2022-09-07T21:09:51Z, dirty=false, arch=linux/amd64, go=go1.19
  • Using NGINX or NGINX Plus
    nginx version: nginx/1.21.6 (nginx-plus-r27) built by gcc 9.3.0 (Ubuntu 9.3.0-10ubuntu2) built with OpenSSL 1.1.1f 31 Mar 2020 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --build=nginx-plus-r27 --with-http_auth_jwt_module --with-http_f4f_module --with-http_hls_module --with-http_session_log_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-plus-1.21.6/debian/debuild-base/nginx-plus-1.21.6=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'
lucacome commented

Hi @ddimych

I can see that there are some negative values in the free memory slots that I can't really make sense of

  "wallarm_stat": {
...
      "32": {
        "used": 4,
        "free": -4,
        "reqs": 392011001,
        "fails": 139927184851456
      },
...
      "256": {
        "used": 782543507,
        "free": -782543506,
        "reqs": 80,
        "fails": 19
      },
...
      "2048": {
        "used": 100,
        "free": -20,
        "reqs": 510,
        "fails": 0
      }
    }
  },

Can you maybe share more details about wallarm_stat? Some config?

ddimych commented

Hi @lucacome

We are using wallarm web-application firewall on this hosts.

This looks like a bunch of wallarm monitoring stats identified by exporter like nginx server stats

Here is a config

`server {
listen 127.0.0.8:80;

server_name localhost;

allow 127.0.0.0/8;
deny all;

wallarm_mode off;
access_log off;

location /wallarm-status {
wallarm_status on format=prometheus;
}
}`

ddimych commented

Bug in wallarm module has been confirmed by Wallarm dev team, will be fixed in next module release.
Thanks for you help.

  • 👍1
lucacome commented

Thanks for letting us know!