namespace as label

[popov-ilya]

First of all thank you for this exporter, it helps a lot.

This is more like question than issue, but why don't use namespaces as label("vhost" or "server" for example) instead of metrics prefix? For example, from:

app1_http_response_count_total{method="GET",status="405"}
app2_http_response_count_total{method="GET",status="405"}

to:

http_response_count_total{method="GET",status="405",vhost=~"app1|app2"}

Now in Grafana dashboard we need to make multiple queries for each metric because of this prefix. If we use label we can use only one query per panel. Thanks.

[martin-helmich]

but why don't use namespaces as label("vhost" or "server" for example) instead of metrics prefix?

No particular reason. You do present a valid use case, though. I'll see if I find some backwards-compatible way to adjust the metrics generation like this.

[popov-ilya]

Thanks!

[xanonid]

This would really be helpful. My suggestion would be to add a (experimental) feature flag that moves the namespace prefix into a label.

[tass6773]

This would be helpful. I would be will to try and figure this out if I had some guidances.

[zk101]

I ran into exactly this issue. I wanted all my nginxlog metrics to be in one grafana dashboard and have selectors to pick them out. You can move the namespace to a label and replace the namespace using metric_relabel. For example, in prometheus.yml:

  # Nginx Log Exporter
  - job_name: 'nginxlog_exporter'
    static_configs:
    - targets:
        - 'server:4040'
    relabel_configs:
      - source_labels: [__address__]
        regex: '(.*):.*'
        target_label: 'instance'
    metric_relabel_configs:
      - source_labels: [__name__]
        regex: '([a-zA-Z0-9]+)_http_.*'
        target_label: 'namespace'
      - source_labels: [__name__]
        regex: '[a-zA-Z0-9]+_http_(.*)'
        replacement: 'nginxlog_http_$1'
        target_label: '__name__'

I left my address relabel in, but this isn't needed, it just drops the port from the address which I prefer. The essence is, take the name, grab upto the first _ (this is the namespace), filtering only for metrics that also have http as the next piece. This is important as there are other metrics you don't want to match with this rule. All the actual metrics are namespace_http_blah. Dump the namespace into a label. The last bit replaces the namespace with nginxlog. That is what I wanted. The exporter already has some http_ metrics so I didn't want to collide with other naming. There is some hidden magic, which can be reveled in the prometheus config documentation. This example represents my own use case, hack as needed.

[hadmut]

Actually this is rather a design bug than an enhancement, since prometheus is designed to have similar time series under the same metrics with different labels.

Is there any chance to get this fixed or is this project orphaned?

regards

[tplueer]

I am currently struggling with the same issue. I want to create a dashboard variable inside to grafana to choose the website I want to see metrics for. Anybody knows a way to achieve this ?

[frenkye]

As @hadmut mention, this is design bug. The metrics should be under same name and differ by labels, otherwise it cannot be used specially in big enviroments.

How would you setup 10+ logs within single exporter in grafana? One by one? It does not make any sense. The label could be used as variable in dashboards to select just single namespace and/or use other custom lables for much more detailed view.

Would be nice to see change even with flag for start, which does:

1. Change namespace for prefix "nginxlog_" or something like that
2. Move namespace into label.

Edit: For our use case, we have over 400 servers runinng nginx for different customers. From single server to HA setups, which contains single site or hundreds for webhosting. This change would mean that i set up grafana for our env in like an hour or years (now).

[helmich-bot]

There has not been any activity to this issue in the last 30 days. It will automatically be closed after 7 more days, if there is no activity.

[martin-helmich]

Shut up, @helmich-bot.

Both from the number of comments in this issue and from the amount of 👍's I acknowledge that this is an important issue. However, as correctly stated, this is a design issue that would require quite a bit of refactoring (which would get even harder when maintaining backwards-compatibility).

I can't make any promises as to when (or if) I'll ever get to this, but I'm happy to review and accept any PR's coming my way.

[martin-helmich]

To everyone involved in this issue: It's been almost three years since this issue was initially reported, and it was finally fixed in #121 by @rivik, who found a very elegant solution for this issue. Thanks for your work on this. 🎉

In order to preserve backwards-compatibility, this is an opt-in feature, though. The readme tells more.

The new configuration options are available in v1.7.0.