dotnet-counters does not report values from multiple Metrics Meters with distinct tags
PJB3005 opened this issue · 2 comments
Description
I was adding built-in support for System.Diagnostics.Metrics to a library I maintain. In the library I want it to be possible to report separate metrics for each instance of a network socket, so I made each object instantiate a Meter and have an API for passing in a list of tags to said meter to distinguish them. Then I can just instantiate a new observable counter/gauge for each property I want to report and it looks pretty nice. For reference, here is the code.
However, dotnet counters only seems to report the metrics from one meter. In this example I have a separate meter with bind=:: and bind=0.0.0.0 created, but I only get results for bind=::.
Name Current Value
[Lidgren.Network.NetPeer]
messages_dropped (Count)
bind=:: 0
messages_received (Count)
bind=:: 97,375
messages_resent (Count)
bind=::,drop_reason=delay 20
bind=::,drop_reason=hole 481
messages_sent (Count)
bind=:: 106,354
packets_received (Count)
bind=:: 82,033
packets_sent (Count)
bind=:: 102,192
recycle_pool_bytes (bytes)
bind=:: 922,600
sent_bytes (bytes)
bind=:: 8,875,636
storage_allocated_bytes (bytes)
bind=:: 932,688
Cross-referencing against prometheus-net1, these values are correctly reported however.
> curl localhost:44880/metrics --silent | rg -i lidgren
# HELP lidgren_network_netpeer_packets_sent Number of packets sent by this NetPeer. (ObservableCounter`1)
# TYPE lidgren_network_netpeer_packets_sent gauge
lidgren_network_netpeer_packets_sent{bind="0.0.0.0"} 93498
lidgren_network_netpeer_packets_sent{bind="::"} 94293
# HELP lidgren_network_netpeer_packets_received Number of packets received by this NetPeer. (ObservableCounter`1)
# TYPE lidgren_network_netpeer_packets_received gauge
lidgren_network_netpeer_packets_received{bind="0.0.0.0"} 74491
lidgren_network_netpeer_packets_received{bind="::"} 75749
# HELP lidgren_network_netpeer_messages_sent Number of messages sent by this NetPeer. (ObservableCounter`1)
# TYPE lidgren_network_netpeer_messages_sent gauge
lidgren_network_netpeer_messages_sent{bind="0.0.0.0"} 97988
lidgren_network_netpeer_messages_sent{bind="::"} 98752
I already noticed that the same thing happened if I created two separate instruments with identical name but different tags, but I figured that one was at least somewhat reasonable. Instead I ended up just using callback-form that returns multiple measurements for the counter that needed to report tags.
Is this intended behavior or a bug? If it's intended behavior, I suppose I should make a static Meter instance and use it for all peers, but that would be rather annoying and inflexible IMO.
Configuration
Tool: dotnet-counters, version 8.0.510501+8c08c89a0643d31db91e119b1adb463be3e0ffe5
.NET runtime I'm testing against: .NET 8.0.3
OS: Windows 11 Pro, 23H2, build 22631.3296
Architecture: x64
Environment: just running stuff directly from my IDE/terminal, no containers or whatever.
dotnet info
> dotnet --info
.NET SDK:
Version: 8.0.103
Commit: 6a90b4b4bc
Workload version: 8.0.100-manifests.4e94be9c
Runtime Environment:
OS Name: Windows
OS Version: 10.0.22631
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.103\
.NET workloads installed:
Workload version: 8.0.100-manifests.4e94be9c
There are no installed workloads to display.
Host:
Version: 9.0.0-preview.1.24080.9
Architecture: x64
Commit: 1d1bf92fcf
.NET SDKs installed:
8.0.100 [C:\Program Files\dotnet\sdk]
8.0.103 [C:\Program Files\dotnet\sdk]
9.0.100-preview.1.24101.2 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.0-preview.1.24081.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.0-preview.1.24080.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.0-preview.1.24081.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]
Environment variables:
Not set
global.json file:
E:\ss14\space-station-14\RobustToolbox\global.json
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download
Regression?
Meter/instrument tags are a new feature in .NET 8, so I doubt it. I had trouble before because the tags wouldn't show up at all, but that was fixed by just making sure my dotnet-counters was at the latest version.
Footnotes
I'm pretty sure are hitting a known issue tracked here and the current plan is to fix it during .NET 9 for the scenario involving dotnet-counters. I wouldn't take that as a promise, but I think its likely.
Just so you are aware OpenTelemetry.NET also doesn't support multiple Meters distinguished solely by tags and at the moment the OTel spec officially says that isn't supported. Others have also encountered this issue and asked them to please support it in the future. I believe they are looking into it but if you care about OpenTelmetry support you can always open an issue in OTel.NET repo. The changes needed to make dotnet-counters and OTel support this are fully independent of one another.
Hope that helps explain where things are!
-Noah
at the moment the OTel spec officially says that isn't supported.
Damn, that's kind of annoying. Guess I have to change my library code then. Thanks though!