signalfx/splunk-otel-collector

Crashloopbackoff on AKS Cluster with splunk-otel-collector:0.81.0

jmapro opened this issue · 3 comments

jmapro commented

Hi !

I've tried to deploy Splunk Otel Collector in AKS Cluster with the Helm Chart https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector

Everything works well with v0.80.0 but with 0.81.0 I have the following error using the exact same configuration

2023/07/24 09:17:06 settings.go:392: Set config to [/conf/relay.yaml]
2023/07/24 09:17:06 settings.go:445: Set ballast to 165 MiB
2023/07/24 09:17:06 settings.go:461: Set memory limit to 450 MiB
2023-07-24T09:17:06.166Z	info	service/telemetry.go:81	Setting up own telemetry...
2023-07-24T09:17:06.166Z	info	service/telemetry.go:104	Serving Prometheus metrics	{"address": "0.0.0.0:8889", "level": "Basic"}
panic: interface conversion: internal.DetectorConfig is nil, not aks.Config

goroutine 1 [running]:
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor/internal/azure/aks.NewDetector({{{0xc0026f1398, 0x11}, {0x0, 0x0}}, {0xc002872bd0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, 0x0, ...}, ...}, ...)
	github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor@v0.81.0/internal/azure/aks/aks.go:34 +0x10a
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor/internal.(*ResourceProviderFactory).getDetectors(0xc000425608, {{{0xc0026f1398, 0x11}, {0x0, 0x0}}, {0xc002872bd0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, ...}, ...}, ...)
	github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor@v0.81.0/internal/resourcedetection.go:73 +0x182
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor/internal.(*ResourceProviderFactory).CreateResourceProvider(0xc00028feb8?, {{{0xc0026f1398, 0x11}, {0x0, 0x0}}, {0xc002872bd0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, ...}, ...}, ...)
	github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor@v0.81.0/internal/resourcedetection.go:49 +0x9d
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor.(*factory).getResourceProvider(_, {{{0xc0026f1398, 0x11}, {0x0, 0x0}}, {0xc002872bd0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, ...}, ...}, ...)
	github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor@v0.81.0/factory.go:205 +0x3a5
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor.(*factory).getResourceDetectionProcessor(0xc00277bf60?, {{{0xc0026f1398, 0x11}, {0x0, 0x0}}, {0xc002872bd0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, ...}, ...}, ...)
	github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor@v0.81.0/factory.go:173 +0x145
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor.(*factory).createMetricsProcessor(0x7f029cc93108?, {0x68c3dd8, 0xc00013a000}, {{{0xc0026f1398, 0x11}, {0x0, 0x0}}, {0xc002872bd0, {0x6877700, 0xc002812100}, ...}, ...}, ...)
	github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor@v0.81.0/factory.go:129 +0x9e
go.opentelemetry.io/collector/processor.CreateMetricsFunc.CreateMetricsProcessor(...)
	go.opentelemetry.io/collector/processor@v0.81.0/processor.go:122
go.opentelemetry.io/collector/processor.(*Builder).CreateMetrics(0xc002801b60, {0x68c3dd8, 0xc00013a000}, {{{0xc0026f1398, 0x11}, {0x0, 0x0}}, {0xc002872bd0, {0x6877700, 0xc002812100}, ...}, ...}, ...)
	go.opentelemetry.io/collector/processor@v0.81.0/processor.go:259 +0x16d
go.opentelemetry.io/collector/service/internal/graph.(*processorNode).buildComponent(0xc00282c300, {0x68c3dd8, 0xc00013a000}, {0xc0027dc1c0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, 0x0, {0xc0028108e0}}, ...)
	go.opentelemetry.io/collector@v0.81.0/service/internal/graph/nodes.go:143 +0x45b
go.opentelemetry.io/collector/service/internal/graph.(*Graph).buildComponents(0xc0028227f0, {0x68c3dd8, 0xc00013a000}, {{0xc0027dc1c0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, 0x0, {0xc0028108e0}}, ...})
	go.opentelemetry.io/collector@v0.81.0/service/internal/graph/graph.go:179 +0x2a7
go.opentelemetry.io/collector/service/internal/graph.Build({0x68c3dd8, 0xc00013a000}, {{0xc0027dc1c0, {0x6877700, 0xc002812100}, {0x68a1850, 0xc002801ba0}, 0x0, {0xc0028108e0}}, {{0x5d7f33b, ...}, ...}, ...})
	go.opentelemetry.io/collector@v0.81.0/service/internal/graph/graph.go:63 +0x3d0
go.opentelemetry.io/collector/service.(*Service).initExtensionsAndPipeline(_, {_, _}, {{{0x5d7f33b, 0x7}, {0x0, 0x0}, {0x68571c0, 0x7}}, 0xc002801b50, ...}, ...)
	go.opentelemetry.io/collector@v0.81.0/service/service.go:204 +0x289
go.opentelemetry.io/collector/service.New({_, _}, {{{0x5d7f33b, 0x7}, {0x0, 0x0}, {0x68571c0, 0x7}}, 0xc002801b50, 0xc002801b60, ...}, ...)
	go.opentelemetry.io/collector@v0.81.0/service/service.go:117 +0x6f1
go.opentelemetry.io/collector/otelcol.(*Collector).setupConfigurationComponents(0xc000bab200, {0x68c3dd8, 0xc00013a000})
	go.opentelemetry.io/collector@v0.81.0/otelcol/collector.go:155 +0x54b
go.opentelemetry.io/collector/otelcol.(*Collector).Run(0xc000bab200, {0x68c3dd8, 0xc00013a000})
	go.opentelemetry.io/collector@v0.81.0/otelcol/collector.go:207 +0x65
go.opentelemetry.io/collector/otelcol.NewCommand.func1(0xc00234d800, {0x5d7fd7c?, 0x0?, 0x1?})
	go.opentelemetry.io/collector@v0.81.0/otelcol/command.go:27 +0x96
github.com/spf13/cobra.(*Command).execute(0xc00234d800, {0xc000134010, 0x0, 0x1})
	github.com/spf13/cobra@v1.7.0/command.go:940 +0x862
github.com/spf13/cobra.(*Command).ExecuteC(0xc00234d800)
	github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3bd
github.com/spf13/cobra.(*Command).Execute(0xc00240aa80?)
	github.com/spf13/cobra@v1.7.0/command.go:992 +0x19
main.runInteractive({{0xc00240aa80, 0xc00240ac60, 0xc00240aba0, 0xc00240a810, 0xc00240ac90}, {{0x5d7f33b, 0x7}, {0x0, 0x0}, {0x68571c0, ...}}, ...})
	github.com/signalfx/splunk-otel-collector/cmd/otelcol/main.go:100 +0x5d
main.run(...)
	github.com/signalfx/splunk-otel-collector/cmd/otelcol/main_others.go:26
main.main()
	github.com/signalfx/splunk-otel-collector/cmd/otelcol/main.go:93 +0xcba
omrozowicz-splunk commented

Hey, can you provide your values.yaml config?

jmapro commented

Hey !
Found that it's related to open-telemetry/opentelemetry-collector-contrib#24549

  • 👍1