Getting "frame too large error" with default configuration
Closed this issue · 7 comments
Describe the bug
When I download and install the release build of go-dnscollector, I get errors and nothing is logged:
INFO: 2024/07/23 23:34:03.841482 worker - [tap] dnstap - conn #26 - new connection from 127.0.0.1:59242 (localhost)
INFO: 2024/07/23 23:34:03.841568 worker - [tap] (conn #26) dnstap processor - enabled
INFO: 2024/07/23 23:34:03.842149 worker - [tap] (conn #26) dnstap processor - starting data collection
INFO: 2024/07/23 23:34:03.842253 worker - [tap] (conn #26) transformers applied: [normalize:qname-replace-nonprintable normalize:qname-lowercase]
INFO: 2024/07/23 23:34:03.842307 worker - [tap] (conn #26) dnstap processor - starting monitoring - refresh every 10s
ERROR: 2024/07/23 23:34:03.842582 worker - [tap] dnstap - conn #26 - stream initialization: frame too large error
ERROR: 2024/07/23 23:34:03.842603 worker - [tap] dnstap - conn #26 - framestream reader error: frame too large error
INFO: 2024/07/23 23:34:03.842653 worker - [tap] dnstap - (conn #26 - connection handler terminated
INFO: 2024/07/23 23:34:03.842712 worker - [tap] dnstap - conn #26 - cleanup the connection handler
INFO: 2024/07/23 23:34:03.842733 worker - [tap] (conn #26) dnstap processor - stopping monitor...
INFO: 2024/07/23 23:34:03.842738 worker - [tap] (conn #26) dnstap processor - monitor terminated
INFO: 2024/07/23 23:34:03.842741 worker - [tap] (conn #26) dnstap processor - stopping collect...
INFO: 2024/07/23 23:34:03.842749 worker - [tap] (conn #26) dnstap processor - collection terminated
INFO: 2024/07/23 23:34:03.842774 worker - [tap] dnstap - conn #26 - cleanup connection handler terminated
To Reproduce
- Configure dnsdist to log to localhost port 6000:
Part of /etc/dnsdist/dnsdist.conf
-- Copied from https://github.com/dmachard/lua-dnsdist-config-examples/blob/main/logging_protobuf.lua
-- Needs https://github.com/dmachard/go-dnscollector to receive
-- init remote logger
protobuf_logging = newRemoteLogger("127.0.0.1:6000")
-- log all queries
addAction(AllRule(), RemoteLogAction(protobuf_logging, nil, {serverID="pdns server"}))
-- log all replies
addResponseAction(AllRule(), RemoteLogResponseAction(protobuf_logging, nil, true, {serverID="pdns server"}))
-- log all replies from cache
addCacheHitResponseAction(AllRule(), RemoteLogResponseAction(protobuf_logging, nil, true, {serverID="pdns server"}))
-- default rule
addAction( AllRule(), PoolAction("pdns"))
-
wget https://github.com/dmachard/go-dnscollector/releases/download/v1.0.0-beta1/go-dnscollector_1.0.0-beta1_linux_amd64.tar.gz -
tar xzvf go-dnscollector_1.0.0-beta1_linux_amd64.tar.gz -
./go-dnscollector
Expected behavior
It should "just work"?
Additional context
- Version 1.0.0-beta1
Arghghgle.... wrong protocol selected!
Here's what I should have used in /etc/dnsdist/dnsdist.conf:
-- Copied from https://github.com/dmachard/lua-dnsdist-config-examples/blob/main/logging_dnstap.lua
-- listen on localhost
-- setLocal("0.0.0.0:53", {})
-- backend dns
------ newServer({address = "1.1.1.1:53", pool="default"})
-- init remote logger
tap_logging = newFrameStreamTcpLogger("127.0.0.1:6000")
-- log all queries
addAction(AllRule(), DnstapLogAction("dnsdist_server", tap_logging))
-- log all replies
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist_server", tap_logging))
-- log all replies from cache
addCacheHitResponseAction(AllRule(), DnstapLogResponseAction("dnsdist_server", tap_logging))
-- default rule
addAction( AllRule(), PoolAction("pdns"))
Ok, now I'm confused.
The logging_protobuf.lua example explicitly says
-- dnstap logging for dns traffic, can be used with the remote logger like https://github.com/dmachard/go-dnscollector
However, it doesn't work with the default configuration of go-dnscollector?
Next question: Assuming go-dnscollector can be configured to use either protocol, which protocol is better?
Next question: Assuming go-dnscollector can be configured to use either protocol, which protocol is better?
The default protocol, dnstap, is widely implemented across all DNS servers. On the other hand, the PowerDNS protocol (provided in logging_protobuf.lua ) includes more metadata.
I prefer dnstap because its logs are not dependent on any specific DNS product.
You might want to include the corresponding dns server configurations as comments inside the example lua files; it'll help newbies like me. 😄
Semi-related note: https://dmachard.github.io/posts/0007-dnscollector-install-binary/ has example configuration using an older syntax. May want to update that. 😄
You might want to include the corresponding dns server configurations as comments inside the example lua files; it'll help newbies like me. 😄
Maybe this can help
The problem I had was that DNSDist supports both Protobuf logging and DNSTAP (implemented with Protobuf), and DNSDist's syntax for both is very similar and easily confused.
It might help to include the correct DNSDist config with the matching DNSCollector config in the same file.
Or, if the two protocols are so similar, then it might be worthwhile to combine them and have the code auto-detect which it is.