jtt/dns-collector

High speed passive DNS collector with dnstap support, dns traffic sniffer and more

DNS Collector

Overview

This dns collector acts as a high speed passive analyser for DNS traffic, written in Golang. Which give you the possibility to control dns servers, get statistics and more. It supports several methods to collect dns traffic (dnstap, sniffer, logs, etc.) and can redirect them to multiple destinations with protocol and format (json, text) transformation. This collector can also be used to logs dns answers.

NOTE: The code before version 1.x is considered beta quality and is subject to breaking changes.

Features

• Supported collectors:

  • DNStap streams
  • DNS packets sniffer
  • Tail on log file
  • Protobuf PowerDNS

• Supported loggers:

  • Stdout
  • File
  • DNStap
  • TCP
  • Prometheus
  • Syslog
  • Fluentd
  • Pcap
  • InfluxDB
  • Loki
  • Statsd

• Other features

  • Queries/Replies JSON encoding
  • DNS decoder with extended options support
  • Built-in Grafana dashboards
  • GeoIP support
  • DNS filtering
  • User Privacy
  • Custom text format
  • DNS caching
  • Normalize Qname

End to end testing

Tested with success with the following operating system

Tested with success with the following dns servers

Installation

Run-it from binary

Download the binary from release page. If you want to integrate this tool with systemd, please to follow this guide.

./go-dnscollector -config config.yml

Run-it from dockerhub

Use the default config (dnstap -> stdout + rest api):

docker run -d --name=dnscollector01 dmachard/go-dnscollector

Override the default configuration (/etc/dnscollector/config.yml) with a config file on the host:

-v $(pwd)/config.yml:/etc/dnscollector/config.yml

Configuration

See the full Configuration guide for more details.

Use-cases

As prerequisites, we assume you have a DNS server which supports DNSTap (unbound, bind, powerdns, etc)

For more informations about dnstap, please to read the following page Dnstap: How to enable it on main dns servers

• Use case 1: Capture dns traffic (dnstap) and backup-it to text log files
• Use case 2: Capture dns traffic (dnstap) and get statistics usage with Prometheus + Grafana
• Use case 3: Convert captured dns traffic (dnstap) to JSON format
• Use case 4: Capture dns traffic (dnstap) and follow dns logs with Loki + Grafana
• Use case 5: Forward unix dnstap socket traffic to TLS dnstap stream
• Use case 6: Capture dns traffic with user privacy options enabled

Benchmark

Tested on the following machine: 8 vCPUs, 32 GB memory

packet per sec received	DnsCollector
50k	OK - 0% lost
100k	OK - 0% lost
150k	OK (0.07% lost)

For developers

Run from source

go run .

Execute testunits

go test -timeout 10s ./collectors/ -cover -v
go test -timeout 10s ./loggers/ -cover -v
go test -timeout 10s ./subprocessors/ -cover -v
go test -timeout 10s ./dnsutils/ -cover -v

Execute a test for one specific testcase in a package

go test -timeout 10s -cover -v ./loggers -run TestSyslogRunJsonMode

Building from source. Use the latest golang available on your target system

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o go-dnscollector *.go