netdata/netdata

[Bug]: False system_clock_sync_state Alert on QNAP

cheyngoodman opened this issue · 2 comments

cheyngoodman commented

Bug description

My QNAP has NTP enabled and it's system time appears accurate, but there is an alert in Netdata Cloud stating "the system kernel believes the system clock is not properly synchronized to a reliable server".

Expected behavior

An alert should not be triggered, since the system time is accurate.

Steps to reproduce

  1. Deploy Netdata to QNAPs Kubernetes cluster using Helm
  2. Wait for the QNAP to report to Netdata cloud
  3. An alert is triggered for system.clock_sync_state state == 0

Installation method

helmchart (kubernetes)

System info

Linux NAS46B15C 5.10.60-qnap #1 SMP Wed Mar 22 18:12:13 CST 2023 x86_64 GNU/Linux
/etc/os-release:NAME="QTS"
/etc/os-release:VERSION="5.0.1 (20230322)"
/etc/os-release:ID=qts
/etc/os-release:PRETTY_NAME="QTS 5.0.1 (20230322)"
/etc/os-release:VERSION_ID="5.0.1"

Netdata build info

Version: netdata v1.38.1
Configure options:  '--prefix=/usr' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=/usr/libexec' '--libdir=/usr/lib' '--with-zlib' '--with-math' '--with-user=netdata' '--without-bundled-protobuf' '--disable-dependency-tracking' '--disable-ebpf' 'CFLAGS=-O2 -pipe' 'LDFLAGS='
Install type: oci
    Binary architecture: x86_64
Features:
    dbengine:                   YES
    Native HTTPS:               YES
    Netdata Cloud:              YES 
    ACLK:                       YES
    TLS Host Verification:      YES
    Machine Learning:           YES
    Stream Compression:         YES
Libraries:
    protobuf:                YES (system)
    jemalloc:                NO
    JSON-C:                  YES
    libcap:                  NO
    libcrypto:               YES
    libm:                    YES
    tcalloc:                 NO
    zlib:                    YES
Plugins:
    apps:                    YES
    cgroup Network Tracking: YES
    CUPS:                    NO
    EBPF:                    NO
    IPMI:                    YES
    NFACCT:                  NO
    perf:                    YES
    slabinfo:                YES
    Xen:                     NO
    Xen VBD Error Tracking:  NO
Exporters:
    AWS Kinesis:             NO
    GCP PubSub:              NO
    MongoDB:                 YES
    Prometheus Remote Write: YES
Debug/Developer Features:
    Trace Allocations:       NO

Additional info

There was a similar issue that was closed but I'm not sure what to do to resolve the issue: #12604

From the QNAP OS I can run hwclock and ntpdate like in that issue, but not timedatectl since there's no SystemD.

Is this QNAP OS considered not supported and I should just disable this alarm? Is there an additional configuration I should set on the QNAP to resolve this alarm?

Is there more I can do to troubleshoot this value?

ilyam8 commented

Hi, @cheyngoodman. Netdata reports whatever the system clock state is according to the kernel - there is no additional processing.

If you are sure it is false-positive in your case you can disable the alarm using enabled alarms:

# netdata.conf
[health]
    enabled alarms = !system_clock_sync_state *

See this comment for a more detailed description of how timex collector works.

Is this QNAP OS considered not supported and I should just disable this alarm? Is there an additional configuration I should set on the QNAP to resolve this alarm?

Unfortunately, I don't know/can't answer 🤷‍♂️ I suggest you ask QNAP OS guys:

  • Netdata does ntp_adjtime() call.
  • Checks STA_UNSYNC bit in status.

STA_UNSYNC (read-write)
Clock unsynchronized.

cheyngoodman commented

Thank you, I will follow up with QNAP support regarding the call and status.

It seems the QNAP OS has a few quirks like #11180 too. Hopefully they can straighten things out a bit and if not I'll disable the related checks.