[Bug]: Main process not exit

Question

[Bug]: Main process not exit

aldotms70 opened this issue a month ago · 1 comments

aldotms70 commented a month ago

Bug description

Sending TERM signal to main process, all tasks are closed but the process wait forever the watcher thread.

Expected behavior

All processes have to terminate.

Steps to reproduce

Start netdata with deamon option
Send a TERM signal to main process
All child terminate but not the main and spwan server

Installation method

kickstart.sh

System info

Linux CylonCpu-374 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
/etc/lsb-release:DISTRIB_ID=Ubuntu
/etc/lsb-release:DISTRIB_RELEASE=12.04
/etc/lsb-release:DISTRIB_CODENAME=precise
/etc/lsb-release:DISTRIB_DESCRIPTION="Ubuntu 12.04 LTS"

Netdata build info

Packaging:
    Netdata Version ____________________________________________ : v1.45.0-200-g60e0831a6
    Installation Type __________________________________________ : kickstart-static
    Package Architecture _______________________________________ : x86_64
    Package Distro _____________________________________________ : unknown
    Configure Options __________________________________________ : dummy-configure-command
Default Directories:
    User Configurations ________________________________________ : /opt/netdata/etc/netdata
    Stock Configurations _______________________________________ : /opt/netdata/usr/lib/netdata/conf.d
    Ephemeral Databases (metrics data, metadata) _______________ : /opt/netdata/var/cache/netdata
    Permanent Databases ________________________________________ : /opt/netdata/var/lib/netdata
    Plugins ____________________________________________________ : /opt/netdata/usr/libexec/netdata/plugins.d
    Static Web Files ___________________________________________ : /opt/netdata/usr/share/netdata/web
    Log Files __________________________________________________ : /opt/netdata/var/log/netdata
    Lock Files _________________________________________________ : /opt/netdata/var/lib/netdata/lock
    Home _______________________________________________________ : /opt/netdata/var/lib/netdata
Operating System:
    Kernel _____________________________________________________ : Linux
    Kernel Version _____________________________________________ : 3.2.0-23-generic
    Operating System ___________________________________________ : Ubuntu
    Operating System ID ________________________________________ : precise
    Operating System ID Like ___________________________________ : unknown
    Operating System Version ___________________________________ : 12.04
    Operating System Version ID ________________________________ : none
    Detection __________________________________________________ : Mixed
Hardware:
    CPU Cores __________________________________________________ : 2
    CPU Frequency ______________________________________________ : 3191000000
    RAM Bytes __________________________________________________ : 1043816448
    Disk Capacity ______________________________________________ : 10737418240
    CPU Architecture ___________________________________________ : x86_64
    Virtualization Technology __________________________________ : kvm
    Virtualization Detection ___________________________________ : lscpu
Container:
    Container __________________________________________________ : unknown
    Container Detection ________________________________________ : none
    Container Orchestrator _____________________________________ : none
    Container Operating System _________________________________ : none
    Container Operating System ID ______________________________ : none
    Container Operating System ID Like _________________________ : none
    Container Operating System Version _________________________ : none
    Container Operating System Version ID ______________________ : none
    Container Operating System Detection _______________________ : none
Features:
    Built For __________________________________________________ : Linux
    Netdata Cloud ______________________________________________ : YES
    Health (trigger alerts and send notifications) _____________ : YES
    Streaming (stream metrics to parent Netdata servers) _______ : YES
    Back-filling (of higher database tiers) ____________________ : YES
    Replication (fill the gaps of parent Netdata servers) ______ : YES
    Streaming and Replication Compression ______________________ : YES (zstd lz4 gzip)
    Contexts (index all active and archived metrics) ___________ : YES
    Tiering (multiple dbs with different metrics resolution) ___ : YES (5)
    Machine Learning ___________________________________________ : YES
Database Engines:
    dbengine (compression) _____________________________________ : YES (zstd lz4)
    alloc ______________________________________________________ : YES
    ram ________________________________________________________ : YES
    none _______________________________________________________ : YES
Connectivity Capabilities:
    ACLK (Agent-Cloud Link: MQTT over WebSockets over TLS) _____ : YES
    static (Netdata internal web server) _______________________ : YES
    h2o (web server) ___________________________________________ : YES
    WebRTC (experimental) ______________________________________ : NO
    Native HTTPS (TLS Support) _________________________________ : YES
    TLS Host Verification ______________________________________ : YES
Libraries:
    LZ4 (extremely fast lossless compression algorithm) ________ : YES
    ZSTD (fast, lossless compression algorithm) ________________ : YES
    zlib (lossless data-compression library) ___________________ : YES
    Brotli (generic-purpose lossless compression algorithm) ____ : NO
    protobuf (platform-neutral data serialization protocol) ____ : YES (system)
    OpenSSL (cryptography) _____________________________________ : YES
    libdatachannel (stand-alone WebRTC data channels) __________ : NO
    JSON-C (lightweight JSON manipulation) _____________________ : YES
    libcap (Linux capabilities system operations) ______________ : NO
    libcrypto (cryptographic functions) ________________________ : YES
    libyaml (library for parsing and emitting YAML) ____________ : YES
Plugins:
    apps (monitor processes) ___________________________________ : YES
    cgroups (monitor containers and VMs) _______________________ : YES
    cgroup-network (associate interfaces to CGROUPS) ___________ : YES
    proc (monitor Linux systems) _______________________________ : YES
    tc (monitor Linux network QoS) _____________________________ : YES
    diskspace (monitor Linux mount points) _____________________ : YES
    freebsd (monitor FreeBSD systems) __________________________ : NO
    macos (monitor MacOS systems) ______________________________ : NO
    statsd (collect custom application metrics) ________________ : YES
    timex (check system clock synchronization) _________________ : YES
    idlejitter (check system latency and jitter) _______________ : YES
    bash (support shell data collection jobs - charts.d) _______ : YES
    debugfs (kernel debugging metrics) _________________________ : YES
    cups (monitor printers and print jobs) _____________________ : NO
    ebpf (monitor system calls) ________________________________ : YES
    freeipmi (monitor enterprise server H/W) ___________________ : NO
    nfacct (gather netfilter accounting) _______________________ : YES
    perf (collect kernel performance events) ___________________ : YES
    slabinfo (monitor kernel object caching) ___________________ : YES
    Xen ________________________________________________________ : NO
    Xen VBD Error Tracking _____________________________________ : NO
    Logs Management ____________________________________________ : NO
Exporters:
    AWS Kinesis ________________________________________________ : NO
    GCP PubSub _________________________________________________ : NO
    MongoDB ____________________________________________________ : NO
    Prometheus (OpenMetrics) Exporter __________________________ : YES
    Prometheus Remote Write ____________________________________ : YES
    Graphite ___________________________________________________ : YES
    Graphite HTTP / HTTPS ______________________________________ : YES
    JSON _______________________________________________________ : YES
    JSON HTTP / HTTPS __________________________________________ : YES
    OpenTSDB ___________________________________________________ : YES
    OpenTSDB HTTP / HTTPS ______________________________________ : YES
    All Metrics API ____________________________________________ : YES
    Shell (use metrics in shell scripts) _______________________ : YES
Debug/Developer Features:
    Trace All Netdata Allocations (with charts) ________________ : NO
    Developer Mode (more runtime checks, slower) _______________ : NO

Additional info

The problem is not present when using -D parameter. Forking the process after starting the watcher thread is wrong.
The forked process do not have the same memory space.
Solution: call watcher_thread_start in main after the function become_daemon

Answer 1 · 2024-04-17T17:50:26.000Z

@aldotms70 thank you for the report and the investigation!