netdata/helmchart

No k8s data showing up in cloud

rjbaat opened this issue · 12 comments

Hi All,
I setup the Netdata Agent by the helm chart following the default setup and video on YouTube: https://www.youtube.com/watch?v=-VFq_VtBwOc

There is an child and parent agent and data is showing on the cloud. But It looks like only performance metrics for the operation of netdata itself and its plugins is showing up. So only: web, statsd and aclk.

What do I need to do to get also the whole k8s data showing up?

Hi @rjbaat 👋

You have only netdata parent claimed, correct?

Yes I put in the claim token twice. For both parent and child in the helm chart. Like:

---
  child: 
    claiming: 
      enabled: "true"
      rooms: "same_room_id_from_cloud"
      token: "same_token_id_from_cloud"
      configs:
        netdata:
          data: |
            [global]
              memory mode = dbengine
              history = 3600
            [health]
              enabled = no
  parent: 
    alarms: 
      volumesize: "1Gi"
    claiming: 
      enabled: "true"
      rooms: "same_room_id_from_cloud"
      token: "same_token_id_from_cloud"

btw iam seeing more when doing a port forward like:

kubectl port-forward svc/netdata 19999:19999 -n netdata

Could it be that the cloud can't reach the child nodes?

Anyone? How can I get the child data visible on the cloud? Is there a firewall rulle I need to open or something? So data from the child can be streamed to the cloud or browser?

I'm having this exact same issue. I see a parent node only in cloud even though I specified the token and room for both. Only web, statsd, and aclk show under parent as well.

I think I ran into this as well when upgrading from 1.29.x to 1.30.1.

Hi, guys 👋 Sorry for the late response.

Yes, we need children instances claimed in order to see them on the Cloud. Cloud fetches data directly from the claimed nodes - despite the fact that the children's data is stored on the parent node we need children to have that data too.

So we need:

  • set claiming parameters: enabled, token, and rooms.
  • change memory mode to dbengine or ram.

Default is none, it means no data stored on the child instance => claimed OK, but no metrics on the Cloud.

configs:
netdata:
enabled: true
path: /etc/netdata/netdata.conf
data: |
[global]
memory mode = none


@rjbaat there is an error in your override.yaml: see configs.netdata.data - it is under claiming.

Should be

child:
  claiming:
    enabled: true
    rooms: "ROOMS"
    token: "TOKEN"
  configs:
    netdata:
      data: |
        [global]
          # ram or dbengine, not none
          memory mode = ram
          history = 3600
        [health]
          enabled = no

Another minor error: enabled is boolean, not string, so it is true or false, not "true" or "false".

The following is tested on my k8s cluster in GKE (it is a potato cluster, so I need to set resources.requests, you don't need it).

override.yaml [Click]
sd:
  child:
    resources:
      requests:
        cpu: 1m
        memory: 1Mi

child:
  resources:
    requests:
      cpu: 1m
      memory: 1Mi
  claiming:
    enabled: true
    rooms: "ROOMS"
    token: "TOKEN"
  configs:
    netdata:
      data: |
        [global]
          memory mode = ram
          history = 3600
        [health]
          enabled = no
parent:
  resources:
    requests:
      cpu: 1m
      memory: 1Mi
  claiming:
    enabled: true
    rooms: "ROOMS"
    token: "TOKEN"

Screenshot 2021-05-04 at 13 41 54

Only web, statsd, and aclk show under parent as well.

That is expected, the parent node doesn't collect any data (children instances do it). There are only internal metrics.

Hi @rjbaat,

Do @ilyam8's suggestions solve the issue? Can we close this?

I think yes, I've tested Netdata parent/children claiming and spotted few errors in the @rjbaat config (see detailed desciption #204 (comment)).

@rjbaat @digitalcraig @alechartung if there are still any problems feel free to comment, we will debug and fix them.