sflow/host-sflow

I can't see container graphics on my Ganglia

raimonbosch opened this issue · 10 comments

raimonbosch commented

Hi,

I have installed sflow host in order to monitor my docker containers using:

make DOCKER=yes LIBVIRT=yes VRTDSKPATH=yes
sudo make install
sudo make schedule

After hsflow is running I can see my specific statistics for virtual machines, but not for containers. Is there any extra configuration that we need to change in order to see this graphics?

This is my hsflowd.conf:
sflow{
DNSSD = off
polling = 10
sampling = 512
sampling.http = 100
sampling.memcache = 400

collector {
ip = 127.0.0.1
udpport = 6343
}
}

My gmond.conf:
globals {
daemonize = yes
setuid = yes
user = nobody
debug_level = 0
max_udp_msg_len = 1472
mute = yes /* don't send metrics /
deaf = no / listen for metrics */
allow_extra_data = yes
host_dmax = 0
host_tmax = 20
cleanup_threshold = 300
gexec = no
send_metadata_interval = 0
}
cluster {
name = "DoCluster"
owner = "unspecified"
latlong = "unspecified"
url = "unspecified"
}
host {
location = "unspecified"
}
udp_recv_channel {
port = 8649
}
tcp_accept_channel {
port = 8649
}

udp_recv_channel {
port = 6343
}

sflow {
udp_port = 6343
accept_vm_metrics = yes
}

And my gmetad.conf:
data_source "DoCluster" 10 localhost
all_trusted on
case_sensitive_hostnames 0

Thanks in advance,

sflow commented

Is it common to run VMs and Containers on the same server? It looks like
hsflowd assumes that they are mutually exclusive.

We'll have to make some changes to support the use case where you run both.

Neil

Neil McKee
InMon Corp.
http://www.inmon.com

On Wed, Feb 3, 2016 at 3:51 AM, Raimon Bosch notifications@github.com
wrote:

Hi,

I have installed sflow host in order to monitor my docker containers using:

make DOCKER=yes LIBVIRT=yes VRTDSKPATH=yes
sudo make install
sudo make schedule

After hsflow is running I can see my specific statistics for virtual
machines, but not for containers Is there any extra configuration that we
need to change in order to see this graphics?

This is my hsflowdconf
`sflow{
DNSSD = off
polling = 10
sampling = 512
samplinghttp = 100
samplingmemcache = 400

collector {
ip = 127001
udpport = 6343
}
#collector {

ip = serverIP2

udpport = 6343

#}
}
end configuration file`

And my gmondconf
`/* Configuration settings for a pure sFlow receiver
/ / Delete all udp_send_channel, modules, collection_group and include
sections

/ globals { daemonize = yes setuid = yes user = nobody debug_level = 0
max_udp_msg_len = 1472 mute = yes / don't send metrics
/ deaf = no / listen for metrics */
allow_extra_data = yes
host_dmax = 0
host_tmax = 20
cleanup_threshold = 300
gexec = no
send_metadata_interval = 0
}

cluster {
name = "DoCluster"
owner = "unspecified"
latlong = "unspecified"
url = "unspecified"
}
host {
location = "unspecified"
}

/* channel to receive gmetric messages
/ / eliminate mcast_join - sFlow is a unicast protocol

/ udp_recv_channel { port = 8649 } / channel to service requests for
XML data from gmetad */
tcp_accept_channel {
port = 8649
}

/* channel to receive sFlow
/ / 6343 is the default sFlow port, an explicit sFlow

/ / configuration section is needed to override default
*/
udp_recv_channel {
port = 6343
}

/* Optional sFlow settings */
sflow {
udp_port = 6343
accept_vm_metrics = yes
accept_jvm_metrics = yes multiple_jvm_instances = no accept_http_metrics
= yes multiple_http_instances = no accept_memcache_metrics = yes multiple_memcache_instances
= no

}
/* end of configuration file */`

And my gmetadconf:

data_source "DoCluster" 10 localhost
case_sensitive_hostnames 0

Thanks in advan

—
Reply to this email directly or view it on GitHub
#6.

raimonbosch commented

If I compile with:

make DOCKER=yes

I bump into the same problem. In this case I don't see neither VM stats nor container stats. I'm not sure what is failing.

[UPDATE]
I get this warnings during compilation time:
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.2 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -DHSF_VRT -I/usr/include/libvirt -I/usr/include/libxml2 -c hsflowd.c

hsflowd.c:285:14: warning: ‘getContainerPeerAdaptors’ defined but not used [-Wunused-function]
static int getContainerPeerAdaptors(HSP *sp, HSPVMState *vm, SFLAdaptorList *peerAdaptors)
^
hsflowd.c:1215:14: warning: ‘dockerContainerCB’ defined but not used [-Wunused-function]
static int dockerContainerCB(void *magic, char *line) {
^
hsflowd.c:1224:14: warning: ‘dockerInspectCB’ defined but not used [-Wunused-function]
static int dockerInspectCB(void *magic, char *line) {
^

raimonbosch commented

Ok,

I kind of solved it. But as you said, it's not compatible with both systems.

If you compile the last version of the code with:

make DOCKER=yes LIBVIRT=no XEN_DDK=no

you avoid the warning and then you get statistics about the containers as if they were virtual machines.

sflow commented

FYI, the latest version of hsflowd (1.29.1) will monitor both Docker containers and libvirt KVMs at the same time. Just build from sources with "make LIBVIRT=yes DOCKER=yes" to include both features. If you test this, please let me know how it goes.

sflow commented

Thanks for testing. "sync_receiver" should be "sync_agent". I have
checked in the change. Please "git pull" and try again.

Neil

Neil McKee
InMon Corp.
http://www.inmon.com

On Thu, Apr 14, 2016 at 7:45 AM, Raimon Bosch notifications@github.com
wrote:

Now I am trying the same setup "make DOCKER=yes" on:

Distributor ID: Debian
Description: Debian GNU/Linux 7.8 (wheezy)
Release: 7.8
Codename: wheezy

No luck until the moment and the log is really uninformative. I had to go
back to version 1.25.3 to avoid the problems with glibc so compilation
seems correct now, but I don't see any statistic related with my docker
containers.

How can I debug possible problems?

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#6 (comment)

raimonbosch commented

Hi Neil,

How can we avoid problems with glibc on:

Distributor ID: Debian
Description:    Debian GNU/Linux 7.8 (wheezy)
Release:    7.8
Codename:   wheezy

I got this error while compiling with DOCKER=yes on version 1.26.0 of host-sflow:

sudo make DOCKER=yes

cd src/sflow; make
make[1]: Entering directory `/home/rbosch/host-sflow/src/sflow'
gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_agent.c
gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_sampler.c
gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_poller.c
gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_receiver.c
ar cru libsflow.a sflow_agent.o sflow_sampler.o sflow_poller.o sflow_receiver.o
ranlib libsflow.a
make[1]: Leaving directory `/home/rbosch/host-sflow/src/sflow'
cd src/json; make
make[1]: Entering directory `/home/rbosch/host-sflow/src/json'
gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c cJSON.c
ar cru libjson.a cJSON.o
ranlib libjson.a
make[1]: Leaving directory `/home/rbosch/host-sflow/src/json'
PLATFORM=`uname`; \
    MYVER=`./getVersion`; \
        MYREL=`./getRelease`; \
        cd src/$PLATFORM; make VERSION=$MYVER RELEASE=$MYREL
make[1]: Entering directory `/home/rbosch/host-sflow/src/Linux'
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowconfig.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c dnsSD.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowd.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c util.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readInterfaces.c
readInterfaces.c: In function ‘readContainerInterfaces’:
readInterfaces.c:606:7: warning: implicit declaration of function ‘setns’ [-Wimplicit-function-declaration]
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readCpuCounters.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNvmlCounters.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readMemoryCounters.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readDiskCounters.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readHidCounters.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNioCounters.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readPackets.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readJSON.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readContainerCounters.c
gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -o hsflowd hsflowconfig.o dnsSD.o hsflowd.o util.o readInterfaces.o readCpuCounters.o readNvmlCounters.o readMemoryCounters.o readDiskCounters.o readHidCounters.o readNioCounters.o readPackets.o readJSON.o readContainerCounters.o ../sflow/libsflow.a -lresolv -lpthread ../json/libjson.a -lm ../json/libjson.a -lm -lcap
readInterfaces.o: In function `readContainerInterfaces':
readInterfaces.c:(.text+0xdc3): undefined reference to `setns'
collect2: error: ld returned 1 exit status
make[1]: *** [hsflowd] Error 1
make[1]: Leaving directory `/home/rbosch/host-sflow/src/Linux'
make: *** [hsflowd] Error 2

Is it possible to avoid this dependencies and still get statistics from Docker containers?

Thanks,

sflow commented

I don't have a Debian 7.8 system to test on right now, but here's the
relevant section of code:

https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L669-L678

The "setns" invocation is the __NR_setns system call, and my understanding
was that prior to kernel 3.0.0 it had to be invoked a different way. So
it's possible that you have to tweak this section of code -- perhaps just
changing the (3,0,0) to something like (3,8,0) or whatever you see in the
output of "uname -a" so that it invokes the system call the old way. Then
"make clean" before "make DOCKER=yes".

If I get time to download a Debian 7.8 then I'll have a look myself.

If you don't care about filling in the network I/O stats for the containers
then you could avoid this step altogether. You would still get the CPU and
memory stats.

If you can use a newer OS, however, then you have more options. For
example, Google managed to get a new feature added to the Linux kernel
(3.19 and later), that allows us to do this:
http://blog.sflow.com/2016/04/docker-network-visibility-demonstration.html

Neil

Neil McKee
InMon Corp.
http://www.inmon.com

On Thu, Apr 14, 2016 at 8:37 AM, Raimon Bosch notifications@github.com
wrote:

Hi Neil,

How can we avoid problems with glibc on:

Distributor ID: Debian
Description: Debian GNU/Linux 7.8 (wheezy)
Release: 7.8
Codename: wheezy

I got this error while compiling with Docker option on 1.26.0 "sudo make
DOCKER=yes LIBVIRT=no XEN_DDK=no":

cd src/sflow; make
make[1]: Entering directory /home/rbosch/host-sflow/src/sflow' gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_agent.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_sampler.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_poller.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_receiver.c ar cru libsflow.a sflow_agent.o sflow_sampler.o sflow_poller.o sflow_receiver.o ranlib libsflow.a make[1]: Leaving directory/home/rbosch/host-sflow/src/sflow'
cd src/json; make
make[1]: Entering directory /home/rbosch/host-sflow/src/json' gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c cJSON.c ar cru libjson.a cJSON.o ranlib libjson.a make[1]: Leaving directory/home/rbosch/host-sflow/src/json'
PLATFORM=uname;
MYVER=./getVersion;
MYREL=./getRelease;
cd src/$PLATFORM; make VERSION=$MYVER RELEASE=$MYREL
make[1]: Entering directory /home/rbosch/host-sflow/src/Linux' gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowconfig.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c dnsSD.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowd.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c util.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readInterfaces.c readInterfaces.c: In function ‘readContainerInterfaces’: readInterfaces.c:606:7: warning: implicit declaration of function ‘setns’ [-Wimplicit-function-declaration] gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readCpuCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNvmlCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readMemoryCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readDiskCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readHidCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNioCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readPackets.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readJSON.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readContainerCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -o hsflowd hsflowconfig.o dnsSD.o hsflowd.o util.o readInterfaces.o readCpuCounters.o readNvmlCounters.o readMemoryCounters.o readDiskCounters.o readHidCounters.o readNioCounters.o readPackets.o readJSON.o readContainerCounters.o ../sflow/libsflow.a -lresolv -lpthread ../json/libjson.a -lm ../json/libjson.a -lm -lcap readInterfaces.o: In functionreadContainerInterfaces':
readInterfaces.c:(.text+0xdc3): undefined reference to setns' collect2: error: ld returned 1 exit status make[1]: *** [hsflowd] Error 1 make[1]: Leaving directory/home/rbosch/host-sflow/src/Linux'
make: *** [hsflowd] Error 2

Is it possible to avoid this dependencies and still get statistics from
Docker containers?

Thanks,

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#6 (comment)

sflow commented

I installed a Debian 7.8 system. /usr/include/linux/version.h had
LINUX_VERSION_CODE 197198 (=> KERNEL_VERSION(3,2,78) while uname -r showed
"3.2.0-4-amd64". Somewhat unexpected, but anyway, changing
src/Linux/readInterfaces.c:670 to:

#if LINUX_VERSION_CODE < KERNEL_VERSION(3,2,79)

or just:

#if 1

meant that hsflowd could compile OK.

I didn't go any further than that. Getting Docker to work properly on an
OS that old seems fraught (moby/moby#16878)
but perhaps you already went through all that.

Neil

Neil McKee
InMon Corp.
http://www.inmon.com

On Thu, Apr 14, 2016 at 9:32 AM, Neil McKee neil.mckee@inmon.com wrote:

I don't have a Debian 7.8 system to test on right now, but here's the
relevant section of code:

https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L669-L678

The "setns" invocation is the __NR_setns system call, and my
understanding was that prior to kernel 3.0.0 it had to be invoked a
different way. So it's possible that you have to tweak this section of
code -- perhaps just changing the (3,0,0) to something like (3,8,0) or
whatever you see in the output of "uname -a" so that it invokes the system
call the old way. Then "make clean" before "make DOCKER=yes".

If I get time to download a Debian 7.8 then I'll have a look myself.

If you don't care about filling in the network I/O stats for the
containers then you could avoid this step altogether. You would still get
the CPU and memory stats.

If you can use a newer OS, however, then you have more options. For
example, Google managed to get a new feature added to the Linux kernel
(3.19 and later), that allows us to do this:
http://blog.sflow.com/2016/04/docker-network-visibility-demonstration.html

Neil

Neil McKee
InMon Corp.
http://www.inmon.com

On Thu, Apr 14, 2016 at 8:37 AM, Raimon Bosch notifications@github.com
wrote:

Hi Neil,

How can we avoid problems with glibc on:

Distributor ID: Debian
Description: Debian GNU/Linux 7.8 (wheezy)
Release: 7.8
Codename: wheezy

I got this error while compiling with Docker option on 1.26.0 "sudo make
DOCKER=yes LIBVIRT=no XEN_DDK=no":

cd src/sflow; make
make[1]: Entering directory /home/rbosch/host-sflow/src/sflow' gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_agent.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_sampler.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_poller.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_receiver.c ar cru libsflow.a sflow_agent.o sflow_sampler.o sflow_poller.o sflow_receiver.o ranlib libsflow.a make[1]: Leaving directory/home/rbosch/host-sflow/src/sflow'
cd src/json; make
make[1]: Entering directory /home/rbosch/host-sflow/src/json' gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c cJSON.c ar cru libjson.a cJSON.o ranlib libjson.a make[1]: Leaving directory/home/rbosch/host-sflow/src/json'
PLATFORM=uname;
MYVER=./getVersion;
MYREL=./getRelease;
cd src/$PLATFORM; make VERSION=$MYVER RELEASE=$MYREL
make[1]: Entering directory /home/rbosch/host-sflow/src/Linux' gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowconfig.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c dnsSD.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowd.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c util.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readInterfaces.c readInterfaces.c: In function ‘readContainerInterfaces’: readInterfaces.c:606:7: warning: implicit declaration of function ‘setns’ [-Wimplicit-function-declaration] gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readCpuCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNvmlCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readMemoryCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readDiskCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readHidCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNioCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readPackets.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readJSON.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readContainerCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -o hsflowd hsflowconfig.o dnsSD.o hsflowd.o util.o readInterfaces.o readCpuCounters.o readNvmlCounters.o readMemoryCounters.o readDiskCounters.o readHidCounters.o readNioCounters.o readPackets.o readJSON.o readContainerCounters.o ../sflow/libsflow.a -lresolv -lpthread ../json/libjson.a -lm ../json/libjson.a -lm -lcap readInterfaces.o: In functionreadContainerInterfaces':
readInterfaces.c:(.text+0xdc3): undefined reference to setns' collect2: error: ld returned 1 exit status make[1]: *** [hsflowd] Error 1 make[1]: Leaving directory/home/rbosch/host-sflow/src/Linux'
make: *** [hsflowd] Error 2

Is it possible to avoid this dependencies and still get statistics from
Docker containers?

Thanks,

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#6 (comment)

sflow commented

I added a test for GLIBC >= 2.14 which should take care of this:
dc3fa92

Neil McKee
InMon Corp.
http://www.inmon.com

On Thu, Apr 14, 2016 at 2:41 PM, Neil McKee neil.mckee@inmon.com wrote:

I installed a Debian 7.8 system. /usr/include/linux/version.h had
LINUX_VERSION_CODE 197198 (=> KERNEL_VERSION(3,2,78) while uname -r showed
"3.2.0-4-amd64". Somewhat unexpected, but anyway, changing
src/Linux/readInterfaces.c:670 to:

#if LINUX_VERSION_CODE < KERNEL_VERSION(3,2,79)

or just:

#if 1

meant that hsflowd could compile OK.

I didn't go any further than that. Getting Docker to work properly on an
OS that old seems fraught (moby/moby#16878)
but perhaps you already went through all that.

Neil

Neil McKee
InMon Corp.
http://www.inmon.com

On Thu, Apr 14, 2016 at 9:32 AM, Neil McKee neil.mckee@inmon.com wrote:

I don't have a Debian 7.8 system to test on right now, but here's the
relevant section of code:

https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L669-L678

The "setns" invocation is the __NR_setns system call, and my
understanding was that prior to kernel 3.0.0 it had to be invoked a
different way. So it's possible that you have to tweak this section of
code -- perhaps just changing the (3,0,0) to something like (3,8,0) or
whatever you see in the output of "uname -a" so that it invokes the system
call the old way. Then "make clean" before "make DOCKER=yes".

If I get time to download a Debian 7.8 then I'll have a look myself.

If you don't care about filling in the network I/O stats for the
containers then you could avoid this step altogether. You would still get
the CPU and memory stats.

If you can use a newer OS, however, then you have more options. For
example, Google managed to get a new feature added to the Linux kernel
(3.19 and later), that allows us to do this:
http://blog.sflow.com/2016/04/docker-network-visibility-demonstration.html

Neil

Neil McKee
InMon Corp.
http://www.inmon.com

On Thu, Apr 14, 2016 at 8:37 AM, Raimon Bosch notifications@github.com
wrote:

Hi Neil,

How can we avoid problems with glibc on:

Distributor ID: Debian
Description: Debian GNU/Linux 7.8 (wheezy)
Release: 7.8
Codename: wheezy

I got this error while compiling with Docker option on 1.26.0 "sudo make
DOCKER=yes LIBVIRT=no XEN_DDK=no":

cd src/sflow; make
make[1]: Entering directory /home/rbosch/host-sflow/src/sflow' gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_agent.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_sampler.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_poller.c gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c sflow_receiver.c ar cru libsflow.a sflow_agent.o sflow_sampler.o sflow_poller.o sflow_receiver.o ranlib libsflow.a make[1]: Leaving directory/home/rbosch/host-sflow/src/sflow'
cd src/json; make
make[1]: Entering directory /home/rbosch/host-sflow/src/json' gcc -D_GNU_SOURCE -DSTDC_HEADERS -O3 -DNDEBUG -Wall -Wcast-align -I. -c cJSON.c ar cru libjson.a cJSON.o ranlib libjson.a make[1]: Leaving directory/home/rbosch/host-sflow/src/json'
PLATFORM=uname;
MYVER=./getVersion;
MYREL=./getRelease;
cd src/$PLATFORM; make VERSION=$MYVER RELEASE=$MYREL
make[1]: Entering directory /home/rbosch/host-sflow/src/Linux' gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowconfig.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c dnsSD.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c hsflowd.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c util.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readInterfaces.c readInterfaces.c: In function ‘readContainerInterfaces’: readInterfaces.c:606:7: warning: implicit declaration of function ‘setns’ [-Wimplicit-function-declaration] gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readCpuCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNvmlCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readMemoryCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readDiskCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readHidCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readNioCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readPackets.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readJSON.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -c readContainerCounters.c gcc -std=gnu99 -I. -I../sflow -O3 -DNDEBUG -Wall -Wstrict-prototypes -Wunused-value -D_GNU_SOURCE -DHSP_VERSION=1.26.0 -DUTHEAP -DHSF_ULOG -DHSF_JSON -I../json -DHSF_DOCKER -I../json -o hsflowd hsflowconfig.o dnsSD.o hsflowd.o util.o readInterfaces.o readCpuCounters.o readNvmlCounters.o readMemoryCounters.o readDiskCounters.o readHidCounters.o readNioCounters.o readPackets.o readJSON.o readContainerCounters.o ../sflow/libsflow.a -lresolv -lpthread ../json/libjson.a -lm ../json/libjson.a -lm -lcap readInterfaces.o: In functionreadContainerInterfaces':
readInterfaces.c:(.text+0xdc3): undefined reference to setns' collect2: error: ld returned 1 exit status make[1]: *** [hsflowd] Error 1 make[1]: Leaving directory/home/rbosch/host-sflow/src/Linux'
make: *** [hsflowd] Error 2

Is it possible to avoid this dependencies and still get statistics from
Docker containers?

Thanks,

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#6 (comment)

raimonbosch commented

With the last changes seems to work fine on my version of Debian Wheezy. I was able to print docker and VM stats.

Thanks!