ydbdash.sh script

Question

ydbdash.sh script

wwongpai opened this issue 3 years ago · 21 comments

Hi Raman,

Thank you for this wonderful extension.
Anyway I have been trying to scrape a metric from port 8001, but nothing shown up in Prometheus server side. I run "./ydbdash.sh start" with port 8001, I can't see 8001 as LISTEN port. I'm not sure if I miss somethings. If possible, pls kindly advise on this.

Thanks!

Answer 1 · 2021-09-10T12:02:25.000Z

Hi,

Try running the python script directly. First run:

export YOTTA_PROM_PORT="8001"

and then:

./ydbdash.py

This will run the process to the foreground

Also make sure that the top line of the script references an actual executable i.e. if /usr/bin/python doesn't exist on your system, change the line to the path of your installed Python.

Answer 2 · 2021-09-10T12:43:00.000Z

Thanks a lot!

I have been trying following your advise, found this error..
host:~/YottaDB-dashboard$ ./ydbdash.py Traceback (most recent call last): File "./ydbdash.py", line 442, in <module> REGISTRY.register(CustomCollector()) File "/home/ubuntu/.local/lib/python3.8/site-packages/prometheus_client/registry.py", line 26, in register names = self._get_names(collector) File "/home/ubuntu/.local/lib/python3.8/site-packages/prometheus_client/registry.py", line 66, in _get_names for metric in desc_func(): File "./ydbdash.py", line 285, in collect a.add_metric([title], int(stats[0])) ValueError: invalid literal for int() with base 10: ''

I am trying to run from python3 and python2. got the same error. Is this about passing float() into int()?

Answer 3 · 2021-09-10T13:55:07.000Z

Do you have YottaDB installed on the system and a link to the ydb executable in the system path?

Answer 4 · 2021-09-10T16:04:53.000Z

Hi Raman,

Yes, there's YottaDb installed and link to the ydb executable properly

host:~/.yottadb$ tree
.
├── V6.3-010_x86_64 -> r1.32_x86_64
├── r
│   ├── TESTROUT.m
│   ├── _zmgsilog.m
│   ├── gvstat.m
│   ├── gvstatprom.m

├── r1.32_x86_64
│   ├── g
│   │   ├── %ydbaim.dat
│   │   ├── %ydbocto.dat
│   │   ├── %ydbocto.mjl
│   │   ├── yottadb.dat
│   │   ├── yottadb.gld
│   │   └── yottadb.mjl
│   ├── o
│   │   ├── TESTROUT.o
│   │   ├── gvstat.o
│   │   ├── gvstatprom.o
│   │   └── utf8
│   └── r

Anyway do I miss anything?
Thanks and really appreciate your response.

Answer 5 · 2021-09-10T17:12:21.000Z

Do you have any routines at /data/yottadb/r1.32_x86_64/r level?

Also try to manually enter the yottadb environment from the command line through entering:

ydb

Then enter:

D proc^gvstatprom("DEFAULT")

and:

D accproc^gvstatprom("DEFAULT")

See if you get any errors.

Answer 6 · 2021-09-10T17:29:29.000Z

Looking at the line numbers associated with the error, it may actually be the following command that is the issue:

ydb <<< "D ^%FREECNT" | awk -v reg="DEFAULT" '$0 ~ reg { gsub("[%)]","",$5);printf "%s,%s,%s",$2,$3,$5 }'

Try and run it manually from the command line

Answer 7 · 2021-09-11T01:50:09.000Z

Hi Raman,

I run above commands the result is

YDB>D proc^gvstatprom("DEFAULT")
BTD:8,CAT:439,CFE:0,CFS:0,CFT:0,CFavg:0,CFsigma:0,CFvar:,CQS:0,CQT:0,CQavg:0,CQsigma:0,CQvar:,CTN
:437,CYS:0,CYT:0,CYavg:0,CYsigma:0,CYvar:,DEX:0,DFL:1,DFS:1,DRD:0,DTA:178,DWT:10,GET:400,JBB:4803
2,JEX:0,JFB:65536,JFL:0,JFS:4,JFW:5,JRE:0,JRI:1,JRL:436,JRO:3,JRP:6,KIL:1,LKF:0,LKS:2,LKfrate:0,N
BR:2190,NBW:399,NR0:0,NR1:0,NR2:0,NR3:0,NTR:658,NTW:437,ORD:72,QRY:0,SET:435,TBR:0,TBW:0,TC0:0,TC
1:0,TC2:0,TC3:0,TC4:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,TRB:0,TTR:0,TTW:0,ZPR:3,ZTR:0,

YDB>D accproc^gvstatprom("DEFAULT")
BTD:25,CAT:1126,CFE:0,CFS:0,CFT:0,CFavg:0,CFsigma:0,CFvar:0,CQS:0,CQT:0,CQavg:0,CQsigma:0,CQvar:0
,CTN:1119,CYS:0,CYT:0,CYavg:0,CYsigma:0,CYvar:0,DEX:0,DFL:5,DFS:5,DRD:1,DTA:532,DWT:24,GET:868,JB
B:115920,JEX:0,JFB:163840,JFL:0,JFS:10,JFW:16,JRE:2,JRI:3,JRL:1117,JRO:9,JRP:14,KIL:2,LKF:0,LKS:6
,LKfrate:0,NBR:5522,NBW:1095,NR0:0,NR1:0,NR2:0,NR3:0,NTR:1642,NTW:1119,ORD:216,QRY:0,SET:1115,TBR
:0,TBW:0,TC0:0,TC1:0,TC2:0,TC3:0,TC4:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,TRB:0,TTR:0,TTW:0,ZPR:8,ZTR:
0,

and

ubuntu@host:~/.yottadb/r1.32_x86_64/r$ ydb <<< "D ^%FREECNT" | awk -v reg="DEFAULT" '$0 ~ reg { gsub("[%)]","",$5);printf "%s,%s,%s",$2,$3,$5 }'
4987,5000,99.7

However once run ydbdash.py, still got the same error as earlier.

I have routines at yottadb/r1.32_x86_64/r level as below,

ubuntu@host:~/.yottadb$ tree
.
├── V6.3-010_x86_64 -> r1.32_x86_64
├── r
│   ├── TESTROUT.m
│   ├── _zmgsilog.m
│   ├── gvstat.m
│   ├── gvstatprom.m
├── r1.32_x86_64
│   ├── g
│   │   ├── %ydbaim.dat
│   │   ├── %ydbocto.dat
│   │   ├── %ydbocto.mjl
│   │   ├── yottadb.dat
│   │   ├── yottadb.gld
│   │   └── yottadb.mjl
│   ├── o
│   │   ├── TESTROUT.o
│   │   ├── gvstat.o
│   │   ├── gvstatprom.o
│   │   └── utf8
│   └── r
│   ├── _zmgsilog.m
│   ├── gvstat.m
│   └── gvstatprom.m

If we run ydbdash.sh instead, how to fix it to run as a foreground to be exposed metric for scraping?
Thanks.

Answer 8 · 2021-09-11T08:49:31.000Z

What versions of Python 2 and 3 are you using?

Answer 9 · 2021-09-11T09:12:39.000Z

From the error output, it looks like stats[0] is returning a blank string. Should be 4987 though?

Run the following code in a python console:

import subprocess
from prometheus_client.core import GaugeMetricFamily, StateSetMetricFamily, REGISTRY, CounterMetricFamily
from prometheus_client import start_http_server, Enum
cmd = "ydb <<< "D ^%FREECNT" | awk -v reg="" + reg + "" '$0 ~ reg { gsub("[%)]","",$5);printf "%s,%s,%s",$2,$3,$5 }'" process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True) result = process.communicate()
print(result)
print(result[0])
stats=result[0].decode('ascii')
print(stats)
stats=str(stats).split(",")
print(stats)
a = GaugeMetricFamily("Free","Free Space", labels=[job])
a.add_metric([title], int(stats[0]))
yield a

Answer 10 · 2021-09-15T08:05:57.000Z

Since you are getting a syntax error with awk, can I ask you to post your version of awk? Type:

awk -V

at the command prompt. A lot of the awk code uses functions specifics to the GNU version of awk

Answer 11 · 2021-09-15T08:32:54.000Z

Hi Raman,

After input reg = '8001', then code can be run successfully. However the final result is still NULL as stats value got blank.

I have checked and found there's something wrong since running result = process.communicate()

>>> print(process)
<subprocess.Popen object at 0x7f0ab32eb100>
>>> result = process.communicate()
>>> print(result)
(b'', b'/bin/sh: 1: Syntax error: redirection unexpected\n')

I re-check whether there's output from %FREECNT and it works properly.

YDB>D ^%FREECNT
Region          Free     Total          Database file
------          ----     -----          -------------
DEFAULT         4998      5000 (100.0%) /home/ubuntu/.yottadb/r1.32_x86_64/g/yottadb.dat
YDBAIM         19998     20000 (100.0%) /home/ubuntu/.yottadb/r1.32_x86_64/g/%ydbaim.dat
YDBOCTO         9998     10000 (100.0%) /home/ubuntu/.yottadb/r1.32_x86_64/g/%ydbocto.dat

What's the expected outcome from process.communicate()?
Thanks.

Answer 12 · 2021-09-15T08:43:45.000Z

The value of the variable reg is determined by the environmental variable YOTTA_PROM_REG. If this is not set, reg will default to "DEFAULT"

Answer 13 · 2021-09-15T08:46:26.000Z

Are you running bash or sh from the command line when it runs successfully?

Answer 14 · 2021-09-15T08:58:54.000Z

Hi Raman,

I am running from Python console,

Python 3.8.10 (default, Jun  2 2021, 10:49:15) 

>>> import subprocess
>>> from prometheus_client.core import GaugeMetricFamily, StateSetMetricFamily, REGISTRY, CounterMetricFamily
>>> from prometheus_client import start_http_server, Enum
>>> reg='DEFAULT'
>>> cmd = "ydb <<< \"D ^%FREECNT\" | awk -v reg=\"" + reg + "\" '$0 ~ reg { gsub(\"[%)]\",\"\",$5);printf \"%s,%s,%s\",$2,$3,$5 }'"
>>> process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True) 
>>> print(process)
<subprocess.Popen object at 0x7fd6e624f460>
>>> result = process.communicate()
>>> print(result)
(b'', b'/bin/sh: 1: Syntax error: redirection unexpected\n')

run from python script got the same error,

ubuntu@host:~/YottaDB-dashboard$ export YOTTA_PROM_PORT="8001"
ubuntu@host:~/YottaDB-dashboard$ ./ydbdash.py
Traceback (most recent call last):
  File "./ydbdash.py", line 442, in <module>
    REGISTRY.register(CustomCollector())
  File "/home/ubuntu/.local/lib/python3.8/site-packages/prometheus_client/registry.py", line 26, in register
    names = self._get_names(collector)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/prometheus_client/registry.py", line 66, in _get_names
    for metric in desc_func():
  File "./ydbdash.py", line 285, in collect
    a.add_metric([title], int(stats[0]))
ValueError: invalid literal for int() with base 10: ''

Answer 15 · 2021-09-15T15:18:14.000Z

OK from the feedback so far, the command line:

ubuntu@host:~/.yottadb/r1.32_x86_64/r$ ydb <<< "D ^%FREECNT" | awk -v reg="DEFAULT" '$0 ~ reg { gsub("[%)]","",$5);printf "%s,%s,%s",$2,$3,$5 }'
4987,5000,99.7

Returns the data required. Namely the Amount of Free space in the ydb DEFAULT region, the total space in the ydb DEFAULT region and the % free space in the ydb DEFAULT region.

When the command is actioned from Python, the command is failing though.

Try print(cmd) in Python to ensure that the cmd generated from within Python matches the one actioned from the command line (the command that works).

Answer 16 · 2021-09-15T15:29:26.000Z

Strictly the awk command should have brackets around printf and so should be:

ydb <<< "D ^%FREECNT" | awk -v reg="DEFAULT" '$0 ~ reg { gsub("[%)]","",$5);printf("%s,%s,%s",$2,$3,$5) }'

Therefore try:

import subprocess
from prometheus_client.core import GaugeMetricFamily, StateSetMetricFamily, REGISTRY, CounterMetricFamily
from prometheus_client import start_http_server, Enum
reg='DEFAULT'
cmd = "ydb <<< "D ^%FREECNT" | awk -v reg="" + reg + "" '$0 ~ reg { gsub("[%)]","",$5);printf( "%s,%s,%s",$2,$3,$5) }'"
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
result = process.communicate()
print(result)

Answer 17 · 2021-09-16T01:53:05.000Z

Got the same result,

Python 3.8.10 (default, Jun 2 2021, 10:49:15)

>>> import subprocess
>>> from prometheus_client.core import GaugeMetricFamily, StateSetMetricFamily, REGISTRY, CounterMetricFamily
>>> from prometheus_client import start_http_server, Enum
>>> reg='DEFAULT'
>>> cmd = "ydb <<< "D ^%FREECNT" | awk -v reg="" + reg + "" '$0 ~ reg { gsub("[%)]","",$5);printf( "%s,%s,%s",$2,$3,$5) }'"
  File "<stdin>", line 1
    cmd = "ydb <<< "D ^%FREECNT" | awk -v reg="" + reg + "" '$0 ~ reg { gsub("[%)]","",$5);printf( "%s,%s,%s",$2,$3,$5) }'"
                    ^
SyntaxError: invalid syntax

Then change to run this instead,

>>> cmd = "ydb <<< \"D ^%FREECNT\" | awk -v reg=\"" + reg + "\" '$0 ~ reg { gsub(\"[%)]\",\"\",$5);printf(\"%s,%s,%s\",$2,$3,$5) }'"
>>> print(cmd)
ydb <<< "D ^%FREECNT" | awk -v reg="DEFAULT" '$0 ~ reg { gsub("[%)]","",$5);printf("%s,%s,%s",$2,$3,$5) }'
>>> process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
>>> print(process)
<subprocess.Popen object at 0x7feb69244460>
>>> result = process.communicate()
>>> print(result)
(b'', b'/bin/sh: 1: Syntax error: redirection unexpected\n')

Is it correct that the process return "<subprocess.Popen object at 0x7feb69244460>"?

Thanks,
Warach

Answer 18 · 2021-09-16T08:25:52.000Z

OK, I get the feeling that your default shell isn't bash and so we may need to change the cmd variable to explicitly specify bash. Try:

import subprocess
from prometheus_client.core import GaugeMetricFamily, StateSetMetricFamily, REGISTRY, CounterMetricFamily
from prometheus_client import start_http_server, Enum
reg='DEFAULT'
cmd = "/bin/bash -c \"ydb <<< \\\"D ^%FREECNT\\\" | awk -v reg=\\\"" + reg + "\\\" '\$0 ~ reg { gsub(\\\"[%)]\\\",\\\"\\\",\\$5);printf(\\\"%s,%s,%s\\\",\\$2,\\$3,\\$5)}'\""
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
print(process)
<subprocess.Popen object at 0x7feb69244460>
result = process.communicate()
print(result)

Answer 19 · 2021-09-22T14:21:31.000Z

Did you manage to get any further with this?

Answer 20 · 2021-09-22T14:43:12.000Z

Hi Raman,

Sorry for late response.
I managed to run on Python console successfully. It was all about shell that causing the error before, even my default shell is bash. Once change the cmd variable to explicitly specify bash, then all codes can be run successfully in the Python console.

I have been trying to change in the ydbdash.py code accordingly to explicitly specify bash in every lines that is required bash to run ydb command. It's still not 100% completed. Anyway let close this issue.

Thanks for asking, really appreciate.
Warach

Answer 21 · 2021-09-22T15:00:26.000Z

No worries Warach, I'll close the issue.