Bulkstat Parser

A parser for Cisco StarOS Bulkstats. The purpose is to identify potential relationship between counters. This is done by comparing how counters increment over a certain amount of bulkstat period. There is a tolerance to accept some deviation.

The tool can also compute a KPI formula on the fly.

As the same counter could be seen on different schemas, and also be seen on several lines generated by the same schema, we need to create a unique identity for each schema line. It might be possible that you see collision of names. In this case, please report the problem to gdufour@cisco.com.

Installation

Get the package using git

git clone git@github.com:dufourgilles/bulkstat_parser.git

Inside the git directory, install NPM packages.

npm install

Build the tool

npm run build

Usage

$ node lib/index.js --help
Options:
      --version     Show version number                                [boolean]
  -k, --key         Counter key.
  -c, --configfile  StarOS Config file or SSD.
  -b, --bulks       Bulkstat files comma separated and time base ordered.
  -d, --dir         directory containing the BS files
  -f, --formula     KPI Formula                                         [string]
  -o, --filterOut   Exclude line matching filter
  -s, --start       start time                                          [number]
  -e, --end         end time                                            [number]
  -p, --pace        step increment for file selection                   [number]
  -t, --tolerance   Tolerance percentage for counter match.[number] [default: 1]
      --help        Show help                                          [boolean]

Find counters that are tied to each other

In this example, we want to find all counters that follow %tun-sent-cresessrespdeniedOtherCause%. And we want to check bulkstats from 00h00:00 to 06h30.

The tool starts by reading the configuration file (or SSD) to extract the bulkstat schema format. It then reads the bulkstat files. Finally, it does extract the unique counter names matching the key provided. There could be multiple unique names from a single key. The tool will then check for similar counters for every counter generated from the key.

The key generated counter name is first displayed with "=====". Then every matching counter is lister with its values and the diff between 2 consecutive values.

export NODE_OPTIONS="--max-old-space-size=8192"; \
node lib/index.js -c /Downloads/VFIndia/INVIGJ02GNR1SPCP02CO_SSD_11032022.txt\
-d /Downloads/Bharti/BS-MAR28/20220323/ k '%tun-sent-cresessrespdeniedOtherCause%'
-k '%tun-sent-cresessrespdeniedOtherCause%' -s 0 -e 63000

Parsing config file /Downloads/VFIndia/INVIGJ02GNR1SPCP02CO_SSD_11032022.txt
Reading bulk file /Downloads/Bharti/BS-MAR28/20220323/INVIGJ02GNR1SPCP02CO_bulkstats_20220323_000000_IST_15_15.csv
...
Value constant for %tun-sent-cresessrespdeniedOtherCause%@egtpcSch16/VPN-gwctx/SERVICE-egtp-pgw-s5s8 during interval
Value constant for %tun-sent-cresessrespdeniedOtherCause%@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s5s8 during interval
done
========= egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s11 ============
%tun-sent-cresessrespdeniedOtherCause%:           0000232988 0000233211 0000233413 0000233667 0000233826 0000234006 0000234152 0000234324 0000234492 0000234725 0000234883 0000235130 0000235341 0000235477 0000235620 0000235846 0000236092 0000236311 0000236505 0000236737 00
00236878 0000237066 0000237210 0000237411 0000237664 0000238041 0000238252
DIFF:                                            0000000000 0000000223 0000000202 0000000254 0000000159 0000000180 0000000146 0000000172 0000000168 0000000233 0000000158 0000000247 0000000211 0000000136 0000000143 0000000226 0000000246 0000000219 0000000194 0000000232 000
0000141 0000000188 0000000144 0000000201 0000000253 0000000377 0000000211
%total-recv-noRspReq%:                            0000234237 0000234460 0000234663 0000234917 0000235076 0000235257 0000235404 0000235576 0000235744 0000235977 0000236135 0000236382 0000236593 0000236725 0000236868 0000237094 0000237340 0000237559 0000237753 0000237985 00
00238126 0000238314 0000238458 0000238659 0000238913 0000239290 0000239501
DIFF:                                            0000000000 0000000223 0000000203 0000000254 0000000159 0000000181 0000000147 0000000172 0000000168 0000000233 0000000158 0000000247 0000000211 0000000132 0000000143 0000000226 0000000246 0000000219 0000000194 0000000232 000
0000141 0000000188 0000000144 0000000201 0000000254 0000000377 0000000211

Compute KPI

Using the same tool, we can compute a KPI formula (using basic operations like + - * / ). First try the tool with your formula using schema counter name.

The tool will tell you if those counters are not unique. A list of possible unique counter names will be given.

node lib/index.js -c /Downloads/VFIndia/INVIGJ02GNR1SPCP02CO_SSD_11032022.txt \
-d /Downloads/Bharti/BS-MAR28/20220323/ \
-k '%tun-sent-cresessrespdeniedOtherCause%' -s 0 -e 63000  \
-f '(%tun-sent-cresessrespaccept%+%tun-sent-cresessrespdeniedUserAuthFailed% + %tun-sent-cresessrespdeniedPrefPdnTypeUnsupported% + %tun-sent-cresessrespdeniedCtxtNotFound%) / %tun-recv-cresess%  * 100'

Parsing config file /Downloads/VFIndia/INVIGJ02GNR1SPCP02CO_SSD_11032022.txt
Reading bulk file /Downloads/Bharti/BS-MAR28/20220323/INVIGJ02GNR1SPCP02CO_bulkstats_20220323_000000_IST_15_15.csv
...
Reading bulk file /Downloads/Bharti/BS-MAR28/20220323/INVIGJ02GNR1SPCP02CO_bulkstats_20220323_063000_IST_15_15.csv
Comparing bulk stats
Please chose which extact stat
[
  '%tun-sent-cresessrespaccept@egtpcSch1/VPN-gwctx/SERVICE-egtp-pgw-s5s8%',
  '%tun-sent-cresessrespaccept@egtpcSch1/VPN-gwctx/SERVICE-egtp-sgw-s11%',
  '%tun-sent-cresessrespaccept@egtpcSch1/VPN-gwctx/SERVICE-egtp-sgw-s5s8%',
  '%tun-sent-cresessrespaccept@pgw-egtpc-s5s8Sch46/VPN-gwctx/SERVICE-egtp-pgw-s5s8%',
  '%tun-sent-cresessrespaccept@pgw-egtpc-s2bSch27/VPN-gwctx/SERVICE-egtp-pgw-s5s8%'
]
Please chose which extact stat
[
  '%tun-sent-cresessrespdeniedUserAuthFailed@egtpcSch16/VPN-gwctx/SERVICE-egtp-pgw-s5s8%',
  '%tun-sent-cresessrespdeniedUserAuthFailed@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s11%',
  '%tun-sent-cresessrespdeniedUserAuthFailed@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s5s8%'
]
Please chose which extact stat
[
  '%tun-sent-cresessrespdeniedPrefPdnTypeUnsupported@egtpcSch16/VPN-gwctx/SERVICE-egtp-pgw-s5s8%',
  '%tun-sent-cresessrespdeniedPrefPdnTypeUnsupported@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s11%',
  '%tun-sent-cresessrespdeniedPrefPdnTypeUnsupported@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s5s8%'
]
Please chose which extact stat
[
  '%tun-sent-cresessrespdeniedCtxtNotFound@egtpcSch16/VPN-gwctx/SERVICE-egtp-pgw-s5s8%',
  '%tun-sent-cresessrespdeniedCtxtNotFound@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s11%',
  '%tun-sent-cresessrespdeniedCtxtNotFound@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s5s8%'
]
Please chose which extact stat
[
  '%tun-recv-cresess@egtpcSch1/VPN-gwctx/SERVICE-egtp-pgw-s5s8%',
  '%tun-recv-cresess@egtpcSch1/VPN-gwctx/SERVICE-egtp-sgw-s11%',
  '%tun-recv-cresess@egtpcSch1/VPN-gwctx/SERVICE-egtp-sgw-s5s8%',
  '%tun-recv-cresess@pgw-egtpc-s5s8Sch46/VPN-gwctx/SERVICE-egtp-pgw-s5s8%',
  '%tun-recv-cresess@pgw-egtpc-s2bSch27/VPN-gwctx/SERVICE-egtp-pgw-s5s8%'
]

Modify your formula to use the unique counter names.

node lib/index.js -c /Downloads/VFIndia/INVIGJ02GNR1SPCP02CO_SSD_11032022.txt \
-d /Downloads/Bharti/BS-MAR28/20220323/ \
-k '%tun-sent-cresessrespdeniedOtherCause%' -s 0 -e 63000  \
-f '(%tun-sent-cresessrespaccept@egtpcSch1/VPN-gwctx/SERVICE-egtp-sgw-s11%+%tun-sent-cresessrespdeniedUserAuthFailed@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s11%+%tun-sent-cresessrespdeniedPrefPdnTypeUnsupported@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s11%+%tun-sent-cresessrespdeniedCtxtNotFound@egtpcSch16/VPN-gwctx/SERVICE-egtp-sgw-s11%) / %tun-recv-cresess@egtpcSch1/VPN-gwctx/SERVICE-egtp-sgw-s11%  * 100'

Parsing config file /Downloads/VFIndia/INVIGJ02GNR1SPCP02CO_SSD_11032022.txt
Reading bulk file /Downloads/Bharti/BS-MAR28/20220323/INVIGJ02GNR1SPCP02CO_bulkstats_20220323_000000_IST_15_15.csv
...
Comparing bulk stats
[
   96.6378832219478, 96.63720619626486,
  96.63684516930434, 96.63556579465056,
  96.63544591500445, 96.63706898508615,
  96.63735263816275, 96.63702105086574,
  96.63663303918919, 96.63583095560341,
  96.63602569267023, 96.63527471573649,
  96.63447542577062, 96.63462280741118,
  96.63472326368931, 96.63369030553042,
  96.63277059090882, 96.63189070373926,
  96.63196228068361, 96.63133355490132,
  96.63205234531985, 96.63183955891449,
  96.63238801277252, 96.63224975953285,
  96.63103717291196, 96.63081507546812,
  96.63027285792256
]