Pay-as-you-go (paug-workload-dashboard) dashboarding framework for workload specific resource usage metering & monitoring using open source technologies
- Compute resource consumption monitoring & metering for pay-as-you-go model.
- Resource consumption to be measured at workload level and not at operating systems level.
- Capability to capture excess resource consumption at user defined interval.
- Capability to show excess workload specific capacity usage for user specified/selected time period
Proposed frame work is primarily comprises of 3 stages,
- Data Generation: Generated at end points / hosts, using custom created collector script(s).
- Data Ingestion: Ingesting data generated at end point into centrally hosted time-series database (influxDB).
- Data Visualization: Visualizing ingested data points using centrally hosted Grafana dashboars.
- Capturing excess memory usage over and above user defined threshold for HANA workload
- Dashboard for monitoring, metering & billing excess memory usage at user defined interval
- SAP HANA Instance
- /bin/bash
- SUSE Linux Enterprise Server 15 (ppc64le)
- nmeasure_linux_ppc64le: Thanks to Nigel Griffiths for creating this helpful utility for pushing data into InfluxDB and it works like a charm !
- OS: RHELRed Hat Enterprise Linux 7.7 (Maipo)
- InfluxDB version: 1.7.7
- Grafana: v6.2.5
- architecture: systems architecture details (x86, ppc64le etc.)
- mtm: machine type model of physical server
- serial_no: serial no of physical server
- host: hostnam
- os: operating system of host
- oscache: operating system file cache
- osfree: unused memory (MemFree and SwapFree in /proc/meminfo)
- ostotal: total installed memory
- osused: used memory (calculated as total - free - buffers - cache)
- hanacommitted: comitted memory usage for hana workload
- hanaexcesss: excess ram usage by hana workload
- hanaramusage: total ram usage by hana workload
- Refer this useful post for installing InfluxDB on IBM Power server systems OR
- Refer InfluxDB portal for installing InfluxDB on x86 server systems
- Login to newly installed influxDB [command: "influx"]
- Create a new database named testdb [command: "create database testdb"]
- Complete HANA & OS user setup for compute monitoring at each LPAR hosting HANA DB
- Download collector script, collector.sh
- Download nmeasure_linux_ppc64le to USER1 home directory
- Assign executable permission to both [chmod u+x collector.sh nmeasure_linux_ppc64le]
- Move "nmeasure_linux_ppc64le" as "nmeasure" to bin in your PATH [command: mv measure_linux_ppc64le /home/USER1/bin/nmeasure]
- Make crontab entry with "crontab -e" command [crontab entry: 00 00 * * * /home/USER1/collector.sh 1 60 > /home/USER1/collector.log 2>&1]
Post completing step-1 & step-2, it would start populating data into InfluxDB and we are ready setting up data visualization
- Refer this useful post for installing Grafana on IBM Power server systems OR
- Refer Grafana portal for installing Grafana on x86 server systems
- Login to Grafana [http://<Grafana_Server_IP>:3000/]
- Import Grafana dashboard for paug-workload-dashboard-ramstat using steps listed here
- Optionally Customize "name"
- Select your InfluxDB name from dropdown next to "InfluxDB-1"
- Click on dashboard name to load
References
- SAP HANA Memory Usage Explained
- M_HOST_RESOURCE_UTILIZATION Table
- Grafana dashboard for paug-workload-dashboard-ramstat