/paug-workload-dashboard

Primary LanguageShellGNU General Public License v3.0GPL-3.0

Pay-as-you-go (paug-workload-dashboard) dashboarding framework for workload specific resource usage metering & monitoring using open source technologies

Requirement

  1. Compute resource consumption monitoring & metering for pay-as-you-go model.
  2. Resource consumption to be measured at workload level and not at operating systems level.
  3. Capability to capture excess resource consumption at user defined interval.
  4. Capability to show excess workload specific capacity usage for user specified/selected time period

Technical Framework

Proposed frame work is primarily comprises of 3 stages,

  1. Data Generation: Generated at end points / hosts, using custom created collector script(s).
  2. Data Ingestion: Ingesting data generated at end point into centrally hosted time-series database (influxDB).
  3. Data Visualization: Visualizing ingested data points using centrally hosted Grafana dashboars. Alt text

Demonstrated use case

  • Capturing excess memory usage over and above user defined threshold for HANA workload
  • Dashboard for monitoring, metering & billing excess memory usage at user defined interval

Technology Stack

At each LPAR hosting HANA DB

  • SAP HANA Instance
  • /bin/bash
  • SUSE Linux Enterprise Server 15 (ppc64le)
  • nmeasure_linux_ppc64le: Thanks to Nigel Griffiths for creating this helpful utility for pushing data into InfluxDB and it works like a charm !

At central dashboard hosting server

  • OS: RHELRed Hat Enterprise Linux 7.7 (Maipo)
  • InfluxDB version: 1.7.7
  • Grafana: v6.2.5

Terminologies

  • architecture: systems architecture details (x86, ppc64le etc.)
  • mtm: machine type model of physical server
  • serial_no: serial no of physical server
  • host: hostnam
  • os: operating system of host
  • oscache: operating system file cache
  • osfree: unused memory (MemFree and SwapFree in /proc/meminfo)
  • ostotal: total installed memory
  • osused: used memory (calculated as total - free - buffers - cache)
  • hanacommitted: comitted memory usage for hana workload
  • hanaexcesss: excess ram usage by hana workload
  • hanaramusage: total ram usage by hana workload

Step-1: Setting up centrally hosted influxDB

  • Refer this useful post for installing InfluxDB on IBM Power server systems OR
  • Refer InfluxDB portal for installing InfluxDB on x86 server systems
  • Login to newly installed influxDB [command: "influx"]
  • Create a new database named testdb [command: "create database testdb"]

Step-2: Setting up end point "data collection" & "ingestion" to influxDB

  • Complete HANA & OS user setup for compute monitoring at each LPAR hosting HANA DB
  • Download collector script, collector.sh
  • Download nmeasure_linux_ppc64le to USER1 home directory
  • Assign executable permission to both [chmod u+x collector.sh nmeasure_linux_ppc64le]
  • Move "nmeasure_linux_ppc64le" as "nmeasure" to bin in your PATH [command: mv measure_linux_ppc64le /home/USER1/bin/nmeasure]
  • Make crontab entry with "crontab -e" command [crontab entry: 00 00 * * * /home/USER1/collector.sh 1 60 > /home/USER1/collector.log 2>&1]

Post completing step-1 & step-2, it would start populating data into InfluxDB and we are ready setting up data visualization

Step-3: Setting up "data visualization"

  • Refer this useful post for installing Grafana on IBM Power server systems OR
  • Refer Grafana portal for installing Grafana on x86 server systems
  • Login to Grafana [http://<Grafana_Server_IP>:3000/]
  • Import Grafana dashboard for paug-workload-dashboard-ramstat using steps listed here
  • Optionally Customize "name"
  • Select your InfluxDB name from dropdown next to "InfluxDB-1"
  • Click on dashboard name to load Alt text

References