The data for the application statistics is being regularly collected in several locations.
These scripts collect the information, clean the data and produce csv files with the statistics.
To generate the statistics use the following procedure:
cp config.example config
# edit config with correct dates and settings
make
make reporting # copy data from csv files into excel
This will do the following:
1. Run pbs-report over date range specified in config as root on wlm01 (uses ssh if username or hostname is not correct)
2. Clean output to correct for any "|" characters in job names and to remove header and footer lines
3. Count the number of cores used by each job (gpu nodes always counted as 24 cores) and put that in a separate csv file
4. Collect application information in a separate csv file
5. Collect information about all users who submitted jobs in a separate csv file
6. Collect project information in a separate csv file
7. Collect GPUs requested in a separate csv file
8. Collect last dump of storage utilisation
9. Collect last dump of AM project status
10. Run a script to parse archive pbsnodes data to calculate online %, utilisation rate and unused reserved resources
11. Run R scripts to generate statistics
12. Run R scripts to generate Excel file and png graphs
To add a stakeholder:
1. Modify config.example and PBS-Application-Stats-Common.R to update list of stakeholders
2. Modfy script to generate usernames.${suffix}.csv so that users are assigned to stakeholder
3. Perform any necessary changes for project classification, make-totals (for utilisation.csv)