This is a python package which contains tools for processing Recorder traces.
pip install recorder-viz
We relie on a few libraries for visualization. Please install them first.
Dependencies: pandas, bokeh and prettytable.
import recorder_viz
from recorder_viz import RecorderReader
reader = RecorderReader("path/to/Recorder-traces-folder")
recorder_viz.generate_report(reader, "output.html")The generate_report API will write out a HTML format visualization report.
The visualization process can be slow for large traces. We recommend to use it only for small scale runs, e.g., < 128 ranks runs.
Below are some example graphs generated from the FLASH traces.
The RecorderReader class contains all infomration about the Recorder traces.
class RecorderReader:
self.GM: instance of GlobalMetadata
self.LMs: list of LocalMetadata objects, one for each rank
self.records: self.records[i] is a list of Record objects of rank i.GlobalMetadta, LocalMetadata and Record are three Python wrappers of C structures.
class LocalMetadata(Structure):
self.total_records: int
self.num_files: int
self.filemap: set()
self.function_count: []
class RecorderMetadata(Structure):
_fields_ = [
("total_ranks", c_int),
("start_ts", c_double),
("time_resolution", c_double),
("ts_buffer_elements", c_int),
("ts_compression_algo", c_int),
]
class Record(Structure):
_fields_ = [
("tstart", c_double),
("tend", c_double),
("level", c_ubyte),
("func_id", c_ubyte),
("tid", c_int),
("arg_count", c_ubyte),
("args", POINTER(c_char_p)),
]Here's an example on how to use the provided classes.
from recorder_viz import RecorderReader
reader = RecorderReader("path/to/Recorder-traces-folder")
for rank in range(reader.GM.total_ranks):
LM = reader.LMs[rank]
print("Rank: %d, Number of trace records: %d" %(rank, LM.total_records))