DUNE-DAQ/dfmodules

Add component list printout to hdf5dump.py

Closed this issue · 3 comments

When using our hdf5dump.py utility to look at the data that we have written to disk, I would find it useful to have the option of printing out the component list that is stored in the TriggerRecordHeader. (This information would tell us which APAs and Links were requested to be in the TriggerRecord.)

It would also be nice for the "TRH" mode of hdf5dump.py to print out the number of expected Fragments, the actual number of Fragments in the TR, and possibly the difference (which would be the number of missing Fragments).

dingp commented

In the latest commit, I've switched to h5py.visititems() for traversing the HDF5 file. It works a bit faster than previous custom function. Also, as a proof of concept, the script is printing out the expected component list in TRH.

To compare the list of expected fragments and the actual fragments in the TR, the script will need to process the file by the unit of TriggerRecordXXX, of which the subgroups are TriggerRecordXXX/FELIX/APAXXX/LINKXX and TriggerRecordXXX/TriggerRecordHeader.

In the future, I'm planning to define several classes inherited from dict (in separate issues):

  • TriggerRecord,
  • TriggerRecordHeader
  • Fragment

One TriggerRecord object will contain one TriggerRecordHeader object and a list of Fragment objects.

The callable function for h5py.visititems will process and append TriggerRecordXXX to the global trigger_record list.

dingp commented
This latest commit completes the implementation of issue #55.

However, when working on `hdf5_dump.py` script, several possioble
improvments came to my mind, which might help speeding up adding new
fea`tures in the future or providing better user experience:

1. when traversing the file, contruct a a list of TriggerRecord, of which each element contains:
 - a list of fragments, each contains raw binary data;
 - a trigger record header in raw binary data;
2. define classes of:
 - DataFile (?)
 - TriggerRecrod
 - TriggerRecordHeader
 - Component
 - Fragment
3. Classes of TriggerRecordHeader, Component and Fragment will each have an `unpack` function to properly unpack the data when explicitly called.

Several findings when implementing #55:
1. using `h5py.visititems()` is very fast to traverse through the file;
2. unpacking data is relatively expensive, only unpack when necessary
(thus the proposed classes will only unpack when explicitly called);
3. `struck.unpack()` and `struct.iter_unpack()` is useful; the details
of format string can be found here:
https://docs.python.org/3/library/struct.html