google/vulkan-performance-layers

Introduce event logging abstraction.

Closed this issue · 4 comments

An EventLogger takes a stream of Event as input and produces a log based on the desired format (CSV, Chrome trace event, etc).

flowchart LR
    EventCreator --Event --> EventLogger --log--> OutputLogger
Loading

The EventLogger would be an abstract class with some public methods: AddEvent(), StartLog(), and FinalizeLog(). It also requires a flush mechanism. Either a public Flush() method triggered from outside, or an implicitly triggered Flush() invoked when the number of current events reaches a certain threshold.
Each format may require prefixes and suffixes around the final log. For instance, CSV requires adding a header to the first line of the file. StartLog() and FinalizeLog() should be implemented according to the specifications of each format.

To generate a log, these steps are required:

  1. Converting each Event into an entry:
    Each Event is an entry in the final log output. The entry format is defined by the EventLogger implementation. For instance: The entry for Event(name: "compile_time_ns", timestamp: 1234, hash:0xabc, duration:123) would look like this in the CSV format:
compile_time,1234,0xabc,123

and this in the Chrome event trace format:

{name:"compile_time_ns", timestamp: 1234, hash: 0xabc, duration: 123}
  1. Combining entries:
    This step, generates the log's body by combining the entries. In CSV, the entries are separated by a new line and in Chrome event trace, they're comma separated.
    The pseudo code for the EventLogger would be like this:
class EventLogger {
 public:
  void InsertEvent(Event* e) = 0;
  // Flush may receive an input (an instance of LogOutput), pass the string generated from concatenating the events to it, and 
  // delete the current events.
  void Flush(...) = 0;
  void StartLog() = 0;
  void FinishLog() = 0;
 protected:
  std::vector<Event*> events_;
};

class EventLoggerImpl: public EventLogger {
 public:
  void Flush(...) override {
    // Convert Events to entries.
    ...
    // Combine entries.
    ...
    // Delete current events
    ...
    // Use LogOutput to print the log
  }
}
kuhar commented

Thanks for writing this up!

A few comments and questions:

An EventLogger takes Event(s) as input and produces a log based on the desired format (CSV, Chrome trace event, etc).

We could say that the input is a stream or sequence of Events.

The EventLogger would be an abstract class with 2 public methods: AddEvent() and GenerateLog(). Each format implements the bodies based on its specifications.

Why do we need GenerateLog()? I'd think that it can generate logs when it decides to do so (e.g., with each new event, each n events, etc.) or when a flush method is called.

and this in the Chrome event trace format:
...

Is this the final format that you decided on or just an example?

Adding header/footers to generate the final log:

Ah, I think I know why you wanted to have GenerateLog(): to generate the prefix and suffix. Doesn't this force us to print the whole log at once instead of streaming the results to the output? What do you think about exposing StartLog() and FinalizeLog() instead?

Yeah, the main reason for keeping all the events and using the GenerateLog() method was prefix and suffix. Using StartLog() and FinalizeLog() is a good idea because it moves out the job of dealing with the prefixes and suffixes to another function and the EventLogger can focus on turning the events into entries.

kuhar commented

@miladHakimi can we close this?

@miladHakimi can we close this?

Sure!