Propose to add "number of epochs" column to the "Training Time" ranking result
jgong5 opened this issue · 2 comments
Since the "number of epochs" is the primary factor determining the actual computation required for training, is it possible to list it in the "Training Time" ranking result as well? Thanks.
@jgong5 all the information is available in the TSVs to calculate "the number of epochs", but I don't see how this makes sense given our focus on end-to-end time. Similar to throughput (e.g., images per second), number of epochs seems like a proxy for training time that can be misleading. In some cases, epochs might not even have the same meaning. If there were submissions that did some form of importance sampling, some examples would be seen many times before others are seen at all. As a result, training might only take a few passes through the data, but each pass could be much longer.
@codyaustun Your comments make sense. That would bring more confusion. Thanks for the explanation.