The returned `Evaluate_Time` does not include results from all metrics
npatki opened this issue · 0 comments
npatki commented
Environment Details
- SDGym version: 0.8.0 (latest)
Error Description
According to the SDGym docs, the final results include a column called Evaluate_Time
. This column is supposed to include the total time it took to evaluate the synthetic data, based on the metrics provided.
But in practice, we see that the Evaluate_Time
computation is hardcoded such that it only ever includes the time to run the quality report, and the NewRowSynthesis metric (if provided):
Lines 329 to 335 in 50eac8e
Expected Behavior
When running a benchmark, it is possible to input additional SDMetrics to be used for evaluation. If I add new metrics, I expect that the time it takes to compute them should be included in the final Evaluate_Time
that is reported.