Op Mix per Hour vs Actual Op Mixes per Hour
Closed this issue · 5 comments
Hi @rvesse I was trying to understand what's the difference in a multithread bench, between Operation Mixes per Hour and Actual Operation Mixes per Hour. Any pointer or help is more than welcome. Best.
This is discussed briefly in the wiki content:
https://sourceforge.net/p/sparql-query-bm/wiki/BenchmarkingMethodology/
More specifically the basic statistics are based on total time whereas the actual statistics are based on observed time. This is probably easiest to explain with an example.
Suppose we had an operation mix that took precisely 10 minutes to run. In a single threaded benchmark we would thus calculate that we could perform 6 operation mixes per hour. Now suppose we ran with 2 threads, the total time would be 20 minutes and we would calculate 3 operation mixes per hour. But as we ran in parallel the observed time would only be 10 minutes so we would calculate actual operation mixes per hour as 6.
Thanks for the quick answer. What you are saying is that the relevant value for threaded bench is to take into account the Op Mix per hour. Btw, would it be possible to have access to this link which seems 404 http://www.revelytix.com/sites/default/files/TripleStorePerformanceTestingMethodolgy.pdf ? Maybe it gives more details regarding the different statistics used in the bench.
No, the actual stats are more relevant because they indicate the performance that the end users will experience.
The referenced paper was published by a now-defunct private company and I don't have a copy myself. You may be able to find this in the Internet archive.
Great! Thanks.