Create a representative performance benchmark
Closed this issue · 4 comments
Anecdotal evidence from uber internal is ok, but it would be nicer to have a benchmark project we could point. Maybe we could reuse RxJava's jmh tests, or the classic reactive streams shakespeare/scrabble benchmark?
Other considerations:
- jmh/android benchmark/both?
Notes from talking with @dlew
- RxJava/scrabble benchmarks might not be good options since they're focused on stream throughput, we care mostly about overhead of just subscribe points
- Should have both simple
just(1).subscribe()
example as well as one where the stream is kept open
Other notes from myself
- MemoryPerf from rxjava still probably relevant, can adapt to the above focus areas
Open question - still maybe worth doing the full rx tests just for more "real world" examples to know if the cost is effectively amortized
FWIW, I wrote some caliper benchmarks for a lib that solves the same problem to compare against another lib. Should be able to modify those tests to run against dogtags with a couple lines of code if you wanted.
I think the concerns I had when I was originally writing the tests were similar to yours. There are a few notes about the benchmarks in the readme if you're interested.
Thanks! We might give that a shot. Fortunately this is pretty targeted in the sense that it only happens at throughput (guarded delegate calls) and subscribe-time.
Closing this out since we have good benchmarks, documentation and a nice little script that outputs with the format we're looking for.