performance tracker
Opened this issue · 8 comments
An issue to track performance otherwise lost to us in #448. Benchmarks are done with the admittedly highly simplistic https://github.com/faassen/welcome-bench
Morepath 0.15 benchmarks like this on Python 2.7 on my desktop:
ms rps tcalls funcs
morepath 0.15 7683 13016 135 97
Morepath master has a lot of changes in it, in particular Reg dispatch methods, including all kinds of Reg refactorings for performance done by @taschini and myself -- code generation, a dict-based cache and so on. Morepath also now special cases a URL that has no expected URL parameters which also gives a boost to performance.
So Morepath master now clocks at:
Benchmarking frameworks: morepath
ms rps tcalls funcs
morepath 5302 18861 110 80
Here we compare master with other web frameworks in welcome-bench:
ms rps tcalls funcs
bottle 2168 46129 53 31
django 11135 8981 190 85
falcon 1559 64131 26 24
flask 15710 6365 270 125
morepath 5337 18736 110 80
pyramid 3948 25327 72 57
tornado 16950 5900 284 114
wheezy.web 1254 79740 25 23
wsgi 171 584956 8 8
Ordered by: internal time
List reduced from 92 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
100000 0.465 0.000 0.679 0.000 response.py:87(__init__)
100000 0.416 0.000 1.836 0.000 request.py:23(__init__)
400000 0.386 0.000 1.286 0.000 argextract.py:15(__call__)
100000 0.316 0.000 0.377 0.000 mapply.py:4(mapply)
100000 0.297 0.000 1.583 0.000 predicate.py:139(get_key)
100000 0.291 0.000 1.159 0.000 publish.py:75(consume)
100000 0.250 0.000 1.320 0.000 view.py:45(__call__)
100000 0.223 0.000 0.448 0.000 traject.py:325(consume)
100000 0.221 0.000 0.267 0.000 response.py:1043(_abs_headerlist)
100000 0.203 0.000 8.006 0.000 benchmark.py:81(<lambda>)
The profile for current master.
I looked into improving response generation. We can get about 300 more requests per second by constructing the Response directly in root_default in the benchmark instead of returning a string. This is what Pyramid does in its benchmark, but it's less idiomatic Morepath so I didn't implement it.
Latested master of Morepath and Reg:
ms rps tcalls funcs
morepath 5202 19222 109 80
After the latest changes we get quite a boost:
ms rps tcalls funcs
morepath 4889 20455 105 79
I've refactored the performance tracking system so I can do multiple benchmarks. The previous 'welcome' benchmark is now hello in this tool:
https://github.com/faassen/howareyou
On the same benchmark master gets this:
Benchmark : hello
Frameworks: pyramid, morepath
ms rps tcalls funcs
pyramid 4051 24687 74 58
morepath 3563 28067 81 63
I've included Pyramid as we now are faster and it feels a bit satisfying to compare now. :)
I've also included a new benchmark that tests a path with a single variable in a bunch of frameworks. Here are Morepath and Pyramid:
Benchmark : variable
Frameworks: pyramid, morepath
ms rps tcalls funcs
pyramid 4796 20850 78 60
morepath 4562 21919 98 71
With latest Morepath and Reg:
Benchmark : hello
Frameworks: morepath
ms rps tcalls funcs
morepath 3406 29358 79 62
and the variable benchmark:
Benchmark : variable
Frameworks: morepath
ms rps tcalls funcs
morepath 4516 22142 96 70
With Webob 1.7.0rc1:
Benchmark : hello
Frameworks: morepath
ms rps tcalls funcs
morepath 3392 29482 76 61
For some reason there's a slight performance regression with the variable benchmark:
Benchmark : variable
Frameworks: morepath
ms rps tcalls funcs
morepath 5087 19658 105 77
I'm a bit confused why -- I don't know why it would be touching more of WebOb than the non-variable benchmark does.