Lack of Metadata caches for CakePHP and Doctrine
markstory opened this issue · 17 comments
The CakePHP and Doctrine ORMs both recommend that metadata caching be enabled to get better performance in production environments. I noticed that these benchmarks do not enable metadata caching. Would you be open to a pull request adding metadata caching for these ORMs?
Almost all ORMs recommends metadata caching. Including Phalcon, Eloquent, etc. As you can see I don't cache metadata for all ORMs, not only for Doctrine and CakePHP. It is important to measure w/o improvements. Because all of ORMs has its own tricks and tips. However, we can make a separate set of tests using metadata caching for all ORMs.
Feel free to send PR :)
You can put in a pull request if you would like.
However, i don't think caching would be accepted as @sergeyklay has mentioned before
Note: I don't do improvements for any ORMs. It is important to measure w/o improvements. Because all of ORMs has its own tricks and tips
If we enabled caching on some ORM's we would have to enable caching for the rest.
and phalcon supports caching as well which we having enabled/used for that exact reason.
@sergeyklay That reasoning is flawed. The default usage of those ORMs is with this meta-data cache enabled.
Otherwise it is the same as putting sleep(1) all over the place to artificially slow down the "competition".
You need to understand and accept how a tool is supposed to be run, otherwise it is more like comparing apples to cars. Which is not helpful for anyone.
I only mentioned the lack of metadata caching for doctrine and CakePHP as those are the ORMs I know how to use. I wasn't aware of metadata caching used in other ORMs. With that said, I don't think I'm the right person to setup caching for anything other CakePHP and Doctrine.
@JABirchall The phalcon documentation you linked to appears to be for result set/query caching, which is not the type of caching I'm referring to with CakePHP and doctrine. Did you intend to link to Phalcon's metadata caching?
@dereuromark Enabling metadata caching is more like comparing caching techniques. Isn't that? Current tests show how much work you need to do and what you get out of the box.
Well, which adapter do we need to use? File caching or for example memcached? I think some ORMs may not support memcached. In this case, the adapter must be the same. Is it true? But anyway, I'm ok with introducing benchmarking using metadata caching. I don't say no. If someone has a few hours of free time - PR are always welcome :)
@markstory Let's create a test suite for ORMs which support metadata caching. I can do this for Phalcon, @JABirchall for Eloquent I think and you for CakePHP and Doctrine.
@sergeyklay You do know what metadata cache means, right?
Just to clarify: It means once you have the schema of the tables and columns, you do not need to re-fetch that schema (since it doesnt change as long as you do not change the DB schema).
That info MUST always be cached by default, otherwise you are not even using the ORM at all as it was built.
You are completely destroying the tool itself, since this is a setup process and not part of the ORM fetching data. This is only done once and stored persistently for the actual use of the ORM. So please fix the current benchmark in itself, otherwise this whole comparison is completely wrong and your results worthless.
I will make it a bit more clear for you with an analogy:
So you have the E-engine car, and we are running the Fuel based car.
Sure, yours runs on energy in batteries and doesnt need any fuel, cool.
Now, if we race those cars, and you force the Fuel based one to only get a drop of fuel each time, and force it to refuel every 10meters, of course, you will reach the finish line faster, no matter what kind of power you got, since you cheat the system and artificially slow down the other one with a major impact (the fueling is part of the initial setup just as loading your batteries).
If you would allow the other car to have the Fuel stored as intended, the race is a bit more fair in total, wouldn't you agree?
We do not force your tool to stay in pure PHP either, do we? Just saying, you are converting to a different low-level language (C) and then compare it to the original language. That itself is already questionworthy. But at least you should give the other tools the proper start equipment and not make it worse than it already is.. My 5 cents.
Yes, but note: for first (single) test call the metadata caching doesn't improve performance at all. Second test, that uses 10-fold method call always runs on the same call stack.
Just set the flag to enabled (true) and the problem is fixed :)
@dereuromark I'm not a big expert with Doctrine. Could you please explain a bit more? Which flag do you mean? 🤔
I can only refer to CakePHP: #5 fixes things.
@sergeyklay I don't think eloquent has metadata caching. If it does I have never ran into it. But I can set it up for phalcon.
@markstory Hmmm... It seems there is cheating from Doctrine (at least). Let me explain. In these tests Doctrine uses annotations. Annotations are read from docblocks in classes, methods and properties. So Phalcon, for example uses SQL (extra) query to the Database as default introspection strategy. In other hand Phalcon support annotations introspection strategy too :)
I agree with mark here, without metadata caching you are doing a benchmark for performance of all the orms in Dev Mode. Which is like comparing car diesel output in a testing facility. Not the real thing ;-)
Well, I've introduced support of tests with metadata caching. Could someone send a PR for Doctrine?
Added results for Doctrine with metadata caching: 0733c9b