Pythons -O flag makes an insane difference
void4 opened this issue ยท 6 comments
This is just something I noted
Running a script that spends most of its time in pywasm's exec:
$ time python main.py
real 2m0,034s
user 2m0,305s
sys 0m0,644s
And with optimizations enabled:
$ time python -O main.py
real 0m0,449s
user 0m0,908s
sys 0m0,481s
According to this -O eliminates assert statements. Not sure how much this impacts security.
Edit: ok, it seems that the code is not executed correctly, I'll have a look
I get this:
File "main.py", line 193, in play
runtime = pywasm.load(runtime, {
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/__init__.py", line 98, in load
module = binary.Module.from_reader(f)
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/binary.py", line 1084, in from_reader
type_section = TypeSection.from_reader(section_reader)
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/binary.py", line 445, in from_reader
o.data = [FunctionType.from_reader(r) for _ in range(n)]
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/binary.py", line 445, in <listcomp>
o.data = [FunctionType.from_reader(r) for _ in range(n)]
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/binary.py", line 70, in from_reader
o.args = ResultType.from_reader(r)
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/binary.py", line 51, in from_reader
o.data = [ValueType.from_reader(r) for i in range(n)]
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/binary.py", line 51, in <listcomp>
o.data = [ValueType.from_reader(r) for i in range(n)]
File "/home/one/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pywasm/binary.py", line 33, in from_reader
return ValueType(ord(r.read(1)))
Exception ord() expected a character, but string of length 0 found
Ah, looking through the pywasm code it seems that the asserts have side effects, that are necessary for it to be executed correctly.
Thank you for your discovery. I have noticed the execution efficiency of pywasm. It spends a lot of time on https://github.com/mohanson/pywasm/blob/master/pywasm/execution.py#L18, because every time it access any value, it always need to regenerate the python object (int/float) from the byte(s).
The current performance of pywasm is not particularly good, I will try to optimize this part of the code.
That would be great! I'm using pywasm now to run the chess tournaments between programs (see here: https://rlc-chess.com/competitions :) (site takes a bit too load, but this is due to the frontend code, not pywasm in this case))
I recently started learning chess (because of The Queen's Gambit
Maybe I will participate :)
I definitely want to create more experiments like this, say, let people write their own simple routing algorithms and see which one is the most resource-effective one: https://www.youtube.com/watch?v=Y8MvSAZ-YMk (https://github.com/void4/netsim)
For this I used Stackless Python because it supports suspending programs (and resuming them!) after some number of instruction steps: https://stackless.readthedocs.io/en/latest/library/stackless/pickling.html But Python cannot safely run untrusted programs, which is why WebAssembly would be a great target again.
With pywasm running faster, these kinds of projects will be much more feasible.
Ah, looking through the pywasm code it seems that the asserts have side effects, that are necessary for it to be executed correctly.
It is not a good habit to produce side effects in assertions, so I will remove it, thanks.
I fixed the side effects of assertion in pywasm==1.0.7
, and at the same time improved the performance (about ~10%). I think I have a chance to double the running speed, but I need to think more (that is, not in this version).
-O
won't break the program anymore.