Labels don't seem to be handled correctly with Python<3.9
Closed this issue · 5 comments
P403n1x87 commented
I am currently working with the following abstract code
stopiter = Label()
loop = Label()
genexit = Label()
exc = Label()
propagate = Label()
instrs[-1:-1] = [
Instr("DUP_TOP", lineno=lineno),
Instr("STORE_FAST", "__ddgen", lineno=lineno),
Instr("LOAD_ATTR", "asend", lineno=lineno),
Instr("STORE_FAST", "__ddgensend", lineno=lineno),
Instr("LOAD_FAST", "__ddgen", lineno=lineno),
Instr("LOAD_ATTR", "__anext__", lineno=lineno),
Instr("CALL_FUNCTION", 0, lineno=lineno),
loop,
Instr("SETUP_EXCEPT" if PY < (3, 8) else "SETUP_FINALLY", stopiter, lineno=lineno),
Instr("GET_AWAITABLE", lineno=lineno),
Instr("LOAD_CONST", None, lineno=lineno),
Instr("YIELD_FROM", lineno=lineno),
Instr("POP_BLOCK", lineno=lineno),
Instr("SETUP_EXCEPT" if PY < (3, 8) else "SETUP_FINALLY", genexit, lineno=lineno),
Instr("YIELD_VALUE", lineno=lineno),
Instr("POP_BLOCK", lineno=lineno),
Instr("LOAD_FAST", "__ddgensend", lineno=lineno),
Instr("ROT_TWO", lineno=lineno),
Instr("CALL_FUNCTION", 1, lineno=lineno),
Instr("JUMP_ABSOLUTE", loop, lineno=lineno),
stopiter, # except StopAsyncIteration:
Instr("DUP_TOP", lineno=lineno),
Instr("LOAD_CONST", StopAsyncIteration, lineno=lineno),
compare_exc(propagate, lineno),
jump_if_false(propagate, lineno),
Instr("POP_TOP", lineno=lineno),
Instr("POP_TOP", lineno=lineno),
Instr("POP_TOP", lineno=lineno),
Instr("POP_EXCEPT", lineno=lineno),
Instr("LOAD_CONST", None, lineno=lineno),
Instr("RETURN_VALUE", lineno=lineno),
propagate, # finally:
Instr("END_FINALLY" if PY < (3, 9) else "RERAISE", lineno=lineno),
genexit, # except GeneratorExit:
Instr("DUP_TOP", lineno=lineno),
Instr("LOAD_CONST", GeneratorExit, lineno=lineno),
compare_exc(exc, lineno),
jump_if_false(exc, lineno),
Instr("POP_TOP", lineno=lineno),
Instr("POP_TOP", lineno=lineno),
Instr("POP_TOP", lineno=lineno),
Instr("LOAD_FAST", "__ddgen", lineno=lineno),
Instr("LOAD_ATTR", "aclose", lineno=lineno),
Instr("CALL_FUNCTION", 0, lineno=lineno),
Instr("GET_AWAITABLE", lineno=lineno),
Instr("LOAD_CONST", None, lineno=lineno),
Instr("YIELD_FROM", lineno=lineno),
Instr("POP_EXCEPT", lineno=lineno),
Instr("RETURN_VALUE", lineno=lineno),
exc, # except:
Instr("POP_TOP", lineno=lineno),
Instr("POP_TOP", lineno=lineno),
Instr("POP_TOP", lineno=lineno),
Instr("LOAD_FAST", "__ddgen", lineno=lineno),
Instr("LOAD_ATTR", "athrow", lineno=lineno),
Instr("LOAD_CONST", sys.exc_info, lineno=lineno),
Instr("CALL_FUNCTION", 0, lineno=lineno),
Instr("CALL_FUNCTION_EX", 0, lineno=lineno),
Instr("GET_AWAITABLE", lineno=lineno),
Instr("LOAD_CONST", None, lineno=lineno),
Instr("YIELD_FROM", lineno=lineno),
Instr("POP_EXCEPT", lineno=lineno),
Instr("RETURN_VALUE", lineno=lineno),
]When compiled to concrete Python 3.9 bytecode I get what I would expect (note the correct jump to RERAISE)
12 DUP_TOP
14 STORE_FAST 1 (__ddgen)
16 LOAD_ATTR 0 (asend)
18 STORE_FAST 2 (__ddgensend)
20 LOAD_FAST 1 (__ddgen)
22 LOAD_ATTR 1 (__anext__)
24 CALL_FUNCTION 0
>> 26 SETUP_FINALLY 22 (to 50)
28 GET_AWAITABLE
30 LOAD_CONST 2 (None)
32 YIELD_FROM
34 POP_BLOCK
36 SETUP_FINALLY 34 (to 72)
38 YIELD_VALUE
40 POP_BLOCK
42 LOAD_FAST 2 (__ddgensend)
44 ROT_TWO
46 CALL_FUNCTION 1
48 JUMP_ABSOLUTE 26
>> 50 DUP_TOP
52 LOAD_CONST 3 (<class 'StopAsyncIteration'>)
54 JUMP_IF_NOT_EXC_MATCH 70
56 NOP
58 POP_TOP
60 POP_TOP
62 POP_TOP
64 POP_EXCEPT
66 LOAD_CONST 2 (None)
68 RETURN_VALUE
>> 70 RERAISE
>> 72 DUP_TOP
74 LOAD_CONST 4 (<class 'GeneratorExit'>)
76 JUMP_IF_NOT_EXC_MATCH 102
78 NOP
80 POP_TOP
82 POP_TOP
84 POP_TOP
86 LOAD_FAST 1 (__ddgen)
88 LOAD_ATTR 2 (aclose)
90 CALL_FUNCTION 0
92 GET_AWAITABLE
94 LOAD_CONST 2 (None)
96 YIELD_FROM
98 POP_EXCEPT
100 RETURN_VALUE
>> 102 POP_TOP
104 POP_TOP
106 POP_TOP
108 LOAD_FAST 1 (__ddgen)
110 LOAD_ATTR 3 (athrow)
112 LOAD_CONST 5 (<built-in function exc_info>)
114 CALL_FUNCTION 0
116 CALL_FUNCTION_EX 0
118 GET_AWAITABLE
120 LOAD_CONST 2 (None)
122 YIELD_FROM
124 POP_EXCEPT
126 RETURN_VALUE
128 RETURN_VALUE
However, with earlier Python versions the jump to the propagate label is not resolved correctly and actually ends up targeting the exc label (END_FINALLY is now in place of the newer RERASE, but the jump is not there!):
12 DUP_TOP
14 STORE_FAST 1 (__ddgen)
16 LOAD_ATTR 0 (asend)
18 STORE_FAST 2 (__ddgensend)
20 LOAD_FAST 1 (__ddgen)
22 LOAD_ATTR 1 (__anext__)
24 CALL_FUNCTION 0
>> 26 SETUP_FINALLY 22 (to 50)
28 GET_AWAITABLE
30 LOAD_CONST 2 (None)
32 YIELD_FROM
34 POP_BLOCK
36 SETUP_FINALLY 34 (to 72)
38 YIELD_VALUE
40 POP_BLOCK
42 LOAD_FAST 2 (__ddgensend)
44 ROT_TWO
46 CALL_FUNCTION 1
48 JUMP_ABSOLUTE 26
>> 50 DUP_TOP
52 LOAD_CONST 3 (<class 'StopAsyncIteration'>)
54 COMPARE_OP 10 (exception match)
56 POP_JUMP_IF_FALSE 102
58 POP_TOP
60 POP_TOP
62 POP_TOP
64 POP_EXCEPT
66 LOAD_CONST 2 (None)
68 RETURN_VALUE
70 END_FINALLY
>> 72 DUP_TOP
74 LOAD_CONST 4 (<class 'GeneratorExit'>)
76 COMPARE_OP 10 (exception match)
78 POP_JUMP_IF_FALSE 102
80 POP_TOP
82 POP_TOP
84 POP_TOP
86 LOAD_FAST 1 (__ddgen)
88 LOAD_ATTR 2 (aclose)
90 CALL_FUNCTION 0
92 GET_AWAITABLE
94 LOAD_CONST 2 (None)
96 YIELD_FROM
98 POP_EXCEPT
100 RETURN_VALUE
>> 102 POP_TOP
104 POP_TOP
106 POP_TOP
108 LOAD_FAST 1 (__ddgen)
110 LOAD_ATTR 3 (athrow)
112 LOAD_CONST 5 (<built-in function exc_info>)
114 CALL_FUNCTION 0
116 CALL_FUNCTION_EX 0
118 GET_AWAITABLE
120 LOAD_CONST 2 (None)
122 YIELD_FROM
124 POP_EXCEPT
126 RETURN_VALUE
128 RETURN_VALUE
This seems to point to a wrong resolution of the branching label.
MatthieuDartiailh commented
Just to help me follow could you post some equivalent code/pseudo-code ?
P403n1x87 commented
This is an async_generator wrapper. This should be close to the following Python code
async def w(g):
c = g.__anext__
while True:
_ = c()
try:
_ = await _
except StopAsyncIteration:
return
try:
s = yield _
except GeneratorExit:
await g.aclose()
except:
await g.athrow(*sys.exc_info())
c = lambda: g.asend(s)MatthieuDartiailh commented
Can you also explicit your two functions compare_exc and pop_jump_if_false, so that I know what the bytecode looks for both versions ?
P403n1x87 commented
Big facepalm moment! Your last question helped me find the actual issue, which was due to copy-pasta in my code! 😢 These are the functions
def compare_exc(label, lineno):
return (
Instr("COMPARE_OP", Compare.EXC_MATCH, lineno=lineno)
if PY < (3, 9)
else Instr("JUMP_IF_NOT_EXC_MATCH", label, lineno=lineno)
)
def jump_if_false(label, lineno):
return Instr("POP_JUMP_IF_FALSE", label, lineno=lineno) if PY < (3, 9) else Instr("NOP", lineno=lineno)
which are defined locally inside another function. I had exc in place of label in jump_if_false. So indeed the generated bytecode was correct as it was getting exc from the outer scope! 😭 . Now that gives me RuntimeError: Failed to compute stacksize, got negative size but I should be able to fix this.
Apologies for this non-issue!
MatthieuDartiailh commented
Always happy to help !