Assert is triggered in luaL_addgsub_test: Assertion `((B)->b != (B)->init.b) ? lua_touserdata(B->L, boxidx) != ((void)0) : lua_touserdata(B->L, boxidx) == (void)B' failed.

Question

Assert is triggered in luaL_addgsub_test: Assertion `((B)->b != (B)->init.b) ? lua_touserdata(B->L, boxidx) != ((void)0) : lua_touserdata(B->L, boxidx) == (void)B' failed.

Closed this issue a year ago · 2 comments

INFO: seed corpus: files: 594 min: 1b max: 2336b total: 146060b rss: 27Mb
luaL_addgsub_test: lauxlib.c:548: char *prepbuffsize(luaL_Buffer *, size_t, int): Assertion `((B)->b != (B)->init.b) ? lua_touserdata(B->L, boxidx) != ((void*)0) : lua_touserdata(B->L, boxidx) == (void*)B' failed.
==3586== ERROR: libFuzzer: deadly signal

GH Actions logs: https://github.com/ligurio/lua-c-api-tests/actions/runs/4977823137/jobs/8907335034

Answer 1 · 2023-05-16T19:13:45.000Z

Assertion triggered when CFLAGS -DLUAI_ASSERT=1 -DLUA_USE_APICHECK=1 are enabled.
According to description for luaL_Buffer ¹ there are two ways of using Buffer. First one works fine, second one failed with assertion.

https://www.lua.org/manual/5.4/manual.html#luaL_Buffer ↩

Answer 2 · 2023-05-16T19:16:08.000Z

Example of using luaL_Buffer in loadlib.c: https://github.com/lua/lua/blob/6443185167c77adcc8552a3fee7edab7895db1a9/loadlib.c#L471-L478

Footnotes