carlosabalde/libvmod-cfg

LuaJIT Bitwise operations extension doesn't work

Closed this issue · 5 comments

Following the recommendations on the LuaJIT website (http://luajit.org/extensions.html) we add a require line at the top of our script

local bit = require("bit")

This already installed module should be ignored by LuaJIT, however it causes an error:

[CFG][script_check_callback:1232] Remote successfully compiled (script=permissions, location=<REDACTED>, is_backup=0, function=f_de835ef87440a4a728c8d3d31c1a17336301333040250a341fb8bd7389e8abec, code=local bit = require("bit")#012#012if tonumber(bit.band(arg[1], arg[2])) > 0 then #012  --...)

[CFG][execute:1099] Failed to execute script (script=permissions, function=f_de835ef87440a4a728c8d3d31c1a17336301333040250a341fb8bd7389e8abec, code=local bit = require("bit")#012#012if tonumber(bit.band(arg[1], arg[2])) > 0 then #012  --...): @varnish_script:2: varnish_script:2: module 'bit' not found:#012#011no field package.preload['bit']#012#011no file './bit.lua'#012#011no file '/usr/local/share/luajit-2.0.5/bit.lua'#012#011no file '/usr/local/share/lua/5.1/bit.lua'#012#011no file '/usr/local/share/lua/5.1/bit/init.lua'#012#011no file './bit.so'#012#011no file '/usr/local/lib/lua/5.1/bit.so'#012#011no file '/usr/local/lib/lua/5.1/loadall.so'


[CFG][vmod_script_execute:1528] Got error while executing script (script=permissions, function=f_de835ef87440a4a728c8d3d31c1a17336301333040250a341fb8bd7389e8abec, code=local bit = require("bit")#012#012if tonumber(bit.band(arg[1], arg[2])) > 0 then #012  --...)

We've tried various libvmod-cfg settings with no success

lua_load_package_lib=1,
lua_load_io_lib=1,
lua_load_os_lib=1

This is happening on Varnish 6.0.3
Running the lua script manually from the command line with luajit works without issue

That's probably a problem with the Varnish daemon environment (LUA_PATH, LUA_CPATH, LD_LIBRARY_PATH, etc.). I assume the following test will pass:

varnishtest ""

server s1 {
    rxreq
    txresp
} -start

varnish v1 -vcl {
    import cfg;

    backend s1 {
        .host = "${s1_addr}";
        .port = "${s1_port}";
    }

    sub vcl_init {
        new script = cfg.script(
            "/dev/null",
            period=0,
            type=lua,
            lua_remove_loadfile_function=false,
            lua_load_package_lib=true,
            lua_load_io_lib=true,
            lua_load_os_lib=true);
    }

    sub vcl_deliver {
        script.init({"
            local bit = require('bit')
            varnish.set_header('X-Success', '1', 'resp')
        "});
        script.execute();
        script.free_result();
    }
} -start

client c1 {
    txreq
    rxresp
    expect resp.http.X-Success == "1"
} -run

If not, you need to play with luarocks path and LD_LIBRARY_PATH. If it passes, you'll need to adjust the environment of the varnish daemon.

The test was unsuccessful
I had to remove type=lua from the script init otherwise an error was generated

**** v1    0.2 CLI RX|Message from VCC-compiler:
**** v1    0.2 CLI RX|Unknown argument 'type'
**** v1    0.2 CLI RX|('<vcl.inline>' Line 14 Pos 13)
**** v1    0.2 CLI RX|            type=lua,
**** v1    0.2 CLI RX|------------####-----
**** v1    0.2 CLI RX|
**** v1    0.2 CLI RX|Running VCC-compiler failed, exited with 2
**** v1    0.2 CLI RX|VCL compilation failed

After removing type=lua i encountered the same error as before
Changing my own config following your suggestion above still produced the same error

I've attached the varnishtest output
varnishtest.txt

That's useful info. Therefore, in addition to the potential problem in the environment of the Varnish daemon, if you've had to remove the type argument, that means you're using and old version of the VMOD. I mean, execution of arbitrary Lua extensions is not possible in the version you're using. You'll need to use last commit in 6.0 branch.

type=lua works using latest code in 6.0 branch

i'm not sure where luajit keeps the bitop functions, apparently its built in so i haven't installed any modules to offer that functionality

That being said, using the 6.0 branch from your git repo and manually installing the bitop module now works properly with both the varnishtest script and my original vcls

Thank you for this project and your time to help with this issue!

Thanks!