Lua: store metatables on vim.b/vim.w/vim.t/vim.v scopes

[tjdevries]

Actual behaviour

let g:var = {}
lua vim.g.x = 1
echo g:var
" Result: {}

Expected behaviour

let g:var = {}
lua vim.g.x = 1
echo g:var
" Result: {'x': 1}

It's confusing that it doesn't do this currently I think for people. Either we should do it automatically or find some way to make it "easy" to happen without completely setting/resetting the dict every time.

[tjdevries]

@norcalli thoughts? We can use metatables to make this work just fine.

[norcalli]

True, I do think it is confusing. My concerns would be:

• Might be too magical. If you keep a reference to a magic dict around, but the variable is overrided to a non dict, then setting that previous reference does... what?
• Dictionaries can contain vim function references, which we currently don't support. I'm actually not sure if g:variables can store functions right now without thinking of the dict case...

The possibilities are:

• fully support vim/lua interop
• support just for dictionaries and ignore incompatibility problems as user error
• raise an error when trying to operate on these returned dicts via metamethods.

Is there any other special type than dictionary and vim functions, btw?

[mg979]

@tjdevries did you mean

let g:var = {}
lua vim.g.var.x = 1
echo g:var
" Result: {'x': 1}

Anyway, how can I set/change a nested value in a vim global (dictionary) variable from a lua script, currently? I couldn't find a way.

[tjdevries]

Sorry, yeah, that's waht I meant to write @mg979

The way you could do it now is:

local t = vim.g.var
t.x = 1
vim.g.var = t

that should work, but it is not nice :'(

[mg979]

@tjdevries thanks

[justinmk]

Current model is simple. Simple is good. Vimscript-Lua bridge is an escape hatch, it is not something we should pour lots of engineering effort into unless it is blocking a long-term use-case. Vimscript <=> Lua is a short-term workaround, most logic should be in Lua.

Also even after expending a lot of effort we will still have strange edge cases. Whereas with the current simple approach, it is clear and easy to understand the limitations.

raise an error when trying to operate on these returned dicts via metamethods.

That might be helpful to plugin authors.

[ipod825]

I have a somewhat related question.

I have a lua class Buffer (table with metatable) corresponding to vim buffer. I'd like to store a reference to a Buffer in a vim buffer:

local lua_buffer = Buffer()
vim.api.nvim_buf_set_var(0, 'lua_buffer', lua_buffer)

This way, I don't need to worry about memory leak after the vim buffer is wiped out.
However, since lua_buffer contains meta table and nvim_buf_set_var does not seem to support translating meta table (I can see the complexity here), so what is stored into vim.b.lua_buffer is only non-meta stuff in lua_buffer.

My current workaround is by having a global table mapping from vim buffer id to lua Buffer and remove Buffer from the table on corresponding vim buffer wipeout autocmd.

Vimscript <=> Lua is a short-term workaround, most logic should be in Lua.

So my question is, is there a long term plan for having lua module replacing vim's scoped variables? For e.g., a table lb with same lifecycle as vim buffer.

vim.lb.var = 'only lives in lua'
assert(vim.b.var==nil)

[justinmk]

So my question is, is there a long term plan for having lua module replacing vim's scoped variables? For e.g., a table lb with same lifecycle as vim buffer.

@ipod825 you're right that lifetimes (b:, w:, t:) are an important use-case. Lua closures (vim.b=function()…) work nicely in the Vimscript-Lua bridge but you are asking specifically about metatable support.

Currently, bridging a Lua table erases the metatable:

local f = function()
  local M = { 'mt1', 'mt2' }
  local l = { a=1 }
  local call = function(_, f)
    if l[f] then
      l[f] = l[f] + 1
    else
      l[f] = 1
    end
    print(vim.inspect(l))
  end
  setmetatable(M, {
     __call = call,
     __index = call,
  })
  vim.b.foo = M
  GlobalM = M
  -- vim.api.nvim_buf_set_var(0, 'foo', M)

  print(vim.b.foo)  -- table: 0x0102605460
  print(GlobalM)    -- table: 0x01026163e0
  print(M)          -- table: 0x01026163e0
end
f()

-- Test cases:
--    :lua print(GlobalM['b'])
--        => works
--    :lua print(GlobalM('test'))
--        => works
--    :echo b:foo
--        => [ 'mt1', 'mt2' ]
--    :call b:foo()
--        => E884

Proposal

What if the bridge worked like this for container types:

1. In Vimscript, it just returns an opaque id/handle
   • nothing "bridged" except the variable name
2. In Vimscript, bridged Lua container types are readonly. In Lua, bridged Vimscript container types are readonly.
   • We should have done this from the beginning. It's essentially the same as current behavior, but less confusing.
3. In Lua, the ref continues to live entirely in Lua-land in Lua-managed b:/w:/t: tables.
   • When a buffer/window/tab closes, Nvim removes the Lua ref from the Lua b:/w:/t: table. Lua will garbage-collect it unless some other Lua stuff is still using it.
   • Not supported for g:/l: since there is no use-case?
4. Rules to document:
   • Lua tables are readonly from Vimscript.
   • Vimscript containers are readonly from Lua.
   • Lua metatable behavior only works from Lua (not v:lua, nor g:/l:/b:/w:/t:).

[ipod825]

The proposal looks good to me. As long as there are complete lua APIs for me to modify nvim internals, I don't really see any usecase to modify something on both sides of the worlds. Using vimscript in lua most of the times is due to lack of lua APIs like augroup/user command pre nvim 0.7.