Complete implementation of plugin host

[myitcv]

Needs to follow ideas laid out in neovim/neovim#1335

• Probably start basic with a plugin host representing the compiled combination of n plugin packages
• Neovim scripts/hooks to load Go plugin host and supported functions

Refer as a starting point to the Python provider

Also reference old conversation here

[tarruda]

Some concept of configuration file, that lists which Go-based plugins are active

That will be done by a "manifest" that is required for every plugin that implemented in an external host. This manifest will be nothing more than a vimscript file(eg: plugin/manifest.vim) that will use the vimscript library I'm implementing in #1335.

Here's an idea of how a plugin manifest can look like:

" Manifest for a go plugin implemented in the file 'x.go' 
call rpc#plugin#Register({
\  'github.com/myitcv/X': {
\      'host': 'Go',
\      'path': expand('<sfile>:p:h').'/x.go',  " assuming the plugin implementation is in x.go
\      'register': [
\         rpc#plugin#Command('XCommand1, {'implementation': 'XCmd1', 'nargs': 1, 'count': 2}),
\         rpc#plugin#Command('XCommand2, {'implementation': 'XCmd2', 'nargs': '*}),
\         rpc#plugin#AutoCommand('BufRead', {'pattern': '*.go', 'implementation': 'XBufRead'}),
\         rpc#plugin#AutoCommand('BufWrite', {'pattern': '*.go', 'implementation': 'XBufWrite'}),
\         rpc#plugin#Function('XFunction', {'implementation': 'XFunc'})
\      ]
})

what the rpc#plugin#Register function will do is define commands, autocommands and functions that have a lazy implementation: The real implementation is defined the first time it is called, and will load the host if it hasn't been loaded already.

Some nice things about this pattern of using vimscript as a manifest format:

• The manifest is loaded at startup, just like a normal vimscript plugin
• The host itself is only loaded when the plugin is required(a command, autocmmand or function is used)
• The plugin host has no knowledge of channel ids(host->channel mapping is done by the vimscript library, which is common to all hosts)

So for example, here's an idea of what will happen when a user executes XCommand1 for the first time:

• The go host will be loaded, it's channel id will be stored
• The host will received a plugin_load request for the full path of "x.go"
• XCommand1 will be redefined to send the msgpack-rpc call command:XCmd1 to the channel of the host.
• The redefined XCommand1 will be executed.

I also think it will be possible for us to design a scheme for automatically-generating this manifest when a plugin is installed(similar to help tags), this way the plugin author won't ever have to touch vimscript.

[tarruda]

I've been searching for a pattern of dynamically loading go code and found this SO answer.

While I will do my best add explicit support host reloading(which is required for static languages like go), perhaps an initial implementation of a go host could simply load the "real host"(containing all compiled plugins) as an external process and talk to it with the rpc/net package?

Here's a small chart showing the processes:

           Nvim                     
           ^  +                     
           |  |                     
           |  |                     
           +  v                     
 Golang host(handle communication between Nvim and the real host)      
           ^  +                     
           |  |                     
           |  |                     
           +  v                     
Real host(with the compiled plugins)

Loading/unloading golang plugins would be handled by the golang host by reloading the "real host" recompiled with the new plugin list.