Registering custom boundary-layer-plugin

[y2k-shubham]

i wrote some (boundary-layer) schemas for my Airflow custom operators. Following same convention as boundary_layer_default_plugin, i bundled all configs, plugin.py and preprocessor.py file into a single folder.
.
now i was wondering how would boundary-layer pick my custom plugin and generate python code that employs those custom operators.
.
for POC, i used the hack where i manually updated entry_points.txt & top_level.txt files in pip's site-packages/boundary_layer-1.6.16.dist-info directory. And it worked: i was able to generate python DAG-definition file that employed my custom operators.
.
but i still don't understand what is the right way to deploy my boundary-layer plugin. i can do this by modifying setup.py file, but that defies the purpose of plugin: i will be modifying the source code of boundary-layer itself.

[y2k-shubham]

UPDATE
.
Today I came across boundary_layer/plugins/plugin_manager.py
.
While I'm looking into it, any pointers will be appreciated

[mchalek]

hi @y2k-shubham thanks for the question and sorry for the delay! The plugin system is based on python's entry_points (there are many resources on this, see e.g. here). The main idea is that your plugin can be packaged up with its own setup.py file, and made available to boundary-layer by defining an entry_point. In your plugin's setup.py, as one of the arguments to the setup() method, you would add a line like this:

    entry_points = {
            'boundary_layer_plugins': [
                    'my-plugin-name=my.package.name:MyPluginClassName',
                ]
        },

Then you would pip install both boundary-layer and your plugin package, and everything should "just work." You will know that your plugin was loaded successfully based on the initial log message when you run the boundary-layer executable, which will look something like:

2019-05-16 17:26:21,876 - boundary-layer v. 1.6.16 - INFO - Loaded plugins default, my-plugin

The specific line that searches for this is here in the plugin_manager code.

Please give this a try and let us know whether you have any trouble with it!