Simpler custom component support

[jbednar]

Right now, the home page says that custom components can be provided by putting them into filters.py, sources.py, transforms.py, or views.py in the directory where you launch lumen. That's great, but having so many files seems like overkill for most users, particularly if they are using multiple projects in different directories, where they will end up copying all those files between multiple project directories. My gut tells me we should just suggest they put things into a single file, e.g. custom.py, and save those other files for more obscure cases where people want to install extensions in some central location.

It might be better to make this more explicit as well, at least allowing a user to state explicitly that a given .py file (of any name) is to be executed before continuing: lumen serve --run custom.py dashboard.yml. (If not --run, maybe --custom or --preload.) That way they can keep the file in a central location covering multiple projects, if desired, and indeed we may want to support running ~/.lumen/custom.py automatically for that reason (though that does have security implications and is less explicit).

[philippjfr]

That way they can keep the file in a central location covering multiple projects

Disgree with this, if they want to keep it in a central location they should make a package out of it and explicitly provide the import path. Having it in some global location will inevitably result in users forgetting to bundle up the file when they share the dashboard specification and result in a non-reproducible environment. So I will strongly argue for only these two approaches being supported:

1. You provide one or more files which must live right alongside the dashboard specification. I agree that this does not have to be separate files for each type of component so I'm happy to standardize this into a single custom.py file but continuing to support separate files (i.e. filters.py, sources.py, transforms.py and views.py).

2. To reference any other external component users should supply the full import path to their custom package in the type field, e.g. type: my_custom_package.some_module.MyCustomSource.

[jbednar]

1 sounds good, but I'm not sure how you imagine 2 to work with Lumen Builder. How would such components become available in the dropdown selectors? That's the case I'm focusing on.

[philippjfr]

How would such components become available in the dropdown selectors?

The plan is to support them as plugins via setuptools entry_points.

[jbednar]

Sounds good. We'll need some good docs explaining these two approaches (app-focused and module-focused) and making it easy for people to get started defining such a module, including how to develop with it (e.g. pip install -e .).