Alternative OMERO configuration directory

[manics]

Alternative OMERO configuration directory conf.d/*

This is a proposal for supporting configuration of OMERO using a conf.d style directory.

Current implementation

OMERO stores most configurable parameters in etc/grid/config.xml.
This is an XML file, and in practice configuration settings are managed using the omero config command.

Problems

This configuration system is difficult to manage with configuration management systems such as Ansible which check for inconsistencies by comparing file differences.
etc/grid/config.xml is rewritten by OMERO on every start, so unless the configuration management system can exactly replicate the formatting used by OMERO it is impractical to use.

Proposed solution

Many server applications use a conf.d/* style directory.
The configuration can be split into multiple files which are automatically merged into one when the server is started.
Benefits of this include:

• Separation of configuration of different components
• Add-ons/modules can add a configuration section without having to parse the existing configuration
• These files are never modified by the server (unless the server includes an explicit configuration utility)

OMERO configuraiton directory

In OMERO.server this would involve the addition of a new configuration directory such as etc/conf.d/.
An alternative to this is to support a system level configuration directory such as /etc/omero/conf.d.

Format of configuration files

These files should be manually editable by administrator and configuration management tools.
Possibilities include:

• omero config edit format: omero.key.name="value"
• OMERO.cli config format: The arguments that would be passed to omero config
• OMERO.cli full format: The arguments that would be passed to omero (sometimes informally referred to as the .omero format).
• JSON: Many (all?) OMERO config values can be treated as JSON objects.
  omero config currently requires the values to be a single line string, if this was a JSON file it would be much easier to modify values by hand.
• YAML: JSON and YAML are mostly interchangeable, YAML has the advantage of being easier to manually edit.

These files should have a known extension e.g. *.conf, *.json, *.yml, depending on what format(s) are supported.
This allow backups to be made be appending an unrecognised extension.

It is not sufficient to just support the equivalent of omero config set since supporting configuration of multiple independent web applications requires appending to some properties.
Therefore it would be necessary to include a subcommand name such as set and append in the configuration file.
This might be limited to just these two subcommands, or all omero config subcommands.

Ordering of files

Files should be read in lexicographic order.
This means administrators can control the ordering applications which append to the same property.
In addition it also means an administrator could effectively invalidate manual changes to etc/grid/config.xml by creating a file named e.g. 00-clear with the equivalent of a configuration or command to clear all properties.

OMERO configuration parsing

An example, assuming a system level directory /etc/omero/conf.d and configuration files in json format.

1. Obtain a list of all configuration files in lexicographic order matching /etc/omero/conf.d/*.json
2. Concatenate all files (this may be in memory or a temporary file).
3. Parse each configuration entry to set/update etc/grid/config.xml as if bin omero config set ... was run.

Backwards compatibility

Changes to etd/grid/config.xml are not removed unless explicitly requested, and an empty or non-existent configuration directory will result in no changes, so this is fully compatible with the existing configuration system.