The definition of conform is "Adapt or conform oneself to new or different conditions". As this library is used to adapt your application to its deployed environment, I think it's rather fitting. It's also a play on the word configuration, and the fact that Conform uses an init-style configuration, maintained in a .conf
file.
Conform is a library for Elixir applications. Its original intended use is in exrm as means of providing a simplified configuration file for deployed releases, but is flexible enough to work for any use case where you want init-style configuration translated to Elixir/Erlang terms. It is inspired directly by basho/cuttlefish
, and in fact uses its .conf parser. Beyond that, you can look at conform as a reduced (but growing!) implementation of cuttlefish in Elixir.
You can use Conform either via its API, which is simple and easy to pick up, or by building the escript and running that via the command line.
Running the escript's help, you'll see how it's used:
↪ ./conform --help
Conform - Translate the provided .conf file to a .config file using the given schema
-------
usage: conform --conf foo.conf --schema foo.schema.exs [options]
Options:
--filename <name>: Names the output file <name>.config
--output-dir <path>: Outputs the .config file to <path>/<sys|name>.config
--config <config>: Merges the translated configuration over the top of
<config> before output
-h | --help: Prints this help
Conform also provides some mix tasks for generating and viewing configuration:
mix conform.new
- Generates a schema from your current project configurationmix conform.configure
- Generates a default .conf from your schema file and current project configurationmix conform.effective
- View the effective configuration for your release.
There are additional options for these tasks, use mix help <task>
to view their documentation.
You'll need to use the conform_exrm plugin. See it's README to get started.
The conform .conf file looks something like the following:
# Choose the logging level for the console backend.
# Allowed values: info, error
lager.handlers.console.level = info
# Specify the path to the error log for the file backend
lager.handlers.file.error = /var/log/error.log
# Specify the path to the console log for the file backend
lager.handlers.file.info = /var/log/console.log
# Remote database hosts
myapp.db.hosts = 127.0.0.1:8000, 127.0.0.2:8001
# Just some atom.
myapp.some_val = foo
# Determine the type of thing.
# * all: use everything
# * some: use a few things
# * none: use nothing
# Allowed values: all, some, none
myapp.another_val = all
# complex data types with wildcard support
complex_list.first.username = "username1"
complex_list.first.age = 20
complex_list.second.username = "username2"
complex_list.second.age = 40
Short and sweet, and most importantly, easy for sysadmins and users to understand and modify. The real power of conform though is when you dig into the schema file. It allows you to define documentation, mappings between friendly setting names and specific application settings in the underlying sys.config, define validation of values via datatype specifications, provide default values, and transform simplified values from the .conf into something more meaningful to your application using translation functions.
A schema is basically a single data structure. A keyword list, containing the following top-level properties, extends
, import
, mappings
, and transforms
. Before we dive in, here's the schema for the .conf file above:
[
extends: [],
import: [],
mappings: [
"lager.handlers.console.level": [
doc: """
Choose the logging level for the console backend.
""",
to: "lager.handlers",
datatype: [enum: [:info, :error]],
default: :info
],
"lager.handlers.file.error": [
doc: """
Specify the path to the error log for the file backend
""",
to: "lager.handlers",
datatype: :binary,
default: "/var/log/error.log"
],
"lager.handlers.file.info": [
doc: """
Specify the path to the console log for the file backend
""",
to: "lager.handlers",
datatype: :binary,
default: "/var/log/console.log"
],
"myapp.db.hosts": [
doc: "Remote database hosts",
to: "myapp.db.hosts",
datatype: [list: :ip],
default: [{"127.0.0.1", "8001"}]
],
"myapp.some_val": [
doc: "Just some atom.",
to: "myapp.some_val",
datatype: :atom,
default: :foo
],
"my_app.complex_list.*": [
to: "my_app.complex_list",
datatype: [:complex],
default: []
],
"my_app.complex_list.*.type": [
to: "my_app.complex_list",
datatype: :atom,
default: :undefined
],
"my_app.complex_list.*.age": [
to: "my_app.complex_list",
datatype: :integer,
default: 30
]
],
transforms: [
"myapp.another_val": fn val ->
case val do
:all -> {:on, [debug: true, tracing: true]}
:some -> {:on, [debug: true]}
:none -> {:off, []}
_ -> {:off, []}
end
end,
"lager.handlers.console.level": fn
_mapping, level, nil when level in [:info, :error] ->
[lager_console_backend: level]
_mapping, level, acc when level in [:info, :error] ->
acc ++ [lager_console_backend: level]
_, level, _ ->
IO.puts("Unsupported console logging level: #{level}")
exit(1)
end,
"lager.handlers.file.error": fn
_, path, nil ->
[lager_file_backend: [file: path, level: :error]]
_, path, acc ->
acc ++ [lager_file_backend: [file: path, level: :error]]
end,
"lager.handlers.file.info": fn
_, path, nil ->
[lager_file_backend: [file: path, level: :info]]
_, path, acc ->
acc ++ [lager_file_backend: [file: path, level: :info]]
end,
"my_app.complex_list.*": fn _, {key, value_map}, acc ->
[[name: key,
username: value_map[:username],
age: value_map[:age]
] | acc]
end
]
]
This looks pretty daunting, but I've provided mix tasks to help you generate the schema from your existing config.exs
file. Once you've gotten the schema tightened up though, you'll start to understand why it's worth a little extra effort up front. Let's talk about the top-level properties of the schema briefly:
extends
allows you to extend the schema of another application in your project.import
allows you to import an application's modules which contain functions you wish to use.mappings
define the mapping between the .conf settings and your actual configuration keys. We'll get more into those later.transforms
define transformations of mapped values for more complex configuration scenarios.
Mappings are defined by four key properties (there are more, just see the Conform.Schema.Mapping
module docs):
:doc
, documentation on what this setting is, and how to use it:to
, if you want friendly names for not so friendly app settings,:to
tells conform what setting this mapping applies to in the generated.config
:datatype
, the datatype of the value, currently supports binary, charlist, atom, integer, float, ip (a tuple of strings{ip, port}
), enum, and lists of one of those types. I currently am planning on supporting user-defined types, but that work has not yet been completed.:default
, optional, the value to use if one is not supplied for this setting. Should be the same form as the datatype for the setting. So for example, if you have a setting,myapp.foo
, with a datatype of[enum: [:info, :warn, :error]]
, then your default value should be one of those three atoms.
After all settings have been mapped, each of the transforms is executed with the PID of the ETS table holding the config. You will need to use the Conform.Conf
module API to query the configuration state using this PID. The value returned from the transform is then paired with the key the transformed is defined against and used in the final configuration.
The following is the output configuration from conform using the .conf and .schema.exs files shown above:
[{lager, [
{handlers, [
{lager_console_backend, info},
{lager_file_backend, [{file, "var/log/error.log"}, {level, error}]},
{lager_file_backend, [{file, "/var/log/console.log"}, {level, info}]}
]}]},
{myapp, [
{another_val, {on, [{debug, true}, {tracing, true}]}},
{some_val, foo},
{db, [{hosts, [{"127.0.0.1", "8001"}]}]},
[complex_list: [first: %{age: 20, username: "username1"},
second: %{age: 40, username: "username2"}]]
]}].
As you can see, if your sysadmins had to work with the above, versus the .conf, it would be quite prone to mistakes, and much harder to understand, particularly with the lack of comments or documentation.
If you are using exrm
and need to import any applications from the your_app/deps
, you can update your you_app.schema.exs
with the import
:
[
import: [
:my_app_dep1,
:my_app_dep2
],
mappings: [
...
...
...
],
transforms: [
...
...
...
]
]
Will be created archive with the myapp.schema.ez
name in the your release which will contain the my_app_dep1
and my_app_dep2
applications. During the sys.config
will be generated by conform script, the applications from the archive will be loaded and you can use any public API from these applications in your transforms. Conform also allows to use a schema with imports without exrm
. There is the special conform.archive
mix task that takes one parameter - path of the schema:
mix conform.archive myapp/config/myapp.schema.exs
Conform
will collect dependencies which are pointed in the import: [....]
and compress them to the myapp/config/myapp.schema.ez
archive. After this you can use the conform
script as always:
mix conform.new --conf myapp/config/myapp.conf --schema myapp/config/myapp.schema.exs
I've also provided mix tasks to handle generating your initial .conf and .schema.exs files, which includes the default options, and the documentation. The end result is an easy to maintain configuration file for your users, and ideally, a powerful tool for managing your own configuration as well.
Conform
provides ability to use custom data types in your schemas:
[
mappings: [
"myapp.val1": [
doc: "Provide some documentation for val1",
to: "myapp.val1",
datatype: MyModule1,
default: 100
],
"myapp.val2": [
doc: "Provide some documentation for val2",
to: "myapp.val2",
datatype: [{MyModule2, [:dev, :prod, :test]}],
default: :dev
]
],
transforms: [
...
...
...
]
]
Where MyModule1
and MyModule2
must be modules which implement the Conform.Type
behaviour:
defmodule MyModule1 do
use Conform.Type
# Return a string to produce documentation for the given type based on it's valid values (if specified).
# If nil is returned, the documentation specified in the schema will be used instead (if present).
def to_doc(values) do
"Document your custom type here"
end
# Converts the .conf value to this data type.
# Should return {:ok, term} | {:error, term}
def convert(val, _mapping) do
{:ok, val}
end
end
Conform is a library for Elixir applications, specifically in the release phase. Elixir already offers a convenient configuration mechanism via config/config.exs
, but it has downsides:
- To change config settings, it requires you to recompile your app to regenerate the app.config/sys.config files used by the VM. Alternatively you can modify the sys.config file directly during deployment, using Erlang terms. Neither of these things are ops-friendly, or necessarily accessible to sysadmins who may not understand Elixir or Erlang semantics.
- You can put comments in
config/config.exs
, but once transformed to app.config/sys.config, those comments are lost, leaving sysadmins lost when trying to understand what configuration values are allowed and what they do. - There is no config validation
- You can't offer a nice interface for configuration to your apps users via something akin to conform's translations. They have to know how to work in Elixir terms, which is pleasant enough for a dev, but not so much for someone unfamiliar with programming.
Conform is intended to fix these problems in the following way:
- It uses an init-style configuration, which should be very familiar to any sysadmin.
- It is intended to be used during the release process, once your app has been deployed.
- Conform works by taking the schema, the .conf file, and
config.exs
if it is being used, and combines them into thesys.config
file used by the Erlang VM. However, unlikeconfig.exs
, you can bring the .conf into production with you, and use it for configuration instead ofsys.config
. This means that the docs provided in your schema file are available to the users configuring your application in production. The .conf is validated when it is parsed as well, so your users will get immediate feedback if they've provided invalid config settings.
I'm glad to hear from anyone using this on what problems they are having, if any, and any ideas you may have. Feel free to open issues on the tracker or come find me in #elixir-lang
on freenode.
The .conf parser in conform_parse.peg
is licensed under Apache 2.0, per Basho. The rest of this project is licensed under the MIT license. Use as you see fit.