fancyflags

TIP: Already a fancyflags user? Check out our usage tips!

The main feature of fancyflags is a nested dict flag, with familiar "dot" overrides for fields. These dict flags:

Can be overridden with “dot” notation, similar to config_flags.
Are typed and validated like standard flags, catching errors before they propagate into your program.
Are easy to mix and match with standard flags, without adding extra files to your codebase.
Can be unpacked into constructor/function calls, e.g. replay(**FLAGS.replay).

Installation

fancyflags can be installed from PyPI using pip:

pip install fancyflags

It can also be installed directly from our GitHub repository:

pip install git+git://github.com/deepmind/fancyflags.git

or alternatively by checking out a local copy of our repository and running:

pip install /path/to/local/fancyflags/

Quickstart

If we have a class Replay, with arguments capacity, priority_exponent and others, we can define a corresponding dict flag in our main script

import fancyflags as ff

_REPLAY_FLAG = ff.DEFINE_dict(
    "replay",
    capacity=ff.Integer(int(1e6)),
    priority_exponent=ff.Float(0.6),
    importance_sampling_exponent=ff.Float(0.4),
    removal_strategy=ff.Enum("fifo", ["rand", "fifo", "max_value"])
)

and **unpack the values directly into the Replay constructor

replay_lib.Replay(**REPLAY_FLAG.value)

ff.DEFINE_dict creates a flag named replay, with a default value of

{
    "capacity": 1000000,
    "priority_exponent": 0.6,
    "importance_sampling_exponent": 0.4,
    "removal_strategy": "fifo",
}

For each item in the dict, ff.DEFINE_dict also generates a dot-delimited "item" flag that can be overridden from the command line. In this example the item flags would be

replay.capacity
replay.priority_exponent
replay.importance_sampling_exponent
replay.removal_strategy

Overriding an item flag from the command line updates the corresponding entry in the dict flag. The value of the dict flag can be accessed by the return value of ff.DEFINE_dict (_REPLAY_FLAG.value in the example above), or via the FLAGS.replay attribute of the absl.flags module. For example, the override

python script_name.py -- --replay.capacity=2000 --replay.removal_strategy=max_value

sets _REPLAY_FLAG.value to

{
    "capacity": 2000,  # <-- Overridden
    "priority_exponent": 0.6,
    "importance_sampling_exponent": 0.4,
    "removal_strategy": "max_value",  # <-- Overridden
}

Nested dicts

fancyflags also supports nested dictionaries:

_NESTED_REPLAY_FLAG = ff.DEFINE_dict(
    "replay",
    capacity=ff.Integer(int(1e6)),
    exponents=dict(
        priority=ff.Float(0.6),
        importance_sampling=ff.Float(0.4),
    )
)

In this example, _NESTED_REPLAY_FLAG.value would be

{
    "capacity": 1000000,
    "exponents" : {
      "priority": 0.6,
      "importance_sampling": 0.4,
    }
}

and the generated flags would be

replay.capacity
replay.exponents.priority
replay.exponents.importance_sampling

Help strings

fancyflags uses the item flag's name as the default help string, however this can also be set manually:

_NESTED_REPLAY_FLAG = ff.DEFINE_dict(
    "replay",
    capacity=ff.Integer(int(1e6), "Maximum size of replay buffer."),
    exponents=dict(
        priority=ff.Float(0.6),  # Help string: "replay.exponents.priority"
        importance_sampling=ff.Float(0.4, "Importance sampling coefficient."),
    )
)

"Auto" flags for functions and other structures.

fancyflags also provides an ff.DEFINE_auto flag generate flags corresponding to a callable. The return value will also carry the correct type information.

This callable could be a constructor

_REPLAY = ff.DEFINE_auto('replay', replay_lib.Replay, "replay flag")

or it could be a dataclass

@dataclasses.dataclass
class DataSettings:
  dataset_name: str = 'mnist'
  split: str = 'train'
  batch_size: int = 128

# In main script.
# Exposes flags: --data.dataset_name --data.split and --data.batch_size.
_DATA_SETTINGS = ff.DEFINE_auto('data', datasets.DataSettings, 'Data config')

def main(argv):
  # del argv  # Unused.
  dataset = datasets.load(_DATA_SETTINGS.value())
  # ...

or any other callable that satisfies the ff.auto requirements. It's also possible to override keyword arguments in the call to .value(), e.g.

test_settings = _DATA_SETTINGS.value(split='test')

Defining a dict flag from a function or constructor.

The function ff.auto returns a dictionary of ff.Items given a function or constructor. This is used to build ff.DEFINE_dict and is also exposed in the top-level API.

ff.auto can be used with ff.DEFINE_dict as follows:

_WRITER_KWARGS = ff.DEFINE_dict('writer', **ff.auto(logging.Writer))

ff.auto may be useful for creating kwarg dictionaries in situations where ff.DEFINE_auto is not suitable, for example to pass kwargs into nested function calls.

Auto requirements

ff.DEFINE_auto and ff.auto will work if:

The function or class constructor has type annotations.
Each argument has a default value.
The types of the arguments are relatively simple types (int, str, bool, float, or sequences thereof).

Notes on using `flagsaver`

abseil-py's flagsaver module is useful for safely overriding flag values in test code. Here's how to make it work well with fancyflags.

Making dotted names work with `flagsaver` keyword arguments.

Since flagsaver relies on keyword arguments, overriding a flag with a dot in its name will result in a SyntaxError:

# Invalid Python syntax.
flagsaver.flagsaver(replay.capacity=100, replay.priority_exponent=0.5)

To work around this, first create a dictionary and then ** unpack it:

# Valid Python syntax.
flagsaver.flagsaver(**{'replay.capacity': 100, 'replay.priority_exponent': 0.5})

Be careful when setting flag values inside a `flagsaver` context

If possible we recommend that you avoid setting the flag values inside the context altogether, and instead pass the override values directly to the flagsaver function as shown above. However, if you do need to set values inside the context, be aware of this gotcha:

This syntax does not work properly:

with flagsaver.flagsaver():
  FLAGS.replay["capacity"] = 100
# The original value will not be restored correctly.

This syntax does work properly:

with flagsaver.flagsaver():
  FLAGS["replay.capacity"].value = 100
# The original value *will* be restored correctly.

fancyflags in more detail

What is an `ff.Float` or `ff.Integer`?

ff.Float and ff.Integer are both ff.Items. An ff.Item is essentially a mapping from a default value and a help string, to a specific type of flag.

The ff.DEFINE_dict function traverses its keyword arguments (and any nested dicts) to determine the name of each flag. It calls the .define() method of each ff.Item, passing it the name information, and the ff.Item then defines the appropriate dot-delimited flag.

What `ff.Item`s are available?

ff.Item	Corresponding Flag
`ff.Boolean`	`flags.DEFINE_boolean`
`ff.Integer`	`flags.DEFINE_integer`
`ff.Enum`	`flags.DEFINE_enum`
`ff.EnumClass`	`flags.DEFINE_enum_class`
`ff.Float`	`flags.DEFINE_float`
`ff.Sequence`	`ff.DEFINE_sequence`
`ff.String`	`flags.DEFINE_string`
`ff.StringList`	`flags.DEFINE_list`
`ff.MultiEnum`	`ff.DEFINE_multi_enum`
`ff.MultiEnumClass`	`flags.DEFINE_multi_enum_class`
`ff.MultiString`	`flags.DEFINE_multi_string`

Defining a new `ff.Item`

Given a flags.ArgumentParser, we can define an ff.Item in a few lines of code.

For example, if we wanted to define an ff.Item that corresponded to flags.DEFINE_spaceseplist, we would look for the parser that this definition uses, and write:

class SpaceSepList(ff.Item):

  def __init__(self, help_string)
    parser = flags.WhitespaceSeparatedListParser()
    super(SpaceSepList, self).__init__(default, help_string, parser)

Note that custom ff.Item definitions do not need to be added to the fancyflags library to work.

Defining `Item` flags only

We also expose a define_flags function, which defines flags from a flat or nested dictionary that maps names to ff.Items. This function is used as part of ff.DEFINE_dict and ff.DEFINE_auto, and may be useful for writing extensions on top of fancyflags.

_writer_items = dict(
    path=ff.String('/path/to/logdir', "Output directory."),
    log_every_n=ff.Integer(100, "Number of calls between writes to disk."),
)

_WRITER_KWARGS = ff.define_flags("writer", _writer_items)

This example defines the flags replay.capacity and replay.priority_exponent only: does not define a dict-flag. The return value (REPLAY) is a dictionary that contains the default values. Any overrides to the individual flags will also update the corresponding item in this dictionary.

Tips

Any direct access, e.g. _DICT_FLAG.value['item'] is an indication that you may want to change your flag structure:

Try to align dict flags with constructors or functions, so that you always **unpack the items into their corresponding constructor or function.
If you need to access an item in a dict directly, e.g. because its value is used in multiple places, consider moving that item to its own plain flag.
Check to see if you should have **unpacked somewhere up the call-stack, and convert function "config" args to individual items if needed.
Don't group things under a dict flag just because they're thematically related, and don't have one catch-all dict flag. Instead, define individual dict flags to match the constructor or function calls as needed.

corypaik/fancyflags