senx/warp10-platform

[Discussion] Improve the naming convention of reducers regarding the treatment of null values

randomboolean opened this issue · 0 comments

The suffixes of reducer names inform how they treat null values.

This naming convention is functional but it has some inconsistencies that can be confusing. It should be possible to improve it.

The suffixes are:

  • exclude-nulls : null values are removed
  • include-nulls : null values are not removed
  • forbids-nulls : an error is raised if a null value is encountered
  • nonnull : return null if at least one null value is encountered
  • ignore-nulls : similar to exclude-null but for APPLY

Inconsistency 1: Missing variants
Some base reducers have some variants with these suffixes, some don't.
(for example reducer.product does not have the same variant than reducer.sum)
This could be fixed by implementing the missing ones.

Inconsistency 2: Overlapping functionality
Does ignore-nulls means the same than exclude-nulls ?

Inconsistency 3: Default behavior
Base reducers don't have the same default behavior to null values.
For example reducer.mean behaves like if it were suffixed nonnull,
whereas reducer.median behaves like if it were suffixed exclude-nulls.