druid-io/druid-io.github.io

KafkaSupervisorIOConfig's RejectionPeriod config

zhaojiandong opened this issue · 0 comments

Does this 2 config mixed up ?
lateMessageRejectionPeriod and earlyMessageRejectionPeriod

http://druid.io/docs/latest/development/extensions-core/kafka-ingestion.html#KafkaSupervisorIOConfig

Field Type Description Required
lateMessageRejectionPeriod ISO8601 Period Configure tasks to reject messages with timestamps earlier than this period before the task was created; for example if this is set to PT1H and the supervisor creates a task at 2016-01-01T12:00Z, messages with timestamps earlier than 2016-01-01T11:00Z will be dropped. This may help prevent concurrency issues if your data stream has late messages and you have multiple pipelines that need to operate on the same segments (e.g. a realtime and a nightly batch ingestion pipeline). no (default == none)
earlyMessageRejectionPeriod ISO8601 Period Configure tasks to reject messages with timestamps later than this period after the task reached its taskDuration; for example if this is set to PT1H, the taskDuration is set to PT1H and the supervisor creates a task at 2016-01-01T12:00Z, messages with timestamps later than 2016-01-01T14:00Z will be dropped. no (default == none)