KafkaSupervisorIOConfig's RejectionPeriod config
zhaojiandong opened this issue · 0 comments
zhaojiandong commented
Does this 2 config mixed up ?
lateMessageRejectionPeriod and earlyMessageRejectionPeriod
http://druid.io/docs/latest/development/extensions-core/kafka-ingestion.html#KafkaSupervisorIOConfig
Field | Type | Description | Required |
---|---|---|---|
lateMessageRejectionPeriod |
ISO8601 Period | Configure tasks to reject messages with timestamps earlier than this period before the task was created; for example if this is set to PT1H and the supervisor creates a task at 2016-01-01T12:00Z, messages with timestamps earlier than 2016-01-01T11:00Z will be dropped. This may help prevent concurrency issues if your data stream has late messages and you have multiple pipelines that need to operate on the same segments (e.g. a realtime and a nightly batch ingestion pipeline). |
no (default == none) |
earlyMessageRejectionPeriod |
ISO8601 Period | Configure tasks to reject messages with timestamps later than this period after the task reached its taskDuration; for example if this is set to PT1H , the taskDuration is set to PT1H and the supervisor creates a task at 2016-01-01T12:00Z, messages with timestamps later than 2016-01-01T14:00Z will be dropped. |
no (default == none) |