AbsaOSS/cobrix

Add custom record extractor index support self-check

Closed this issue · 0 comments

Background

Writing a custom record extractor compatible with Cobrix indexes is tricky. And if the protocol is not followed 100% data loss may occur when using Cobrix indexes.

But Cobrix can do a quick self check of a custom record extractor when loading a data file. It just needs t ensure that offset returned for a second record will return the second record if reused when constructing the record extractor with a non-zero offset.

If the custom record extractor does not conform to the requirement an exception can be thrown with the advice to turn off indexes.

Feature

Add custom record extractor index support self-check.

Example [Optional]

--

Proposed Solution [Optional]

--