Known Issue: Partially completed commit can leave unused Node(s) behind

Discussed in #8

^{Originally posted by grecinto January 21, 2024}
During application crash and SOP transaction(s) are in the middle of "commit", temporary Node(s) used by these transaction(s) can be left orphaned. These Nodes' references are recorded in the registry as "inactive" ID(s). They can be cleaned up by SOP on succeeding transaction(s) involving their "active" Node(s).

BUT in order not to impact I/O or performance of the system, they were decided to be left as such. It is anticipated that occurrence of this kind of partially completed transactions will be very scarce to zero, thus, it will not impact storage efficiency in time. The other approach known, is to issue a "delete" on these Node(s) part of future transactions, thus, doing an auto-cleanup. However, these are expensive operations and potentially be doing nothing, as most leftover unused Node(s) will be serviced by the "Delete Service" or the SOP commit's cleanup phase. (see here where the auto-cleanup for this use-case was being done, was commented out:

sop/in_red_ck/node_repository.go

Line 288 in c6d8a17

// if !kafka.LastEnqueueSucceeded() {

)

If in case your setup produces many occurrences of this kind of transactions, pls. feel free to report it(together with repro steps, as usual).
Or, if you have a suggestion how to better handle this scenario, and how to introduce a cleanup task for such kind of Node(s), pls. do suggest for discussion.

Thanks & kind regards,

SOP maintainer

Being addressed & latest code changes for this are in "value data segment" branch.

Fixed in latest checkin in master branch.