tests/jfill failure

[dkoreshkov]

Hello!
I added a call to srandom(time(NULL)) into sim_reset and ran "while tests/jfill.test; do :; done"
After a while:
jfill.test: tests/jtutil.c:61: jt_check: Assertion `root_offset < raw_size' failed.

At the time of the failure root is before tail: root 862, tail 864, head 872
It seems that the error recovery in journal_enqueue sometimes leaves a largish gap between root and head, so that journal_peek/dequeue get confused.

[dkoreshkov]

Oh, here is another one, takes longer to reproduce:
map.test: tests/map.c:68: check_recurse: Assertion `offset < p_offset' failed.

offset is -1

[dkoreshkov]

Well, the crash in map.test goes away when doing journal_dequeue in map_sync only if raw_gc was successful...

[dlbeer]

On Thu, Feb 18, 2021 at 11:55:05PM -0800, Denis wrote: Hello! I added a call to srandom(time(NULL)) into sim_reset and ran "while tests/jfill.test; do :; done" After a while: jfill.test: tests/jtutil.c:61: jt_check: Assertion `root_offset < raw_size' failed. At the time of the failure root is before tail: root 862, tail 864, head 872 It seems that the error recovery in journal_enqueue sometimes leaves a largish gap between root and head, so that journal_peek/dequeue get confused.

Hi Denis, That's an excellent idea for a test. I've reproduced it here iterating through explicit seeds from 0 up. I'll have a look at what's happening and get back to you within the next few days. Once that's sorted I'll have a look at your other issue and proposed fix. Cheers, Daniel

…

-- Daniel Beer <dlbeer@gmail.com> http://dlbeer.co.nz/ PGP: BA6E 0B26 1F89 246C E3F3 C910 1E58 C43A 160A 553B

[dlbeer]

On Fri, Feb 19, 2021 at 09:29:38AM -0800, Denis wrote: Well, the crash in map.test goes away when doing journal_dequeue in map_sync only if raw_gc was successful... -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #19 (comment)

That's definitely a bug! Fixed in commit a1c3e46.

…

-- Daniel Beer <dlbeer@gmail.com> http://dlbeer.co.nz/ PGP: BA6E 0B26 1F89 246C E3F3 C910 1E58 C43A 160A 553B

[dlbeer]

On Thu, Feb 18, 2021 at 11:55:05PM -0800, Denis wrote: Hello! I added a call to srandom(time(NULL)) into sim_reset and ran "while tests/jfill.test; do :; done" After a while: jfill.test: tests/jtutil.c:61: jt_check: Assertion `root_offset < raw_size' failed. At the time of the failure root is before tail: root 862, tail 864, head 872 It seems that the error recovery in journal_enqueue sometimes leaves a largish gap between root and head, so that journal_peek/dequeue get confused. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #19

Hi Denis, Just pushed a fix in commit 6cc5736. There are more details in the commit message, but this issue shouldn't have been causing any problems in practice. I've altered the behaviour of the journal so that the invariant of a valid root pointer is not violated anyway. Cheers, Daniel

…

-- Daniel Beer <dlbeer@gmail.com> http://dlbeer.co.nz/ PGP: BA6E 0B26 1F89 246C E3F3 C910 1E58 C43A 160A 553B

[dkoreshkov]

Many thanks for your replies, Daniel!

It seems the last fix could be simpler--clear the root just before incrementing the tail:
if (j->tail == j->root)
j->root = DHARA_PAGE_NONE;
j->tail = next_upage(j, j->tail);

[dlbeer]

On Sat, Feb 20, 2021 at 05:06:29AM -0800, Denis wrote: Many thanks for your replies, Daniel! It seems the last fix could be simpler--clear the root just before incrementing the tail: if (j->tail == j->root) j->root = DHARA_PAGE_NONE; j->tail = next_upage(j, j->tail);

Ah -- yes that probably would have been a bit nicer.

…

-- Daniel Beer <dlbeer@gmail.com> http://dlbeer.co.nz/ PGP: BA6E 0B26 1F89 246C E3F3 C910 1E58 C43A 160A 553B