Timestamps don't start in "pending" state

[OttoAllmendinger]

The timestamp state (unknown/pending/complete) seems very time-sensitive: a POST /digest that is immediately followed corresponding by GET /timestamp/$hash yields a 404 Not Found.

However, when inserting a one-second delay (the aggregation interval?), the response is instead 404 Calendar localhost: Pending confirmation in Bitcoin blockchain.

This can be pretty confusing when writing a client. I suggest the server should either respond with the "Pending confirmation" from the start, or with a different "Pending" message.

Script to reproduce:

# in opentimestamps-client
# running opentimestamps-server on localhost

rm -f release-notes.md.ots && \
  ./ots s -m 1 -c http://localhost:14788 release-notes.md && \
  sleep 0 && \
  ./ots u -c http://localhost:14788 release-notes.md.ots

Changing sleep 0 to sleep 1 yields the expected response.

[petertodd]

Hey, sorry, just noticed I missed this. :(

So, I think the reason why you're seeing that is that Bitcoin timestamping is in a separate Stamper thread. When you submit a digest to be timestamped, what actually happens after the digest is aggregated is the tip of that per-second merkle-tree is written to the "journal" - an append-only file(1) containing all the per-second commitments that the calendar has promised to timestamp.

But currently the calendar doesn't directly communicate with the stamper. Rather, the stamper polls the file periodically to check for new commitments.

So when you try to get the timestamp immediately after submitting it, your get ends up at the is_pending() function, which returns "Not Found" because the stamper thread hasn't read it yet. Even worse, the production calendar servers are behind nginx, which caches "Not Found" responses, resulting in a false "Not Found" being returned even after the stamper recovers.

tl;dr: I think you just found an important bug. :) Probably the right fix here is to make the stamper part of the aggregation process, rather than a separate thread, but I need to think a bit about how that interacts with Bitcoin RPC - I've also ran into some issues with poor handling of RPC timeouts there that I need to think about.

1. Specifically, we actually set the append-only file attribute on the journal file itself, preventing the user that the OTS server process runs under from deleting anything already written to that file.