SPIKE Add ability to advance time when network is in devmode
gidonkatten opened this issue · 22 comments
Problem
It is very difficult to test smart contracts that rely on time / block number. The current workaround I use is to add global state variables to smart contracts for the timestamp and block number. Instead of reading from the blockchain, you read them from state instead.
Solution
Solution would be to expose functionality to advance time / block number. Something like goal node advance -t <TIME> -b <BLOCK>. This would only be available when devMode is set to true in the network template JSON file.
Dependencies
None
Urgency
Medium - this would speed up development significantly and avoid the workaround overhead (which may not even be possible when there is limitted storage space + smart contract size).
I think this is really important. As a solidity dev, I can use hardhat to fast forward time, which is super helpful for testing my smart contracts. Having the same feature in dev mode on algorand would be super helpful for my smart contract debugging with pytest, and would enable me to use dev mode when testing my time-gating smart contract functionality
I would bump this up to a high level of urgency. It is critical for testing any smart contracts that rely on time.
Indeed, at Algofi we run a suite of tests that rely on time for calculating lending an borrowing interest. I would second that this should be promoted to a high level of urgency.
This is a huge pain point for us
ideas to solve this problem (needs to be explored)
in devmode:
- remove/relax the constraint about the timestamp of consecutive blocks. (the 20secs max increment)
- add an endpoint for fake time that algod uses. And algod needs to remember the offset for following blocks.
alternatively, have devmode add artificial time each block (pretend that 4 seconds went by). This might be slightly more realistic. But this also requires submitting lots of txns to get time to move forward.
how does this relate to latest timestamp? how does this interact with dryrun and our new simulation endpoint?
are there time mocking libraries in go?
All the smart contracts I write depend on timestamps. Which ends up making my tests last 20-30 minutes. This would save an incredible amount of time
Hi everyone. I created a prototype/POC for this feature and would really appreciate community feedback about the interface.
My proposal used special transaction fields where you can provide hints to the block generation subsystem. Using this system you can instruct the block generator to set a specific timestamp. Since it is part of the transaction you would have to use an SDK to include these fields in your transaction, or send separate transactions for the purpose of modifying state directly.
A PR is available here: #5003
hi, when this issue will be resolved ? any previsions on that ?
@helderjnpinto no predictions on time to complete, but we are actively working on it right now.
We envision a devmode-only algod API that will allow setting timestamp offsets (in seconds) in dev mode block headers. Let's call this v2/devmode/blocks/offset. We currently have two ideas on how to implement this:
1: Set timestamp offsets as deltas between consecutive blocks
You will be able to send requests to control the timestamp deltas between two consecutive blocks:
# Set a timestamp offset of 1000 seconds
> curl -X POST your-algod/v2/devmode/blocks/offset/1000
# Returns OK
> curl your-algod/v2/devmode/blocks/offset # Get current offset
# Returns 1000
# ... send some blocks
# Query block X
> curl your-algod/v2/blocks/X | grep "ts"
"ts": 175589 # time.Now()
# Try the next block, ts delta is 1000
> curl your-algod/v2/blocks/X+1 | grep "ts"
"ts": 176589 # previous ts + 1000
2: Set timestamp offset to the current (real) time
Another approach would be to have the block timestamp to always add the offset to the current time:
curl your-algod/v2/devmode/blocks/offset # Get current offset
# Returns 1000
# ... send some blocks
# Query block X
> curl your-algod/v2/blocks/X | grep "ts"
"ts": 18693868 # time.Now() + offset
# Next block was submitted 1 second after in real time
> curl your-algod/v2/blocks/X+1 | grep "ts"
"ts": 18693869 # previous ts + 1 second in real time + offset
In this case, it would make sense to only allow strictly increasing offsets as decreasing offsets would effectively rewind timestamps. This approach might be better if users want to fast-forward time very sparingly.
We'd love to hear your use cases and opinions on this!
React with ❤️ if you'd like option 1 and react with 🚀 if you'd like option 2!
Just to make sure I'm understanding this correctly, let's say I have an application that is only callable after 1000 seconds have passed since creation (it saves latest timestamp in global state). To test this, I would do the following (assuming option 1, which seems most useful to me).
- Contract creation (block=1, realtime=1, timestamp=1)
- Call contract: FAIL (block=2, realtime=2, timestamp=2)
curl -X POST your-algod/v2/devmode/blocks/offset/1000- Call contract: PASS (block=3, realtime=3, timestamp=1002)
Is that correct?
If so, what would happen if I call the contract again? Would /offset only be applied for that one block or for every block afterwards until it's reset?
- Call conrtract (block=4, realtime=4, timestamp=1003)
OR
- Call contract (block=4, realtime=4, timestamp=2002)
Your first example is correct. For the second one, it would be the latter (offset advances for every block). So you have complete, deterministic control of the timestamp, independent of real time in proposal 1, but the offset is applied to every subsequent block until it is changed.
One caveat is that you can also revert back to real time by setting offset to 0 (for both proposals). Again, this has some consequences if you advanced the block timestamp past current time, and will likely just set the timestamp to the previous timestamp until the real clock catches up. Also, if the real clock is very far ahead of the current timestamp, then block timestamps can only advance by at most MaxTimestampIncrement (config that is by default 25 seconds) per block.
edit: The final implementation of this made it so that setting offset to 0 just freezes the clock, so every subsequent block timestamp will match the previous one, until offset is changed to a non-zero number again.
Great makes perfect sense to me.
At higher levels of abstraction (like the SDKs), it would be useful to have an option for only applying for one block (so not changing the API, but have the SDK set to 1000, send the txn(s), then set back to 0). For example atc.execute(algodClient, 3, 1000)
What about the option to set the timestamp to a specific time? It seems related to this and useful for testing of various contracts. Sometimes deltas are useful but sometimes absolute timestamps would be more natural to work with?
@fergalwalsh that's a good suggestion too. How would it work - you set an arbitrary timestamp, and now that's the time for every subsequent block, until you set it to something else?
How would it work - you set an arbitrary timestamp, and now that's the time for every subsequent block, until you set it to something else?
Probably yes. If you set it to a specific time it means you want explicit control so it it's up to you to advance it further probably. But automatic advancing of some delta per block would probably work too.
Seems like this would be easy to accomplish with the proposed endpoint, assuming the desired timestamp is past the current timestamp. Just call curl -X POST your-algod/v2/devmode/blocks/offset/${Time.now - CURRENT_TIMESTAMP}. Unless you specifically want multiple blocks to have the same timestamp even after the realtime has passed.
Yeah that's a good point @joe-p . Probably no need for the absolute version then.
Seems like this would be easy to accomplish with the proposed endpoint, assuming the desired timestamp is past the current timestamp. Just call
curl -X POST your-algod/v2/devmode/blocks/offset/${Time.now - CURRENT_TIMESTAMP}. Unless you specifically want multiple blocks to have the same timestamp even after the realtime has passed.
The trouble is that there is a race condition, so this does not ensure that the next block will have exactly the timestamp I want, making unit tests difficult (I want to set a timestamp, and then check that my loan interest calculator gets exactly the right value.)
I imagine that after this REST call, the next submitted transaction will have the given offset applied. But if I wait 3 secs to make that transaction, the block will have a timestamp 3 secs later than I had hoped. And, the universe being an annoying place, even if I only wait 0.01 secs, I have a 1 in 100 chance of flipping over the second, and getting a late block.
I'd prefer the opposite approach. Set an exact timestamp, and the offset is an implementation detail, calculated when the next block is generated, so the chain continues forward from there.
I imagine that after this REST call, the next submitted transaction will have the given offset applied. But if I wait 3 secs to make that transaction, the block will have a timestamp 3 secs later than I had hoped. And, the universe being an annoying place, even if I only wait 0.01 secs, I have a 1 in 100 chance of flipping over the second, and getting a late block.
In proposal 1: as soon as the offset is a non-zero value and a block is created, the timestamp is basically "frozen" and determined by previous ts + offset going forward.
So it is possible to make it deterministic, but you'd have to set up the test from the beginning to set the offset to something, create a block, and then set the offset to {Desired timestamp - Current timestamp} to get the desired effect.
Oh, does an empty block get created when the offset is set? That pretty much fixes it. But I still there's a little race condition, because I see the node has time=90 and I want time=100, so I send offset=10, but by the time it arrives, the node is at 91.
Edit: Thanks @winder, sorry I missed that.
Oh, does an empty block get created when the offset is set? That pretty much fixes it. But I still there's a little race condition, because I see the node has time=90 and I want time=10, so I send offset=10, but by the time it arrives, the node is at 91.
I think the proposal is to use an offset from the previous timestamp, not an offset from when the offset was configured. So I don't think there is a timing issue. The only race condition I see is if the node receives an offset config call simultaneously with a txn submission, but there isn't much we can do about that.