Allow specifying a last round
ldruschk opened this issue · 18 comments
This way you don't have to time shutting down the Engine perfectly while still calculating points one last time at the end.
You don't have to shut it down while calculating points (you can't unless you kill the process), you just have to shut it down after it has finished calculating the round you want to be the last. Nevertheless we could allow specifying the last round, either by command line argument or ctf.json.
This should also be added to the checker protocol specification, so that checkers can adjust their behavior based on the progress of the CTF without configuring them with hard-coded round values
Can't they just use the current time of day? Checker writers usually know when the CTF will start
But then you could no longer reuse challenges in later CTFs without requiring modifications to the checker. The advantage of including it in the checker protocol is that the configuration would be centralized on the engine.
I would suggest using endTime over lastRound as there still is some slight drift in the round lengths (right?) and the end time may still be in the middle of a round - in which case at least flag submission should be closed.
Probably the round leading up to the end time should no longer distribute flags?
Sounds good. What broke the last round of ENOWARS4 was that the network was closed at the specified end time in the middle of the round leading to most checker runs being broken. The alternative would have been stopping one round earlier and thus closing the flag submission for the last 1-2 minutes.
Why would you bother with closing the flag submission? If the round is not finished the submitted flags won't count. I didn't understand why you closed the network at all, just turning off the engine after the round calculation would have been sufficient.
If the round is not finished the submitted flags won't count.
And exactly that is the problem. Just because the start time of the engine drifts over the duration of the CTF should not mean that flags submitted in the last 1-2 minutes are not counted. However, it should not be possible to exploit other teams after the end of the CTF and the checker should no longer be able to place new flags/deduct points for mumble/offline services, hence close the flag submission at the end of the CTF but no longer spawn checker tasks in the round which would be incomplete.
And exactly that is the problem. Just because the start time of the engine drifts over the duration of the CTF should not mean that flags submitted in the last 1-2 minutes are not counted.
So you would count the submitted flags of an incomplete round, but not the service health? That is really arbitrary, isn't it? Teams will have an advantage if they just turn their services off in that round. Also, due to the random scheduling some teams will have more flags that can be stolen than others. Why do you take something that is inherently round-based and try to convert it into a time-based thing?
However, it should not be possible to exploit other teams after the end of the CTF and the checker should no longer be able to place new flags/deduct points for mumble/offline services, hence close the flag submission at the end of the CTF
This is exactly the status quo: the ctf ends with the computation of the last round, and everything that happens before is counted, everything that happens after is not counted.
Why do you take something that is inherently round-based and try to convert it into a time-based thing?
Because the duration of a CTF is usually specified as a start and end time and not a start time and a number of rounds which might end at roughly the end time but maybe earlier or later depending on the number of flags submitted.
Is there any reason why a round is delayed based on the time it takes to calculate the points? That seems like something which can be done in parallel, or does any of the tasks being created in a new round depend on the score of the earlier roudns?
Because the duration of a CTF is usually specified as a start and end time and not a start time and a number of rounds which might end at roughly the end time but maybe earlier or later depending on the number of flags submitted.
So do most sports events, but in most (all?) round-based sports humans have decreed that the referee has the final say. Unless the round calculation is under extreme load, the drift should be minimal. Are other A/D ctfs really adhering to the spec? Ructfe always has their flagsub running for a few minutes after the ctf ends iirc, and if I understood faust's code correctly they also don't care about half rounds.
Is there any reason why a round is delayed based on the time it takes to calculate the points?
It is not, the calculation is started after all checkertask insertions are finished to keep the accumulated delay at a minimum. Take a look:
EnoDatabase=# select * from "Rounds" LIMIT 10;
Id | Begin | Quarter2 | Quarter3 | Quarter4 | End
----+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------
1 | 2020-07-18 13:00:57.059319 | 2020-07-18 13:01:42.059319 | 2020-07-18 13:02:27.059319 | 2020-07-18 13:03:12.059319 | 2020-07-18 13:03:57.059319
2 | 2020-07-18 13:03:57.070074 | 2020-07-18 13:04:42.070074 | 2020-07-18 13:05:27.070074 | 2020-07-18 13:06:12.070074 | 2020-07-18 13:06:57.070074
3 | 2020-07-18 13:06:57.074796 | 2020-07-18 13:07:42.074796 | 2020-07-18 13:08:27.074796 | 2020-07-18 13:09:12.074796 | 2020-07-18 13:09:57.074796
4 | 2020-07-18 13:09:57.073359 | 2020-07-18 13:10:42.073359 | 2020-07-18 13:11:27.073359 | 2020-07-18 13:12:12.073359 | 2020-07-18 13:12:57.073359
5 | 2020-07-18 13:12:57.073402 | 2020-07-18 13:13:42.073402 | 2020-07-18 13:14:27.073402 | 2020-07-18 13:15:12.073402 | 2020-07-18 13:15:57.073402
6 | 2020-07-18 13:15:57.073892 | 2020-07-18 13:16:42.073892 | 2020-07-18 13:17:27.073892 | 2020-07-18 13:18:12.073892 | 2020-07-18 13:18:57.073892
7 | 2020-07-18 13:18:57.076904 | 2020-07-18 13:19:42.076904 | 2020-07-18 13:20:27.076904 | 2020-07-18 13:21:12.076904 | 2020-07-18 13:21:57.076904
8 | 2020-07-18 13:21:57.076429 | 2020-07-18 13:22:42.076429 | 2020-07-18 13:23:27.076429 | 2020-07-18 13:24:12.076429 | 2020-07-18 13:24:57.076429
9 | 2020-07-18 13:24:57.077086 | 2020-07-18 13:25:42.077086 | 2020-07-18 13:26:27.077086 | 2020-07-18 13:27:12.077086 | 2020-07-18 13:27:57.077086
10 | 2020-07-18 13:27:57.081298 | 2020-07-18 13:28:42.081298 | 2020-07-18 13:29:27.081298 | 2020-07-18 13:30:12.081298 | 2020-07-18 13:30:57.081298
(10 rows)
EnoDatabase=# select * from "Rounds" ORDER BY "Id" DESC LIMIT 10;
Id | Begin | Quarter2 | Quarter3 | Quarter4 | End
-----+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------
161 | 2020-07-18 21:01:07.200526 | 2020-07-18 21:01:52.200526 | 2020-07-18 21:02:37.200526 | 2020-07-18 21:03:22.200526 | 2020-07-18 21:04:07.200526
160 | 2020-07-18 20:58:07.204296 | 2020-07-18 20:58:52.204296 | 2020-07-18 20:59:37.204296 | 2020-07-18 21:00:22.204296 | 2020-07-18 21:01:07.204296
159 | 2020-07-18 20:55:07.201861 | 2020-07-18 20:55:52.201861 | 2020-07-18 20:56:37.201861 | 2020-07-18 20:57:22.201861 | 2020-07-18 20:58:07.201861
158 | 2020-07-18 20:52:07.201016 | 2020-07-18 20:52:52.201016 | 2020-07-18 20:53:37.201016 | 2020-07-18 20:54:22.201016 | 2020-07-18 20:55:07.201016
157 | 2020-07-18 20:49:07.204557 | 2020-07-18 20:49:52.204557 | 2020-07-18 20:50:37.204557 | 2020-07-18 20:51:22.204557 | 2020-07-18 20:52:07.204557
156 | 2020-07-18 20:46:07.200604 | 2020-07-18 20:46:52.200604 | 2020-07-18 20:47:37.200604 | 2020-07-18 20:48:22.200604 | 2020-07-18 20:49:07.200604
155 | 2020-07-18 20:43:07.201349 | 2020-07-18 20:43:52.201349 | 2020-07-18 20:44:37.201349 | 2020-07-18 20:45:22.201349 | 2020-07-18 20:46:07.201349
154 | 2020-07-18 20:40:07.196561 | 2020-07-18 20:40:52.196561 | 2020-07-18 20:41:37.196561 | 2020-07-18 20:42:22.196561 | 2020-07-18 20:43:07.196561
153 | 2020-07-18 20:37:07.196285 | 2020-07-18 20:37:52.196285 | 2020-07-18 20:38:37.196285 | 2020-07-18 20:39:22.196285 | 2020-07-18 20:40:07.196285
152 | 2020-07-18 20:34:07.196546 | 2020-07-18 20:34:52.196546 | 2020-07-18 20:35:37.196546 | 2020-07-18 20:36:22.196546 | 2020-07-18 20:37:07.196546
(10 rows)
Now let's calculate the drift:
C:\Users\Benni>python
Python 3.8.4 (tags/v3.8.4:dfa645a, Jul 13 2020, 16:46:45) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> from datetime import timedelta
>>> t = datetime.strptime("13:00:57.059319", '%H:%M:%S.%f')
>>> t
datetime.datetime(1900, 1, 1, 13, 0, 57, 59319)
>>> t + timedelta(minutes=(159*3))
datetime.datetime(1900, 1, 1, 20, 57, 57, 59319)
So round 159 was "supposed" to end at 22:57:57,59319 but did end at 20:58:07.201861. We have accumulated a drift of ~10 seconds, but insert peter griffin meme
So what you are saying is we really want to be able to schedule an exact start time?
If you consider starting closer to the planned start time a key performance indicator, yes. Starting comes with some delay (inserting teams, service stats, services, asking checkers how many flags they want, ...), but iirc the majority of the delay this year was caused by not having the ctf.json there in time and not starting the engine in time, which you won't solve by adding more features to the engine 😛
I don't think the majority of our players cares. Games aren't over when the clock says, but when the referee says.
I think players expect it to be over at a rather exact tim. In irc they already posted gg before we shut it off.
Starting. ..not so much.
You could even go one step further and only specify the end time and number of rounds and let the engine figure out the round length itself. This way delays at the beginning would be compensated by shortening all rounds by an equal amount, meaning this will go mostly unnoticed to users. Unless you have significant breaks during the CTF (in which case you might want to adjust the number of rounds/end time manually) you could even compensate those breaks if they are not too close to the end.
The question is is whether that is desirable. I believe users are more likely to notice time differences at the end than a slight deviation from the announced round time.
The question is is whether that is desirable. I believe users are more likely to notice time differences at the end than a slight deviation from the announced round time.
An exact round length will only ever matter for advanced tools like exploiters/throwers, right?
BTW having an end time set allows us to do fancy visualizations like visual timers or YouTube videos of final countdown that appear just at the right time 🙃
I think players expect it to be over at a rather exact tim.
Does any other A/D ctf end in the middle of a round?
The fact that we all probably don't know is proof that it probably doesn't matter :D