Detecting Kingmaker and Minion strats
Kinkelin opened this issue · 5 comments
There has been some discussion in the discord call this morning about how to detect Kingmaker and Minion strats. With the results matrix published (https://drive.google.com/drive/folders/1apW7IKO0nUv8o1Sr_GUJZSPcub8VyPYV), we as the community have the chance to help Cary detect suspicious activity.
As we learned the cheating #1 strat was able to read out opponents and therefore exploit minion strats submitted by others. So I'll start off here, with a list of strategies that the the other top 10 candidates couldn't even score 2 points on avg against, while #1 got > 4.
I created a list with index numbers of potential minion strats in the FMT1_0_1615.npy matrix: (#1 was index number 958 for example). See heatmap below.
These strategies don't nessecarily have to be minions, but the source code should better be checked. If you have any other strats that are suspicious, please add them as well!
I created a heatmap for the potential minions and how they pair against the top 10 strats. Maybe a few false positives, but also some really suspicious candidates. Also looks like all the other top 10 strats are legit and not "king" strats
I found that some of these potential minions had quite a few good scores against them (336 and 971 for example both had >200 strats score >4.5). I also found that 1171 or s1292 was almost definitely a king that these minions were actually intended to help (although the king strat performed quite poorly).
Here's a list of how many >4.5 scores were gotten against each minion candidate (and which strategies scored it if it was small).
134: 1 - (array([958], dtype=int64),)
178: 81 -
237: 3 - (array([ 482, 958, 1171], dtype=int64),)
260: 2 - (array([ 958, 1171], dtype=int64),)
336: 289 -
382: 0 - (array([], dtype=int64),)
389: 0 - (array([], dtype=int64),)
411: 2 - (array([ 958, 1171], dtype=int64),)
417: 2 - (array([ 958, 1171], dtype=int64),)
445: 18 -
452: 20 -
460: 0 - (array([], dtype=int64),)
518: 2 - (array([ 958, 1171], dtype=int64),)
523: 2 - (array([ 958, 1171], dtype=int64),)
538: 2 - (array([ 958, 1171], dtype=int64),)
539: 2 - (array([ 958, 1171], dtype=int64),)
583: 2 - (array([ 958, 1171], dtype=int64),)
585: 2 - (array([ 958, 1171], dtype=int64),)
614: 2 - (array([ 958, 1171], dtype=int64),)
672: 17 -
756: 88 -
812: 0 - (array([], dtype=int64),)
904: 2 - (array([ 958, 1171], dtype=int64),)
952: 276 -
963: 2 - (array([ 958, 1171], dtype=int64),)
964: 0 - (array([], dtype=int64),)
971: 217 -
1079: 41 -
1080: 40 -
1098: 4 - (array([ 422, 578, 958, 1109], dtype=int64),)
1099: 40 -
1100: 40 -
1101: 13 -
1104: 11 -
1125: 41 -
1366: 0 - (array([], dtype=int64),)
1409: 6 -
Strats, that were exploited, but the top ten (including prof x!): couldn't score >= 2 [ 277. 464. 469. 476. 1011. 1399. 1432.]
Strats that were exploited, but daFinnciAgent scored < 2 (in case any top 10 strats were kings, their minions would be here): [ 134. 237. 260. 277. 336. 411. 417. 445. 452. 464. 469. 476.
518. 523. 529. 538. 539. 583. 585. 614. 672. 675. 904. 952.
963. 1011. 1079. 1080. 1098. 1099. 1100. 1101. 1104. 1125. 1152. 1339.
1399. 1432.]
From those, these are what scored >4.5 against.
277: 229 -
464: 1 - (array([687], dtype=int64),)
469: 1 - (array([687], dtype=int64),)
476: 62 -
1011: 4 - (array([ 422, 578, 1109, 1139], dtype=int64),)
1399: 2 - (array([ 748, 1397], dtype=int64),)
1432: 3 - (array([ 748, 1152, 1397], dtype=int64),)
277 and 476 are likely not minions.
687, 748, and 1397 are potential kings, but none are certain. 1397 and 1399 being so close could imply a somebody submitting two back to back though.
Cary has been looking through some of the strategies, and some like 1397 and 1307 have been confirmed to be kingmaker.