Raise Sizes for Flop5Holdem
vishvananda opened this issue · 1 comments
I have been experimenting with the pretrained agents in thsi repository. Using the example code from PokerRL I've loaded a trained agent and tried to play against it:
from PokerRL.game.InteractiveGame import InteractiveGame
from DeepCFR.EvalAgentDeepCFR import EvalAgentDeepCFR
if __name__ == '__main__':
eval_agent = EvalAgentDeepCFR.load_from_disk(
path_to_eval_agent="/home/vishvananda/Deep-CFR/trained_agents/Example_FHP_SINGLE.pkl")
game = InteractiveGame(env_cls=eval_agent.env_bldr.env_cls,
env_args=eval_agent.env_bldr.env_args,
seats_human_plays_list=[0],
eval_agent=eval_agent,
)
game.start_to_play()
I have noticed that the bet/raise sizes in this form are always pot sized. The preflop raise is always to 300 (which would be a pot size raise over the big blind of 100). Post-flop the first bet is 600 (pot sized) and a raise over the 600 will be to 2400 (once again pot sized -- after calling the 600, the pot contains 1800 which is the ammount of the raise). According to the paper from Brown, Flop Hold Em is a limit game which should mean the preflop and flop bet sizes should always be 100 chips. I'm not sure if it was actually trained with pot size raises and it just wasn't mentioned. If it is intended to be limit raising, is there some way to set up the interactive game to use the proper bet sizes?
Here is a log of the interactive game showing the large bet sizing:
$ python3 interactive_user_v_agent.py [40/9667]
_____
_____ _____ |6 |
|2 | _____ |5 || & & |
| & ||3 | _____ | & & || & & | _____
| || & & ||4 || & || & & ||7 |
| & || || & & || & & ||____9|| & & | _____
|____Z|| & || ||____S| |& & &||8 | _____
|____E|| & & | | & & ||& & &||9 |
|____h| |____L|| & & ||& & &|
|& & &||& & &|
|____8||& & &|
|____6|
____________________________________________ TUTORIAL ____________________________________________
Actions:
0 Fold
1 Call
2 Raise according to current fixed limit
****************************
* GAME START *
****************************
___________________________________ preflop - 0 acts ___________________________________
Board:
Last Action: player_None: None None | Main_pot: 0
Player_0:stack: 19950 current_bet: 50 side_pot_rank: -1 hand: Ad, 4s, | Side_pot0: 0
Player_1:stack: 19900 current_bet: 100 side_pot_rank: -1 hand: 2d, As, | Side_pot1: 0
Num raises this round: 1
What action do you want to take as player 0?2
___________________________________ preflop - 1 acts ___________________________________
Board:
Last Action: player_0: 2 300 | Main_pot: 0
Player_0:stack: 19700 current_bet: 300 side_pot_rank: -1 hand: Ad, 4s, | Side_pot0: 0
Player_1:stack: 19900 current_bet: 100 side_pot_rank: -1 hand: 2d, As, | Side_pot1: 0
Num raises this round: 2
___________________________________ flop - 1 acts ___________________________________
Board: Qh, Td, 9s, 7d, Jd,
Last Action: player_1: 1 300 | Main_pot: 600
Player_0:stack: 19700 current_bet: 0 side_pot_rank: -1 hand: Ad, 4s, | Side_pot0: 0
Player_1:stack: 19700 current_bet: 0 side_pot_rank: -1 hand: 2d, As, | Side_pot1: 0
Num raises this round: 0
___________________________________ flop - 0 acts ___________________________________
Board: Qh, Td, 9s, 7d, Jd,
Last Action: player_1: 1 0 | Main_pot: 600
Player_0:stack: 19700 current_bet: 0 side_pot_rank: -1 hand: Ad, 4s, | Side_pot0: 0
Player_1:stack: 19700 current_bet: 0 side_pot_rank: -1 hand: 2d, As, | Side_pot1: 0
Num raises this round: 0
What action do you want to take as player 0?2
___________________________________ flop - 1 acts ___________________________________
Board: Qh, Td, 9s, 7d, Jd,
Last Action: player_0: 2 600 | Main_pot: 600
Player_0:stack: 19100 current_bet: 600 side_pot_rank: -1 hand: Ad, 4s, | Side_pot0: 0
Player_1:stack: 19700 current_bet: 0 side_pot_rank: -1 hand: 2d, As, | Side_pot1: 0
Num raises this round: 1
___________________________________ flop - 1 acts ___________________________________
Board: Qh, Td, 9s, 7d, Jd,
Last Action: player_1: 0 -1 | Main_pot: 0
Player_0:stack: 20300 current_bet: 0 side_pot_rank: -1 hand: Ad, 4s, | Side_pot0: 0
-Player_1:stack: 19700 current_bet: 0 side_pot_rank: -1 hand: 2d, As, | Side_pot1: 0
Num raises this round: 1
Current Winnings per player: [300.0, -300.0]
Hi! Very good point! I think I misinterpreted the game description in the original draft of the Brown et al. paper that was on arxiv when I published this (it wasn't at ICML back then). Luckily, both algorithms in the SD-CFR paper are trained on the same game so it is still fair in terms of evaluation. I will leave the repo as it is, for reproducibility, but will note this change in the conference version of SD-CFR. Thanks!