AI implementation log
Closed this issue · 15 comments
I want to keep a public track of any thoughts I have here as I read the paper and implement pluribus. Admittedly, MCCFR is a new algorithm for me, but will log my thoughts to try and structure this work which should help to plan each next step.
Feedback, notes and criticisms from anyone reading are all very welcome!
I'm relatively confident that the poker engine is largely behaving as it should, and I am ready to add an agent with some degree of intelligence. I have already implemented a random agent, so that will serve as the most basic baseline.
So I've started porting the psuedo code over from the supplimentary material. I'll admit a lot of it doesn't make a lot of sense yet, but I'm hoping things will start to fall into place.
https://github.com/fedden/pluribus-poker-AI/blob/5e9eb0c7e604f9cb11dfa23c412d5d18f1547f8c/pluribus/ai/pluribus.py#L53-L84
I also haven't figured out how this will fit into the poker game engine. Becuase this code traverses a graph of nodes, I think the game engine will need to be much more modular in order to fit in, but we'll get there once the AI side of things is a bit clearer.
Again, if there is anyone reading out there, thoughts are very welcome!
I would suggest using DeepMind's Open Spiel as [MCCFR was implemented there] or copy it over.(https://github.com/deepmind/open_spiel/blob/master/open_spiel/python/algorithms/deep_cfr.py)
Thanks for the comment - yeah looks like a really helpful resource. I'll probably end up using PyTorch, TF2 or Jax (basically something imperative) just to help preserve my sanity during debugging but will definitely reference this!
Added resources on the specific variations of MCCFR in #18
Will start taking a look at what is implemented already tomorrow - I need to go through the pseudo code a bit better myself.
Thanks. I've been on a short hiatus whilst my CTO is in the UK but am starting to take a look again now
@big-c-note Great thanks, let me know if it would help to jump on a brief zoom call, I can run you through the codebase and we could discuss next steps
I've started iterating on AI code in the branch feature/implement-ai
progress？
@Michael-Z keep an eye on this thread this is where we are discussing things and working out next steps. Also will update the ROADMAP in the readme as milestones are met
Hey @fedden , a brief zoom call would be great. I'm in the US (EST). I could do something brief at 12 PM EST today, anytime after 6 PM EST or tomorrow 7:30 AM EST. Thanks!
@big-c-note please reach out to me at leonfedden [at] gmail [dot] com and lets set up something for 12 PM EST today!