Experiments in using ai agent world model development using energy based models. Learning as I go but will add in some interesting findings here.
I am not a PHd or any student (dropped out of CS undergrad) but been doing software development since 2005, reading research papers and did some early ML CV stuff
After listening to the use of EBMs with world base models from this podcast I have decided to try it.
Mostly pulling from the foundation paper I am switching out the MDN step and adding in a EBM.
So would like
# old model
Environment -> ( VAE ) -> (EBM-LTSM) -> Action -> Environment
# current model 03202024
Environment -> AlexNet -> Energy via NCE loss -> LLM -> Action/Description Text -> Environment
Lets see if a regular dude like me can pull it off.
More details coming soon but for now user requirements.txt
Need to have tesseract-ocr
Come chat with me and other developers exploring the AI agent space https://discord.gg/UWd6u5aR