I'd love to research practical ways to apply modern machine learning techniques to video games. Video game AI techniques seem stuck in the 90s. Too many games have been limited by poor AI, and I want to fix that.
I'm currently wanting to study bandit methods and tabular reinforcent learning. These are limited, but simple and predictable, which is important for games.
My team is currently leading the gocoder bomberland competition. I wrote a very very long thread in their discord (help channel) about all the RL AI stuff I tried and how things failed and what finally worked in the end. (Some people are participating as part of their university studies, so they asked for help on how to get started).
Even for a seemingly "simple" game like Bomberman, tabular reinforcent learning isn't going to work. I tried it with a huge table with 1000+ states and 1mio+ transitions but it still couldn't capture the complexity of that game. Plus you can mathematically show that the value estimates aren't going to converge, due to exploding variance.
In short, I believe you'll need serious research to go from the current "state of the art" in RL AI to something that is remotely tolerable in a AAA video game. But that sounds like a interesting idea, so maybe you should get your feet wet by building a small RL AI for Bomberman yourself, so that you know how things work. I have replays and instructions for that in the discord too, search keywords "gocoder-bomberland-dataset" and "behavioral cloning".
Current game AI seems to love their decision trees, and I'm thinking more along the lines of how can you mix a few bandits into that tree or how to manually discretize the environment into a small table (not 1000+ states). I don't think games are going to give up their decision trees, but they might be able to mix in some simple machine learning techniques that make the AI somewhat adaptive.
Not sure I'd have the time to work on this, but gonna drop this old idea here anyways.
A long time ago I had the idea for a Swords and Sandals type game (a gladiator game) where each opponent was actually a simulated neural network and you could peak under the hood much like this.[0] Each gladiator would be simulated against each other and would actually be learning as they played against each other. Difficulty for the player wouldn't be based on any setting, but rather just having to face the AI that's learned the most and gotten the best
Anyways, since then I've figured it doesn't really have to be a gladiator game. It could be tic tac toe or connect4 or chess or whatever else
Hi, I'm the author of a C++ library focused on tabular bandits, mdps and pomdps. It's called AI-Toolbox, and it's one of the largest non NN libraries out there.
The library is fully documented, but the text is probably a bit dry. I'd love for somebody to help me improve its accessibility, and I'd be willing to help them along learning how things work.
My email is my nickname and Gmail, feel free to reach out if you are interested.
I'm currently wanting to study bandit methods and tabular reinforcent learning. These are limited, but simple and predictable, which is important for games.