User:Wilkens Exavier/Artificial intelligence in video games

Assignment 6 - The Monte Carlo Tree Search Method
Artificial intelligence in video games initially started out with two components that made up their behaviors, pathfinding and finite state machines. In pathfinding, it's how the AI can get from point A to point B in the most direct way possible, whereas finite state machines can move between different states at a given moment in time. While this has been the common components to the behavior of video game AI's for the past several decades, development in AI decision making and behavior had gone from simple to complex with the development of the Monte Carlo tree search method. The MCTS method provides a more complex decision making and behavior process that video game AI's undergo to provide a more a complex way of engaging the human player. The AI will now come up with more ways to create more obstacles for the human player to overcome in order to progress. The MCTS consists of a tree diagram in which the AI essentially plays tic-tac-toe to which, depending on the outcome, will follow a select pathway in order to come to the next obstacle for the player, or the next tic-tac-toe board. The tree, specifically in video games that are more complex, contain more branches and possibilities are numerous, providing the player to come up with several strategies to figure out how they can surpass the obstacle and move on to the next level of the game.

The Monte Carlo tree search method consists of four phases:


 * 1) The information that exists within the nodes follow each node down to the end of the search tree.
 * 2) The search tree creates a new node.
 * 3) A simulation occurs in which a winner is determined.
 * 4) The nodes that follow the selected path are updated with new information from the simulation.

This process repeats throughout the process of the game in order to determine each and every possible outcome in an obstacle. Each node can create several paths, both similar and different, in order to provide the AI with several different ways in order to engage the player and create an immersive experience within the game. This method not only involves creating new paths, but also justifying which paths are the right paths to take. The end all purpose is to create new paths, but also exploiting proven paths that end positively for the player. The method of MCTS is combined with Upper Confidence Bound 1, or UCB1, to create an Upper Confidence Bound 1applied to trees, called UCT. The process algorithm adds the exploitation term (wᵢ / sᵢ) to the exploration term, (sqrt(ln sₚ / sᵢ) that results in the UCT, which determines the selection for the best possible path of nodes that the AI can anticipate. The numbers that i represents in the exploitation term represents the the number of situation that ended in a win added to the total number of simulations.