[Presented at the 2016 IGGI Symposium, held at the University of Goldsmiths] Monte Carlo Tree Search is a stochastic tree search technique that has seen much interest within academia due to its strength of play in card and board games. Despite this interest, it has rarely been used in commercial games. In this talk I describe how MCTS works, suggest some of the issues facing its application to commercial games, and suggest some new areas of research that could address these challenges.

1. Challenges for implementing Monte Carlo Tree
Search in commercial games
Matthew Bedder
16/Sept/2016 1

2. 16/Sept/2016 2
Disclaimer
This is pitched at people with little/no knowledge
of the area
(Sorry, AI people)

3. “A game is a series of
interesting choices”
16/Sept/2016 3

4. 16/Sept/2016 4

5. Monte Carlo Tree Search
16/Sept/2016 5

6. 16/Sept/2016 6
Monte Carlo Tree Search
Only generate the bits of the game tree we need

7. 16/Sept/2016 7
MCTS – the basics
Iteratively builds a decision tree

8. 16/Sept/2016 8
MCTS Selection

9. 16/Sept/2016 9
MCTS Expansion

10. 16/Sept/2016 10
MCTS Simulation

11. 16/Sept/2016 11
Monte Carlo evaluations
Performing random actions gives us a (weak)
approximation of state values
X O X O
X
O
X
O
X

12. 16/Sept/2016 12
MCTS Simulation

13. 16/Sept/2016 13
MCTS Backpropagation

14. 16/Sept/2016 14
MCTS – the basics

15. Applications of MCTS
16/Sept/2016 15

16. 16/Sept/2016 16
MCTS applications
• Amazons
• Arimaa
• Connect 4
• Dou Di Zhu
• Go variants
• Havannah
• Hex
• Morpion Solitaire
• Othello
• Settlers of Catan
• Spades
• ...
Many board and card games, including:

17. 16/Sept/2016 17

18. 16/Sept/2016 18
MCTS applications
All of the strongest AI
players of Go use
variants of MCTS

19. 16/Sept/2016 19
MCTS applications
Currently there are very limited applications to
commercial video games or other domains

20. 16/Sept/2016 20
http://www.aifactory.co.uk/AIF_Games_Spades.htm

21. 16/Sept/2016 21
MCTS Successes
http://aigamedev.com/open/coverage/mcts-rome-ii/

22. 16/Sept/2016 22
http://gwaredd.github.io/nuclai_mcts

23. Challenges for MCTS
16/Sept/2016 23

24. 16/Sept/2016 24
Games are hard
There are as many states of Civilization II as
there are atoms in the observable universe
10,000,000,000,000,000,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000
times over

25. 16/Sept/2016 25
Computational cost
This is actually important
Game AIs (usually) are limited to time sliced CPU
AlphaGo used
• 1,920 CPUs and 280 GPUs
• ~1MW

26. 16/Sept/2016 26
Monte Carlo Tree Search optimisations
Generally either:
Domain specific
or
Overly generic

27. 16/Sept/2016 27
Monte Carlo Tree Search optimisations
Replacing random simulations helps
… usually
Adding prior knowledge helps
… usually

28. 16/Sept/2016 28
Our approach
Domain-specific optimisations for arbitrary
domains
I’d love to talk about our work during/after coffee

29. www-users.cs.york.ac.uk/~bedder
matthew@bedder.co.uk
@bedder
Matthew Bedder