SlideShare a Scribd company logo

Application of Monte Carlo Tree Search in a Fighting Game AI (GCCE 2016)

•Download as PPTX, PDF•
•
F
ftgaic

This are the presentation slides for the following paper: Shubu Yoshida, Makoto Ishihara, Taichi Miyazaki, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas, "Application of Monte-Carlo Tree Search in a Fighting Game AI," accepted for presentation at the 5th IEEE Global Conference on Consumer Electronics (GCCE 2016), Kyoto, Japan, Oct. 11-14, 2016.

Technology
1 of 22
Application of Monte- Carlo Tree Search in a Fighting Game AI Shubu Yoshida, Makoto Ishihara, Taichi Miyazaki, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas Intelligent Computer Entertainment Laboratory Ritsumeikan University
Outline 1.Background of this research 2.Monte-Carlo Tree Search 3.Monte-Carlo Tree Search for a Fighting Game 4.Experimental Environment 5.Experimental Method 6.Result 7.Competition result in 2016 8.Conclusion
Background (1/2) A Fighting Game AI Competition is held every year [1] High-ranking AIs = Rule-based (until 2015) Rule-based : a same action in a same situation Human player can easily predict the AI’s action patterns and outsmart it [1] http://www.ice.ci.ritsumei.ac.jp/~ftgaic/
Background (2/2)  Apply the Monte-Carlo Tree Search (MCTS) to a fighting game AI  Decides a next own action by stochastic simulations  Already successful in many games [2][3] We evaluate the effectiveness of MCTS on a fighting game [2] S. Gelly, et al. ”The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions”, Communications of the ACM, Vol. 55, No. 3, pp. 106-113, 2012. [3] N. Ikehata and T. Ito. ”Monte-carlo tree search in ms. pac-man”. In Computational Intelligence and Games (CIG), 2011 IEEE Conference on, pp. 39-46, 2011
Monte-Carlo Tree Search (1/5) selection simulation backpropagation repeat until the set time has elapsed expansion
Monte-Carlo Tree Search (2/5) selection simulation backpropagation repeat until the set time has elapsed expansion
Formula of UCB1 ・ 𝑋𝑖 : the value of an average reward ・𝐶 : The balance parameter ・𝑁𝑖 𝑝 : The total number of times the parent node of node 𝑖 has been visited ・𝑁𝑖 : The total number of times node 𝑖 has been visited 𝑈𝐶𝐵1𝑖 = 𝑋𝑖 + 𝐶 2 ln 𝑁𝑖 𝑝 𝑁𝑖 Preferentially select a child node that has been visited less The evaluation valueExploitation Exploration
Monte-Carlo Tree Search (3/5) selection simulation backpropagation repeat until the set time has elapsed expansion
Monte-Carlo Tree Search (4/5) selection simulation backpropagation repeat until the set time has elapsed expansion
Monte-Carlo Tree Search (5/5) selection simulation backpropagation repeat until the set time has elapsed expansion
MCTS for a Fighting Game (1/2) 𝑈𝐶𝐵1𝑖 = 𝑋𝑖 + 𝐶 2 ln 𝑁𝑖 𝑝 𝑁𝑖 𝑋𝑖 = 1 𝑁𝑖 𝑗=1 𝑁 𝑖 𝑒𝑣𝑎𝑙𝑗 𝑒𝑣𝑎𝑙𝑗 = (𝑎𝑓𝑡𝑒𝑟𝐻𝑃𝑗 𝑚𝑦 − 𝑏𝑒𝑓𝑜𝑟𝑒𝐻𝑃𝑗 𝑚𝑦 ) −(𝑎𝑓𝑡𝑒𝑟𝐻𝑃𝑗 𝑜𝑝𝑝 − 𝑏𝑒𝑓𝑜𝑟𝑒𝐻𝑃𝑗 𝑜𝑝𝑝 )
MCTS for a Fighting Game (2/2) ・・・ Expansion normal fighting game ・・・ ・・・・・ Simulation
Experimental Environment FightingICE Used as the platform of international fighting game AI competition 1 game : 3 rounds －1 round : 60 second 𝑚𝑦𝑆𝑐𝑜𝑟𝑒 = 𝑜𝑝𝑝𝐻𝑃 𝑚𝑦𝐻𝑃+𝑜𝑝𝑝𝐻𝑃 × 1000 Response time : 16.67ms
Experimental Method MCTSAI(AI applying MCTS) vs high ranking 5 AIs of 2015 tournament 5 AIs : Rule-based 100 games (50 games each side) TABLE I THE PARAMETERS USED IN THE EXPERIMENTS Notations Meanings Values C Balance parameter 3 Threshold of the number of visits 10 Threshold of the depth of tree 2 The number of simulations 60 frames 𝑁 𝑚𝑎𝑥 𝐷 𝑚𝑎𝑥 𝑇𝑠𝑖𝑚
Result (1/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
Result (2/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
Result (3/5) P1 : MCTSAI P2 : RatioBot
Result (4/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
Result (5/5) P1 : MCTSAI P2 : Machete
Competition result in 2016 Orange 1st Blue 2nd Green 3rd Total Rank RANK BANZAI 11 DragonSurvivor 12 iaTest 7 IchibanChan 9 JayBot2016 5 KeepYourDistanceBot 10 MctsAi 3 MrAsh 4 Poring 8 Ranezi 2 Snorkel 13 Thunder01 1 Tomatensimulator 6 Triump 14
Conclusion Applied MCTS to fighting game AI Showed that MCTS in fighting game AI is effective Future work In fighting game, random simulation of the enemy behavior is not effective Predict the behavior of the enemy and use this information in simulation
Thank you for listening

Recommended

Deep Reinforcement Learning: Q-Learning
Deep Reinforcement Learning: Q-LearningDeep Reinforcement Learning: Q-Learning
Deep Reinforcement Learning: Q-LearningKai-Wen Zhao
 
Deep Reinforcement Learning
Deep Reinforcement LearningDeep Reinforcement Learning
Deep Reinforcement LearningUsman Qayyum
 
Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017mooopan
 
An introduction to deep reinforcement learning
An introduction to deep reinforcement learningAn introduction to deep reinforcement learning
An introduction to deep reinforcement learningBig Data Colombia
 
Q-learning and Deep Q Network (Reinforcement Learning)
Q-learning and Deep Q Network (Reinforcement Learning)Q-learning and Deep Q Network (Reinforcement Learning)
Q-learning and Deep Q Network (Reinforcement Learning)Thom Lane
 
Reinforcement Learning Q-Learning
Reinforcement Learning Q-Learning Reinforcement Learning Q-Learning
Reinforcement Learning Q-Learning Melaku Eneayehu
 
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex Fridman
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex FridmanMIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex Fridman
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex FridmanPeerasak C.
 
Multi-Agent Reinforcement Learning
Multi-Agent Reinforcement LearningMulti-Agent Reinforcement Learning
Multi-Agent Reinforcement LearningSeolhokim
 

Recommended

Deep Reinforcement Learning: Q-Learning
Deep Reinforcement Learning: Q-LearningDeep Reinforcement Learning: Q-Learning
Deep Reinforcement Learning: Q-LearningKai-Wen Zhao
 
Deep Reinforcement Learning
Deep Reinforcement LearningDeep Reinforcement Learning
Deep Reinforcement LearningUsman Qayyum
 
Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017mooopan
 
An introduction to deep reinforcement learning
An introduction to deep reinforcement learningAn introduction to deep reinforcement learning
An introduction to deep reinforcement learningBig Data Colombia
 
Q-learning and Deep Q Network (Reinforcement Learning)
Q-learning and Deep Q Network (Reinforcement Learning)Q-learning and Deep Q Network (Reinforcement Learning)
Q-learning and Deep Q Network (Reinforcement Learning)Thom Lane
 
Reinforcement Learning Q-Learning
Reinforcement Learning Q-Learning Reinforcement Learning Q-Learning
Reinforcement Learning Q-Learning Melaku Eneayehu
 
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex Fridman
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex FridmanMIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex Fridman
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) by Lex FridmanPeerasak C.
 
Multi-Agent Reinforcement Learning
Multi-Agent Reinforcement LearningMulti-Agent Reinforcement Learning
Multi-Agent Reinforcement LearningSeolhokim
 
Lecture 9 Markov decision process
Lecture 9 Markov decision processLecture 9 Markov decision process
Lecture 9 Markov decision processVARUN KUMAR
 
Combinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learningCombinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learning민재 정
 
Deep Reinforcement Learning
Deep Reinforcement LearningDeep Reinforcement Learning
Deep Reinforcement LearningMeetupDataScienceRoma
 
Imitation Learning for Autonomous Driving in TORCS
Imitation Learning for Autonomous Driving in TORCSImitation Learning for Autonomous Driving in TORCS
Imitation Learning for Autonomous Driving in TORCSPreferred Networks
 
Intro to Reinforcement learning - part III
Intro to Reinforcement learning - part IIIIntro to Reinforcement learning - part III
Intro to Reinforcement learning - part IIIMikko Mäkipää
 
Lec3 dqn
Lec3 dqnLec3 dqn
Lec3 dqnRonald Teo
 
Imitation learning tutorial
Imitation learning tutorialImitation learning tutorial
Imitation learning tutorialYisong Yue
 
Facebook Talk at Netflix ML Platform meetup Sep 2019
Facebook Talk at Netflix ML Platform meetup Sep 2019Facebook Talk at Netflix ML Platform meetup Sep 2019
Facebook Talk at Netflix ML Platform meetup Sep 2019Faisal Siddiqi
 
Efficient Neural Architecture Search via Parameter Sharing
Efficient Neural Architecture Search via Parameter SharingEfficient Neural Architecture Search via Parameter Sharing
Efficient Neural Architecture Search via Parameter SharingJinwon Lee
 
Deep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsDeep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsBill Liu
 
An introduction to reinforcement learning
An introduction to reinforcement learningAn introduction to reinforcement learning
An introduction to reinforcement learningSubrat Panda, PhD
 
Contextual Bandit Survey
Contextual Bandit SurveyContextual Bandit Survey
Contextual Bandit SurveySangwoo Mo
 
Deep Q-learning explained
Deep Q-learning explainedDeep Q-learning explained
Deep Q-learning explainedazzeddine chenine
 
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement LearningSalem-Kabbani
 
Deep Multi-agent Reinforcement Learning
Deep Multi-agent Reinforcement LearningDeep Multi-agent Reinforcement Learning
Deep Multi-agent Reinforcement Learningdeawoo Kim
 
Dynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris GameDynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris GameSuelen Carvalho
 
Multi-Armed Bandit and Applications
Multi-Armed Bandit and ApplicationsMulti-Armed Bandit and Applications
Multi-Armed Bandit and ApplicationsSangwoo Mo
 
Multi-armed Bandits
Multi-armed BanditsMulti-armed Bandits
Multi-armed BanditsDongmin Lee
 
LinkedIn talk at Netflix ML Platform meetup Sep 2019
LinkedIn talk at Netflix ML Platform meetup Sep 2019LinkedIn talk at Netflix ML Platform meetup Sep 2019
LinkedIn talk at Netflix ML Platform meetup Sep 2019Faisal Siddiqi
 
Deep sarsa, Deep Q-learning, DQN
Deep sarsa, Deep Q-learning, DQNDeep sarsa, Deep Q-learning, DQN
Deep sarsa, Deep Q-learning, DQNEuijin Jeong
 
Mcts ai
Mcts aiMcts ai
Mcts aiftgaic
 
"Monte-Carlo Tree Search for the game of Go"
"Monte-Carlo Tree Search for the game of Go""Monte-Carlo Tree Search for the game of Go"
"Monte-Carlo Tree Search for the game of Go"BigMC
 

More Related Content

What's hot

Lecture 9 Markov decision process
Lecture 9 Markov decision processLecture 9 Markov decision process
Lecture 9 Markov decision processVARUN KUMAR
 
Combinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learningCombinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learning민재 정
 
Imitation Learning for Autonomous Driving in TORCS
Imitation Learning for Autonomous Driving in TORCSImitation Learning for Autonomous Driving in TORCS
Imitation Learning for Autonomous Driving in TORCSPreferred Networks
 
Intro to Reinforcement learning - part III
Intro to Reinforcement learning - part IIIIntro to Reinforcement learning - part III
Intro to Reinforcement learning - part IIIMikko Mäkipää
 
Imitation learning tutorial
Imitation learning tutorialImitation learning tutorial
Imitation learning tutorialYisong Yue
 
Facebook Talk at Netflix ML Platform meetup Sep 2019
Facebook Talk at Netflix ML Platform meetup Sep 2019Facebook Talk at Netflix ML Platform meetup Sep 2019
Facebook Talk at Netflix ML Platform meetup Sep 2019Faisal Siddiqi
 
Efficient Neural Architecture Search via Parameter Sharing
Efficient Neural Architecture Search via Parameter SharingEfficient Neural Architecture Search via Parameter Sharing
Efficient Neural Architecture Search via Parameter SharingJinwon Lee
 
Deep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsDeep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsBill Liu
 
An introduction to reinforcement learning
An introduction to reinforcement learningAn introduction to reinforcement learning
An introduction to reinforcement learningSubrat Panda, PhD
 
Contextual Bandit Survey
Contextual Bandit SurveyContextual Bandit Survey
Contextual Bandit SurveySangwoo Mo
 
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement LearningSalem-Kabbani
 
Deep Multi-agent Reinforcement Learning
Deep Multi-agent Reinforcement LearningDeep Multi-agent Reinforcement Learning
Deep Multi-agent Reinforcement Learningdeawoo Kim
 
Dynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris GameDynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris GameSuelen Carvalho
 
Multi-Armed Bandit and Applications
Multi-Armed Bandit and ApplicationsMulti-Armed Bandit and Applications
Multi-Armed Bandit and ApplicationsSangwoo Mo
 
Multi-armed Bandits
Multi-armed BanditsMulti-armed Bandits
Multi-armed BanditsDongmin Lee
 
LinkedIn talk at Netflix ML Platform meetup Sep 2019
LinkedIn talk at Netflix ML Platform meetup Sep 2019LinkedIn talk at Netflix ML Platform meetup Sep 2019
LinkedIn talk at Netflix ML Platform meetup Sep 2019Faisal Siddiqi
 
Deep sarsa, Deep Q-learning, DQN
Deep sarsa, Deep Q-learning, DQNDeep sarsa, Deep Q-learning, DQN
Deep sarsa, Deep Q-learning, DQNEuijin Jeong
 

What's hot (20)

Lecture 9 Markov decision process
Lecture 9 Markov decision processLecture 9 Markov decision process
Lecture 9 Markov decision process
 
Combinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learningCombinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learning
 
Deep Reinforcement Learning
Deep Reinforcement LearningDeep Reinforcement Learning
Deep Reinforcement Learning
 
Imitation Learning for Autonomous Driving in TORCS
Imitation Learning for Autonomous Driving in TORCSImitation Learning for Autonomous Driving in TORCS
Imitation Learning for Autonomous Driving in TORCS
 
Intro to Reinforcement learning - part III
Intro to Reinforcement learning - part IIIIntro to Reinforcement learning - part III
Intro to Reinforcement learning - part III
 
Lec3 dqn
Lec3 dqnLec3 dqn
Lec3 dqn
 
Imitation learning tutorial
Imitation learning tutorialImitation learning tutorial
Imitation learning tutorial
 
Facebook Talk at Netflix ML Platform meetup Sep 2019
Facebook Talk at Netflix ML Platform meetup Sep 2019Facebook Talk at Netflix ML Platform meetup Sep 2019
Facebook Talk at Netflix ML Platform meetup Sep 2019
 
Efficient Neural Architecture Search via Parameter Sharing
Efficient Neural Architecture Search via Parameter SharingEfficient Neural Architecture Search via Parameter Sharing
Efficient Neural Architecture Search via Parameter Sharing
 
Deep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsDeep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its Applications
 
An introduction to reinforcement learning
An introduction to reinforcement learningAn introduction to reinforcement learning
An introduction to reinforcement learning
 
Contextual Bandit Survey
Contextual Bandit SurveyContextual Bandit Survey
Contextual Bandit Survey
 
Deep Q-learning explained
Deep Q-learning explainedDeep Q-learning explained
Deep Q-learning explained
 
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement Learning
 
Deep Multi-agent Reinforcement Learning
Deep Multi-agent Reinforcement LearningDeep Multi-agent Reinforcement Learning
Deep Multi-agent Reinforcement Learning
 
Dynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris GameDynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris Game
 
Multi-Armed Bandit and Applications
Multi-Armed Bandit and ApplicationsMulti-Armed Bandit and Applications
Multi-Armed Bandit and Applications
 
Multi-armed Bandits
Multi-armed BanditsMulti-armed Bandits
Multi-armed Bandits
 
LinkedIn talk at Netflix ML Platform meetup Sep 2019
LinkedIn talk at Netflix ML Platform meetup Sep 2019LinkedIn talk at Netflix ML Platform meetup Sep 2019
LinkedIn talk at Netflix ML Platform meetup Sep 2019
 
Deep sarsa, Deep Q-learning, DQN
Deep sarsa, Deep Q-learning, DQNDeep sarsa, Deep Q-learning, DQN
Deep sarsa, Deep Q-learning, DQN
 

Viewers also liked

Mcts ai
Mcts aiMcts ai
Mcts aiftgaic
 
"Monte-Carlo Tree Search for the game of Go"
"Monte-Carlo Tree Search for the game of Go""Monte-Carlo Tree Search for the game of Go"
"Monte-Carlo Tree Search for the game of Go"BigMC
 
Monte Carlo Tree Search for the Super Mario Bros
Monte Carlo Tree Search for the Super Mario BrosMonte Carlo Tree Search for the Super Mario Bros
Monte Carlo Tree Search for the Super Mario BrosChih-Sheng Lin
 
2016 Fighting Game Artificial Intelligence Competition
2016 Fighting Game Artificial Intelligence Competition2016 Fighting Game Artificial Intelligence Competition
2016 Fighting Game Artificial Intelligence Competitionftgaic
 
What did AlphaGo do to beat the strongest human Go player?
What did AlphaGo do to beat the strongest human Go player?What did AlphaGo do to beat the strongest human Go player?
What did AlphaGo do to beat the strongest human Go player?Tobias Pfeiffer
 
Challenges for implementing Monte Carlo Tree Search in commercial games
Challenges for implementing Monte Carlo Tree Search in commercial gamesChallenges for implementing Monte Carlo Tree Search in commercial games
Challenges for implementing Monte Carlo Tree Search in commercial gamesMatthew Bedder
 
Applying fuzzy control in fighting game ai
Applying fuzzy control in fighting game aiApplying fuzzy control in fighting game ai
Applying fuzzy control in fighting game aiftgaic
 
Alpha go 16110226_김영우
Alpha go 16110226_김영우Alpha go 16110226_김영우
Alpha go 16110226_김영우영우 김
 
A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...
A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...
A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...Carlo Lancia
 
2013 Fighting Game Artificial Intelligence Competition
2013 Fighting Game Artificial Intelligence Competition2013 Fighting Game Artificial Intelligence Competition
2013 Fighting Game Artificial Intelligence Competitionftgaic
 
AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...
AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...
AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...Michael Jongho Moon
 
Bayesian statistics using r intro
Bayesian statistics using r introBayesian statistics using r intro
Bayesian statistics using r introBayesLaplace1
 
An introduction to bayesian statistics
An introduction to bayesian statisticsAn introduction to bayesian statistics
An introduction to bayesian statisticsJohn Tyndall
 
Introduction to Bayesian Methods
Introduction to Bayesian MethodsIntroduction to Bayesian Methods
Introduction to Bayesian MethodsCorey Chivers
 
8 queens problem using back tracking
8 queens problem using back tracking8 queens problem using back tracking
8 queens problem using back trackingTech_MX
 
AlphaGo 알고리즘 요약
AlphaGo 알고리즘 요약AlphaGo 알고리즘 요약
AlphaGo 알고리즘 요약Jooyoul Lee
 

Viewers also liked (18)

Mcts ai
Mcts aiMcts ai
Mcts ai
 
"Monte-Carlo Tree Search for the game of Go"
"Monte-Carlo Tree Search for the game of Go""Monte-Carlo Tree Search for the game of Go"
"Monte-Carlo Tree Search for the game of Go"
 
Monte Carlo Tree Search for the Super Mario Bros
Monte Carlo Tree Search for the Super Mario BrosMonte Carlo Tree Search for the Super Mario Bros
Monte Carlo Tree Search for the Super Mario Bros
 
2016 Fighting Game Artificial Intelligence Competition
2016 Fighting Game Artificial Intelligence Competition2016 Fighting Game Artificial Intelligence Competition
2016 Fighting Game Artificial Intelligence Competition
 
What did AlphaGo do to beat the strongest human Go player?
What did AlphaGo do to beat the strongest human Go player?What did AlphaGo do to beat the strongest human Go player?
What did AlphaGo do to beat the strongest human Go player?
 
Challenges for implementing Monte Carlo Tree Search in commercial games
Challenges for implementing Monte Carlo Tree Search in commercial gamesChallenges for implementing Monte Carlo Tree Search in commercial games
Challenges for implementing Monte Carlo Tree Search in commercial games
 
Applying fuzzy control in fighting game ai
Applying fuzzy control in fighting game aiApplying fuzzy control in fighting game ai
Applying fuzzy control in fighting game ai
 
Alpha go 16110226_김영우
Alpha go 16110226_김영우Alpha go 16110226_김영우
Alpha go 16110226_김영우
 
Monte carlo tree search
Monte carlo tree searchMonte carlo tree search
Monte carlo tree search
 
A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...
A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...
A Markov Chain Monte Carlo approach to the Steiner Tree Problem in water netw...
 
2013 Fighting Game Artificial Intelligence Competition
2013 Fighting Game Artificial Intelligence Competition2013 Fighting Game Artificial Intelligence Competition
2013 Fighting Game Artificial Intelligence Competition
 
Bayesian intro
Bayesian introBayesian intro
Bayesian intro
 
AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...
AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...
AlphaGo: An AI Go player based on deep neural networks and monte carlo tree s...
 
Bayesian statistics using r intro
Bayesian statistics using r introBayesian statistics using r intro
Bayesian statistics using r intro
 
An introduction to bayesian statistics
An introduction to bayesian statisticsAn introduction to bayesian statistics
An introduction to bayesian statistics
 
Introduction to Bayesian Methods
Introduction to Bayesian MethodsIntroduction to Bayesian Methods
Introduction to Bayesian Methods
 
8 queens problem using back tracking
8 queens problem using back tracking8 queens problem using back tracking
8 queens problem using back tracking
 
AlphaGo 알고리즘 요약
AlphaGo 알고리즘 요약AlphaGo 알고리즘 요약
AlphaGo 알고리즘 요약
 

Similar to Application of Monte Carlo Tree Search in a Fighting Game AI (GCCE 2016)

2017 Fighting Game AI Competition
2017 Fighting Game AI Competition2017 Fighting Game AI Competition
2017 Fighting Game AI Competitionftgaic
 
Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...
Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...
Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...Deren Lei
 
2015 Fighting Game Artificial Intelligence Competition
2015 Fighting Game Artificial Intelligence Competition2015 Fighting Game Artificial Intelligence Competition
2015 Fighting Game Artificial Intelligence Competitionftgaic
 
AI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptx
AI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptxAI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptx
AI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptxAsst.prof M.Gokilavani
 
IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...
IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...
IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...Anna Guitart Atienza
 
Dissertation defense
Dissertation defenseDissertation defense
Dissertation defenseBen Weber
 
An analysis of minimax search and endgame databases in evolving awale game pl...
An analysis of minimax search and endgame databases in evolving awale game pl...An analysis of minimax search and endgame databases in evolving awale game pl...
An analysis of minimax search and endgame databases in evolving awale game pl...csandit
 
AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...
AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...
AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...cscpconf
 
Introduction to Alphago Zero
Introduction to Alphago ZeroIntroduction to Alphago Zero
Introduction to Alphago ZeroChia-Ching Lin
 
IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...
IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...
IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...Anna Guitart Atienza
 
Handout simulasi computer
Handout simulasi computerHandout simulasi computer
Handout simulasi computerSyafie ALin
 
A STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYER
A STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYERA STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYER
A STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYERijcseit
 
AlphaGo Zero: Mastering the Game of Go Without Human Knowledge
AlphaGo Zero: Mastering the Game of Go Without Human KnowledgeAlphaGo Zero: Mastering the Game of Go Without Human Knowledge
AlphaGo Zero: Mastering the Game of Go Without Human KnowledgeJoonhyung Lee
 
Game Analytics & Machine Learning
Game Analytics & Machine LearningGame Analytics & Machine Learning
Game Analytics & Machine LearningBen Weber
 
How DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of GoHow DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of GoTim Riser
 
Optimizing search-space-of-othello-using-hybrid-approach
Optimizing search-space-of-othello-using-hybrid-approachOptimizing search-space-of-othello-using-hybrid-approach
Optimizing search-space-of-othello-using-hybrid-approachEditor IJMTER
 
Improving the Performance of MCTS-Based μRTS Agents Through Move Pruning
Improving the Performance of MCTS-Based μRTS Agents Through Move PruningImproving the Performance of MCTS-Based μRTS Agents Through Move Pruning
Improving the Performance of MCTS-Based μRTS Agents Through Move PruningAntonio Mora
 

Similar to Application of Monte Carlo Tree Search in a Fighting Game AI (GCCE 2016) (20)

2017 Fighting Game AI Competition
2017 Fighting Game AI Competition2017 Fighting Game AI Competition
2017 Fighting Game AI Competition
 
Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...
Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...
Learning to Reason in Round-based Games: Multi-task Sequence Generation for P...
 
CoSECiVi 2020 - Parametric Action Pre-Selection for MCTS in Real-Time Strateg...
CoSECiVi 2020 - Parametric Action Pre-Selection for MCTS in Real-Time Strateg...CoSECiVi 2020 - Parametric Action Pre-Selection for MCTS in Real-Time Strateg...
CoSECiVi 2020 - Parametric Action Pre-Selection for MCTS in Real-Time Strateg...
 
2015 Fighting Game Artificial Intelligence Competition
2015 Fighting Game Artificial Intelligence Competition2015 Fighting Game Artificial Intelligence Competition
2015 Fighting Game Artificial Intelligence Competition
 
AI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptx
AI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptxAI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptx
AI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptx
 
IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...
IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...
IEEE CIG 2017 New York, Games and Big Data: A Scalable Multi-Dimensional Chur...
 
Dissertation defense
Dissertation defenseDissertation defense
Dissertation defense
 
An analysis of minimax search and endgame databases in evolving awale game pl...
An analysis of minimax search and endgame databases in evolving awale game pl...An analysis of minimax search and endgame databases in evolving awale game pl...
An analysis of minimax search and endgame databases in evolving awale game pl...
 
AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...
AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...
AN ANALYSIS OF MINIMAX SEARCH AND ENDGAME DATABASES IN EVOLVING AWALE GAME PL...
 
Introduction to Alphago Zero
Introduction to Alphago ZeroIntroduction to Alphago Zero
Introduction to Alphago Zero
 
IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...
IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...
IEEE CIG 2018 Maastricht, The Netherlands, A Machine-Learning Item Recommenda...
 
Handout simulasi computer
Handout simulasi computerHandout simulasi computer
Handout simulasi computer
 
A STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYER
A STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYERA STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYER
A STRATEGIC HYBRID TECHNIQUE TO DEVELOP A GAME PLAYER
 
AlphaGo Zero: Mastering the Game of Go Without Human Knowledge
AlphaGo Zero: Mastering the Game of Go Without Human KnowledgeAlphaGo Zero: Mastering the Game of Go Without Human Knowledge
AlphaGo Zero: Mastering the Game of Go Without Human Knowledge
 
Game Analytics & Machine Learning
Game Analytics & Machine LearningGame Analytics & Machine Learning
Game Analytics & Machine Learning
 
How DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of GoHow DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of Go
 
Optimizing search-space-of-othello-using-hybrid-approach
Optimizing search-space-of-othello-using-hybrid-approachOptimizing search-space-of-othello-using-hybrid-approach
Optimizing search-space-of-othello-using-hybrid-approach
 
Improving the Performance of MCTS-Based μRTS Agents Through Move Pruning
Improving the Performance of MCTS-Based μRTS Agents Through Move PruningImproving the Performance of MCTS-Based μRTS Agents Through Move Pruning
Improving the Performance of MCTS-Based μRTS Agents Through Move Pruning
 
AlphaGo and AlphaGo Zero
AlphaGo and AlphaGo ZeroAlphaGo and AlphaGo Zero
AlphaGo and AlphaGo Zero
 
AI_Session 14 Min Max Algorithm.pptx
AI_Session 14 Min Max Algorithm.pptxAI_Session 14 Min Max Algorithm.pptx
AI_Session 14 Min Max Algorithm.pptx
 

More from ftgaic

2021 Fighting Game AI Competition
2021 Fighting Game AI Competition2021 Fighting Game AI Competition
2021 Fighting Game AI Competitionftgaic
 
2020 Fighting Game AI Competition
2020 Fighting Game AI Competition2020 Fighting Game AI Competition
2020 Fighting Game AI Competitionftgaic
 
2019 Fighting Game AI Competition
2019 Fighting Game AI Competition2019 Fighting Game AI Competition
2019 Fighting Game AI Competitionftgaic
 
2018 Fighting Game AI Competition
2018 Fighting Game AI Competition 2018 Fighting Game AI Competition
2018 Fighting Game AI Competition ftgaic
 
Introduction to the Replay File Analysis Tool
Introduction to the Replay File Analysis ToolIntroduction to the Replay File Analysis Tool
Introduction to the Replay File Analysis Toolftgaic
 
2014 Fighting Game Artificial Intelligence Competition
2014 Fighting Game Artificial Intelligence Competition2014 Fighting Game Artificial Intelligence Competition
2014 Fighting Game Artificial Intelligence Competitionftgaic
 

More from ftgaic (6)

2021 Fighting Game AI Competition
2021 Fighting Game AI Competition2021 Fighting Game AI Competition
2021 Fighting Game AI Competition
 
2020 Fighting Game AI Competition
2020 Fighting Game AI Competition2020 Fighting Game AI Competition
2020 Fighting Game AI Competition
 
2019 Fighting Game AI Competition
2019 Fighting Game AI Competition2019 Fighting Game AI Competition
2019 Fighting Game AI Competition
 
2018 Fighting Game AI Competition
2018 Fighting Game AI Competition 2018 Fighting Game AI Competition
2018 Fighting Game AI Competition
 
Introduction to the Replay File Analysis Tool
Introduction to the Replay File Analysis ToolIntroduction to the Replay File Analysis Tool
Introduction to the Replay File Analysis Tool
 
2014 Fighting Game Artificial Intelligence Competition
2014 Fighting Game Artificial Intelligence Competition2014 Fighting Game Artificial Intelligence Competition
2014 Fighting Game Artificial Intelligence Competition
 

Recently uploaded

Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphNeo4j
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentationphoebematthew05
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April Automation LPDGAPIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 

Recently uploaded (20)

Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort ServiceHot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentation
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April Automation LPDGAPIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 

Application of Monte Carlo Tree Search in a Fighting Game AI (GCCE 2016)

  • 1. Application of Monte- Carlo Tree Search in a Fighting Game AI Shubu Yoshida, Makoto Ishihara, Taichi Miyazaki, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas Intelligent Computer Entertainment Laboratory Ritsumeikan University
  • 2. Outline 1.Background of this research 2.Monte-Carlo Tree Search 3.Monte-Carlo Tree Search for a Fighting Game 4.Experimental Environment 5.Experimental Method 6.Result 7.Competition result in 2016 8.Conclusion
  • 3. Background (1/2) A Fighting Game AI Competition is held every year [1] High-ranking AIs = Rule-based (until 2015) Rule-based : a same action in a same situation Human player can easily predict the AI’s action patterns and outsmart it [1] http://www.ice.ci.ritsumei.ac.jp/~ftgaic/
  • 4. Background (2/2)  Apply the Monte-Carlo Tree Search (MCTS) to a fighting game AI  Decides a next own action by stochastic simulations  Already successful in many games [2][3] We evaluate the effectiveness of MCTS on a fighting game [2] S. Gelly, et al. ”The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions”, Communications of the ACM, Vol. 55, No. 3, pp. 106-113, 2012. [3] N. Ikehata and T. Ito. ”Monte-carlo tree search in ms. pac-man”. In Computational Intelligence and Games (CIG), 2011 IEEE Conference on, pp. 39-46, 2011
  • 5. Monte-Carlo Tree Search (1/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  • 6. Monte-Carlo Tree Search (2/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  • 7. Formula of UCB1 ・ 𝑋𝑖 : the value of an average reward ・𝐶 : The balance parameter ・𝑁𝑖 𝑝 : The total number of times the parent node of node 𝑖 has been visited ・𝑁𝑖 : The total number of times node 𝑖 has been visited 𝑈𝐶𝐵1𝑖 = 𝑋𝑖 + 𝐶 2 ln 𝑁𝑖 𝑝 𝑁𝑖 Preferentially select a child node that has been visited less The evaluation valueExploitation Exploration
  • 8. Monte-Carlo Tree Search (3/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  • 9. Monte-Carlo Tree Search (4/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  • 10. Monte-Carlo Tree Search (5/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  • 11. MCTS for a Fighting Game (1/2) 𝑈𝐶𝐵1𝑖 = 𝑋𝑖 + 𝐶 2 ln 𝑁𝑖 𝑝 𝑁𝑖 𝑋𝑖 = 1 𝑁𝑖 𝑗=1 𝑁 𝑖 𝑒𝑣𝑎𝑙𝑗 𝑒𝑣𝑎𝑙𝑗 = (𝑎𝑓𝑡𝑒𝑟𝐻𝑃𝑗 𝑚𝑦 − 𝑏𝑒𝑓𝑜𝑟𝑒𝐻𝑃𝑗 𝑚𝑦 ) −(𝑎𝑓𝑡𝑒𝑟𝐻𝑃𝑗 𝑜𝑝𝑝 − 𝑏𝑒𝑓𝑜𝑟𝑒𝐻𝑃𝑗 𝑜𝑝𝑝 )
  • 12. MCTS for a Fighting Game (2/2) ・・・ Expansion normal fighting game ・・・ ・・・・・ Simulation
  • 13. Experimental Environment FightingICE Used as the platform of international fighting game AI competition 1 game : 3 rounds －1 round : 60 second 𝑚𝑦𝑆𝑐𝑜𝑟𝑒 = 𝑜𝑝𝑝𝐻𝑃 𝑚𝑦𝐻𝑃+𝑜𝑝𝑝𝐻𝑃 × 1000 Response time : 16.67ms
  • 14. Experimental Method MCTSAI(AI applying MCTS) vs high ranking 5 AIs of 2015 tournament 5 AIs : Rule-based 100 games (50 games each side) TABLE I THE PARAMETERS USED IN THE EXPERIMENTS Notations Meanings Values C Balance parameter 3 Threshold of the number of visits 10 Threshold of the depth of tree 2 The number of simulations 60 frames 𝑁 𝑚𝑎𝑥 𝐷 𝑚𝑎𝑥 𝑇𝑠𝑖𝑚
  • 15. Result (1/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
  • 16. Result (2/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
  • 17. Result (3/5) P1 : MCTSAI P2 : RatioBot
  • 18. Result (4/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
  • 19. Result (5/5) P1 : MCTSAI P2 : Machete
  • 20. Competition result in 2016 Orange 1st Blue 2nd Green 3rd Total Rank RANK BANZAI 11 DragonSurvivor 12 iaTest 7 IchibanChan 9 JayBot2016 5 KeepYourDistanceBot 10 MctsAi 3 MrAsh 4 Poring 8 Ranezi 2 Snorkel 13 Thunder01 1 Tomatensimulator 6 Triump 14
  • 21. Conclusion Applied MCTS to fighting game AI Showed that MCTS in fighting game AI is effective Future work In fighting game, random simulation of the enemy behavior is not effective Predict the behavior of the enemy and use this information in simulation
  • 22. Thank you for listening

Editor's Notes

  1. Hello everyone. My name is shubu yoshida of Intelligent Computer Entertainment Lab, Ritsumeikan University. I’d like to talk about “Application of Monte-Carlo Tree Search in a Fighting Game AI” .
  2. This is the outline of my presentation. I’d like to talk about these contents.
  3. A Fighting Game AI Competition is held every year. In this competition, High-ranking AIs are mainly　well-tuned rule-based AIs which always conduct a same action in a same situation. Rule-based AIs take predetermined actions. Human players can easily predict the AI’s action patterns and outsmart it. And if the parameters of the action changed, Rule-based AI’s strength be changed
  4. In order to solve this problem, we apply MCTS to a Fighting Game AI. MCTS decides a next own action by stochastic simulations. MCTS based approach produces a significantly promising result not only in a board game like Go [2], but also in a realtime based game like Ms.Pac-Man [3]. Then, it is expected that it performs better in a fighting game because this kind of game is similar to Ms.Pac-Man in terms of real-time based. it is expected that it performs better in a fighting game. In this paper, we evaluate the effectiveness of MCTS on a fighting game.
  5. We modified traditional MCTS for a fighting game. This figure is an overview of traditional MCTS. I’ll explain you about this. And after having explained this, I’ll explain you about MCTS for fighting game. MCTS combines the game tree search and the Monte Carlo method. Each node represents a state of the game. Each edge an action.
  6. First, MCTS selects the child node with the highest UCB1 value until it reaches a leaf node. Each child node has a UCB1 value.
  7. UCB1 value is calculated by this formula. In this formula, the first term is the evaluation value. The second term aims that MCTS preferentially selects a child node that has been visited less. So, this formula aims that MCTS selects a child node which not only has high evaluation value but also has been visited less to prevent local search. In short, the first term is exploitation and the second term is exploration.
  8. Second, after arriving at a leaf node, if its number of visits exceeds a pre-defined threshold and the depth of the tree has not reached the upper limit, MCTS will create child nodes from it.
  9. Third, it performs random simulation from the root node to the leaf node. And it simulate until the end of game. In this part, opponent actions are selected randomly and my actions are used in that path. After do these actions, we get reward and state.
  10. Finally, it propagates a result of simulation from the leaf node to the parent node and calculates UCB1 values and repeat propagation until the root node. The above 4 steps are repeated during allowed time budget in MCTS. Then, the child node is chosen with the highest number of visits from the root node.
  11. In fighting games, UCB1 is defined by this formula. The evaluation value of node 𝑖 is the average value of the amount of the opponent character's hit-point changes subtracted by the amount of that of the player character . This value is higher when my AI gives a lot of damage to the opponent and it is not damaged by the opponent. Each parameter shows AI HP before and after j-th simulation. The first term is the own score difference term before and after a simulation. The second term is the opponent’s one.
  12. In the expansion part, traditional MCTS expands only one node at a time. In this paper, we expand all actions or nodes that the AI can act. Fighting games have a lot of actions, and real time games have search time limit. We want to explore all of the nodes once at least. So we expand all actions that the AI can act. In the simulation part, in board games, simulation is done until the end of the game. But real-time games have limited thinking time. So we put restrictions on tree depth. These are the main changes in MCTS for fighting games.
  13. In an experiment, We used FightingICE as the fighting game platform. FightingICE is a 2D fighting game developed by our laboratory for game ai researches. It is used as the platform of international fighting game AI competitions recognized by IEEE CIG. The player AI score or My score is calculated by this formula. If more than 500, my AI’s performance is superior to the opponent AI
  14. Next , experimental method. We let MCTSAI fight 100 times against high ranking 5 AIs of 2015 tournament, while switching each side . Action behaviors of each AI are rule-based. And we used these parameters.
  15. The average score against each AI is shown in Fig. 1. In this figure, the horizontal axis lists the name of high ranking AI. And from left to right, there are 1st ranked to 5th ranked Ais. The vertical axis represents the average scores of MCTSAI againtw high ranking Ais.
  16. From this result, the proposed AI outperformed all opponent AIs, except for the 1st ranked AI Machete.
  17. This video is a fighting game scene where P1 is MCTSAI and P2 is RatioBot. RatioBot is the 4th ranked ai in 2015 tournament. As we can see from this video, MCTSAI has been able to dodge the behavior of RatioBot. It can be said that the simulation of Monte Carlo tree search has been working well. So MCTS is an effective method in this fighting game. ////
  18. But the proposed AI did not show a good performance against Machete.
  19. This video is a fighting game scene where P1 is MCTSAI and P2 is Machete. Machete is a well tuned rule-based AI that repeatedly conducts short actions, requiring less number of frames, which are not well simulated by MCTS RANDOM simulation.
  20. This is the competition result in 2016. the horizontal axis lists the name of AI. And these numbers represent these AIs Ranking. In this competition, our MctsAI came 3rd. So it can be said that Mcts showed good results also in an actual tournament.
  21. In conclusion we applied MCTS to a fighting game AI. Results showed that MCTS in fighting game AI is effective. In this paper, we have found that random simulation of the enemy behavior is not effective in fighting games. So, in the future, we plan to add a mechanism such as behavior prediction of the enemy and use it in simulation. Use of this kind of mechanism should better simulate the opponent.