Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self-play
Por um escritor misterioso
Last updated 19 maio 2024
Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.
How to build your own AlphaZero AI using Python and Keras
Policy or Value ? Loss Function and Playing Strength in AlphaZero
Simple Alpha Zero
From-scratch implementation of AlphaZero for Connect4
Value targets in off-policy AlphaZero: a new greedy backup
PDF) Expediting Self-Play Learning in AlphaZero-Style Game-Playing
Adaptive Warm-Start MCTS in AlphaZero-Like Deep Reinforcement
Does the neural net of AlphaZero only evaluate the score of a
AlphaGo: How it works technically?, by Jonathan Hui
Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect
Policy or Value ? Loss Function and Playing Strength in AlphaZero
Mastering Atari, Go, chess and shogi by planning with a learned
Policy or Value ? Loss Function and Playing Strength in AlphaZero
Reimagining Chess with AlphaZero, February 2022
Warm-Start AlphaZero Self-play Search Enhancements
Recomendado para você
-
AlphaZero - Wikipedia19 maio 2024
-
Chess's New Best Player Is A Fearless, Swashbuckling Algorithm19 maio 2024
-
Mastering Atari, Go, chess and shogi by planning with a learned model19 maio 2024
-
Simple Alpha Zero19 maio 2024
-
How AlphaZero Works – Augmented Lawyer19 maio 2024
-
GitHub - yangrc1234/Gomoku-Zero: A gomoku AI based on Alpha Zero paper.19 maio 2024
-
Efficient Learning for AlphaZero via Path Consistency Poster19 maio 2024
-
PDF) AlphaZero-What's Missing?19 maio 2024
-
Mastering chess and shogi by self-play with a general19 maio 2024
-
Free Course: Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess (Paper Explained) from Yannic Kilcher19 maio 2024
você pode gostar
-
Rainbow Push Bubble Animal Brinquedos, Squeeze Stress Reliever19 maio 2024
-
New posts in Level 8+ (Unconfirmed levels) - The Backrooms Community Community on Game Jolt19 maio 2024
-
Necromancer Games - Book of Lost Spells 1e Feel - Flip eBook Pages19 maio 2024
-
Celebrities at the 2023 US Open Finals: Photos19 maio 2024
-
What Does BTTS Mean in Betting? – Both Teams To Score Guide19 maio 2024
-
Yose on X: Here's a remastered concept I made for BO2 on Ps419 maio 2024
-
Κοινότητα Steam :: Οδηγός :: Guia completo do jogo com todos os19 maio 2024
-
Some stuff that I'll post with no reason~ - 💖~Picrew.me~💖 - Wattpad19 maio 2024
-
Shop Papa Cupcake Picks: Father's Day & Grandpa Cupcake Picks – Sprinkle Bee Sweet19 maio 2024
-
Overlord IV Ep. 1 Sorcerer Kingdom Ains Ooal Gown: Ains Ooal Gown Nation of Leading Darkness19 maio 2024