This paper introduces a new Monte-Carlo planning algorithm called UCT, which applies bandit ideas to guide selective sampling in rollout-based planning. The algorithm is designed for large state-space Markovian Decision Problems (MDPs) and is shown to be consistent, with finite sample bounds on estimation error. Experimental results demonstrate that UCT is significantly more efficient than its alternatives in several domains. The UCT algorithm is based on the UCB1 (Upper Confidence Bounds) algorithm for rollout-based planning. It selects actions in a way that balances exploration and exploitation, ensuring that the algorithm converges to the best action. The algorithm is applied to both MDPs and game-tree search, where it has shown significant performance advantages over other methods. Theoretical analysis shows that UCT is consistent, with the probability of selecting the optimal action converging to 1 as the number of samples increases. The algorithm is tested on two synthetic domains: random (P-game) trees and a stochastic shortest path problem (sailing). In the P-game experiments, UCT converges at a rate of order $ B^{D/2} $, similar to alpha-beta search. In the sailing domain, UCT requires significantly fewer samples to achieve the same error level compared to other algorithms, allowing it to solve larger problems. The paper also discusses related research, including the work of [10], who proposed using rollout-based Monte-Carlo planning with selective action sampling. They compared three strategies: uniform sampling, Boltzmann-exploration, and interval-estimation. The results show that UCT outperforms these strategies in terms of performance and efficiency. The authors conclude that UCT is a promising algorithm for Monte-Carlo planning in large state-space MDPs, with potential applications in real-world game programs. Future work includes analyzing UCT in stochastic shortest path problems and considering the effect of randomized terminating conditions.This paper introduces a new Monte-Carlo planning algorithm called UCT, which applies bandit ideas to guide selective sampling in rollout-based planning. The algorithm is designed for large state-space Markovian Decision Problems (MDPs) and is shown to be consistent, with finite sample bounds on estimation error. Experimental results demonstrate that UCT is significantly more efficient than its alternatives in several domains. The UCT algorithm is based on the UCB1 (Upper Confidence Bounds) algorithm for rollout-based planning. It selects actions in a way that balances exploration and exploitation, ensuring that the algorithm converges to the best action. The algorithm is applied to both MDPs and game-tree search, where it has shown significant performance advantages over other methods. Theoretical analysis shows that UCT is consistent, with the probability of selecting the optimal action converging to 1 as the number of samples increases. The algorithm is tested on two synthetic domains: random (P-game) trees and a stochastic shortest path problem (sailing). In the P-game experiments, UCT converges at a rate of order $ B^{D/2} $, similar to alpha-beta search. In the sailing domain, UCT requires significantly fewer samples to achieve the same error level compared to other algorithms, allowing it to solve larger problems. The paper also discusses related research, including the work of [10], who proposed using rollout-based Monte-Carlo planning with selective action sampling. They compared three strategies: uniform sampling, Boltzmann-exploration, and interval-estimation. The results show that UCT outperforms these strategies in terms of performance and efficiency. The authors conclude that UCT is a promising algorithm for Monte-Carlo planning in large state-space MDPs, with potential applications in real-world game programs. Future work includes analyzing UCT in stochastic shortest path problems and considering the effect of randomized terminating conditions.