This paper presents a method for learning dexterous in-hand manipulation policies using reinforcement learning (RL) in a simulated environment, which can then be transferred to a physical robotic hand. The Shadow Dexterous Hand, a five-fingered robotic hand with 24 degrees of freedom, is used as the physical platform. The policies are trained in a simulated environment with randomized physical properties, such as friction coefficients and object appearances, to improve transferability to the real world. The method does not rely on human demonstrations but naturally discovers human-like manipulation behaviors, including finger gaiting, multi-finger coordination, and controlled use of gravity. The RL system used is the same as the one used to train OpenAI Five. The policies are trained using a distributed RL system and a vision-based pose estimator. The vision model is trained separately to predict object poses from images. The policies are tested on the physical robot and show high dexterity, including the ability to perform various grasps and dynamic manipulation tasks. The results demonstrate that policies trained in simulation can transfer well to the physical robot, despite the differences between the simulated and real environments. The paper also discusses the importance of randomizations, memory in policies, and the use of vision for pose estimation in achieving successful transfer. The results show that the policies can achieve a high number of consecutive successful rotations on both the simulated and physical robots, with the physical robot showing slightly lower performance due to the reality gap. The paper also evaluates the performance of the vision-based pose estimator and finds that it can achieve good results on the physical robot. The study highlights the potential of RL in solving complex real-world robotics problems and the importance of sim-to-real transfer in robotics research.This paper presents a method for learning dexterous in-hand manipulation policies using reinforcement learning (RL) in a simulated environment, which can then be transferred to a physical robotic hand. The Shadow Dexterous Hand, a five-fingered robotic hand with 24 degrees of freedom, is used as the physical platform. The policies are trained in a simulated environment with randomized physical properties, such as friction coefficients and object appearances, to improve transferability to the real world. The method does not rely on human demonstrations but naturally discovers human-like manipulation behaviors, including finger gaiting, multi-finger coordination, and controlled use of gravity. The RL system used is the same as the one used to train OpenAI Five. The policies are trained using a distributed RL system and a vision-based pose estimator. The vision model is trained separately to predict object poses from images. The policies are tested on the physical robot and show high dexterity, including the ability to perform various grasps and dynamic manipulation tasks. The results demonstrate that policies trained in simulation can transfer well to the physical robot, despite the differences between the simulated and real environments. The paper also discusses the importance of randomizations, memory in policies, and the use of vision for pose estimation in achieving successful transfer. The results show that the policies can achieve a high number of consecutive successful rotations on both the simulated and physical robots, with the physical robot showing slightly lower performance due to the reality gap. The paper also evaluates the performance of the vision-based pose estimator and finds that it can achieve good results on the physical robot. The study highlights the potential of RL in solving complex real-world robotics problems and the importance of sim-to-real transfer in robotics research.