Video2Game is a novel approach that automatically converts videos of real-world scenes into realistic and interactive game environments. The system consists of three core components: a neural radiance fields (NeRF) module that captures the geometry and visual appearance of the scene, a mesh module that distills knowledge from NeRF for faster rendering, and a physics module that models interactions and physical dynamics among objects. By following a carefully designed pipeline, Video2Game constructs an interactive and actionable digital replica of the real world. The system is benchmarked on both indoor and large-scale outdoor scenes, demonstrating the ability to produce highly-realistic renderings in real-time and build interactive games. The NeRF module is enhanced with additional features such as depth estimation, surface normals, semantics, and regularization to improve the quality of the rendered geometry. The system then converts the NeRF into a game-engine compatible mesh with neural texture maps, significantly improving rendering efficiency while maintaining quality. The scene is decomposed into individual actionable entities, each equipped with respective physics models, enabling physical interactions. The resulting virtual environment is photo-realistic, interactive, and runs in real-time. Video2Game is compatible with current game engines and can be integrated into WebGL-based game engines, allowing users to play and interact with the virtual world in real time on their personal browser. The system has been tested on various datasets, including KITTI-360 and VR-NeRF, demonstrating its effectiveness in generating high-quality depth maps, surface normals, and realistic physics simulations. The results show that Video2Game outperforms existing methods in terms of rendering quality and interactive compatibility. The system enables the creation of interactive virtual environments from a single video, with applications in robot simulation and digital game design.Video2Game is a novel approach that automatically converts videos of real-world scenes into realistic and interactive game environments. The system consists of three core components: a neural radiance fields (NeRF) module that captures the geometry and visual appearance of the scene, a mesh module that distills knowledge from NeRF for faster rendering, and a physics module that models interactions and physical dynamics among objects. By following a carefully designed pipeline, Video2Game constructs an interactive and actionable digital replica of the real world. The system is benchmarked on both indoor and large-scale outdoor scenes, demonstrating the ability to produce highly-realistic renderings in real-time and build interactive games. The NeRF module is enhanced with additional features such as depth estimation, surface normals, semantics, and regularization to improve the quality of the rendered geometry. The system then converts the NeRF into a game-engine compatible mesh with neural texture maps, significantly improving rendering efficiency while maintaining quality. The scene is decomposed into individual actionable entities, each equipped with respective physics models, enabling physical interactions. The resulting virtual environment is photo-realistic, interactive, and runs in real-time. Video2Game is compatible with current game engines and can be integrated into WebGL-based game engines, allowing users to play and interact with the virtual world in real time on their personal browser. The system has been tested on various datasets, including KITTI-360 and VR-NeRF, demonstrating its effectiveness in generating high-quality depth maps, surface normals, and realistic physics simulations. The results show that Video2Game outperforms existing methods in terms of rendering quality and interactive compatibility. The system enables the creation of interactive virtual environments from a single video, with applications in robot simulation and digital game design.