This paper introduces Temporal Segment Networks (TSN), a novel framework for video-based action recognition that effectively models long-range temporal structures. TSN combines a sparse temporal sampling strategy with video-level supervision to enable efficient and effective learning using the entire action video. The framework extracts short snippets from a long video sequence with sparse sampling, where the samples are uniformly distributed along the temporal dimension. A segmental structure is then used to aggregate information from the sampled snippets, allowing the model to capture long-range temporal structure across the entire video. The framework also incorporates a series of good practices for learning ConvNets on video data, including cross-modality pre-training, regularization, and enhanced data augmentation. The proposed method achieves state-of-the-art performance on the HMDB51 (69.4%) and UCF101 (94.2%) datasets. It also visualizes the learned ConvNet models, which qualitatively demonstrate the effectiveness of TSN and the proposed good practices. The framework is built upon the successful two-stream architecture, with modifications to address the challenges of modeling long-range temporal structures and learning with limited training samples. The method uses a sparse temporal sampling strategy to reduce computational cost while preserving relevant information, enabling end-to-end learning over long video sequences. The paper also explores various input modalities for two-stream ConvNets, including RGB images, stacked RGB difference, stacked optical flow fields, and stacked warped optical flow fields. The experiments show that combining these modalities improves performance. The method is tested on two challenging action recognition datasets, UCF101 and HMDB51, and outperforms existing methods. The results demonstrate the effectiveness of TSN in modeling long-term temporal structures, which is crucial for better understanding of actions in videos. The framework is also shown to be effective in reducing overfitting through the use of good practices such as cross-modality pre-training and regularization. The paper concludes that TSN provides a solid framework for video-based action recognition, with a reasonable computational cost and high performance.This paper introduces Temporal Segment Networks (TSN), a novel framework for video-based action recognition that effectively models long-range temporal structures. TSN combines a sparse temporal sampling strategy with video-level supervision to enable efficient and effective learning using the entire action video. The framework extracts short snippets from a long video sequence with sparse sampling, where the samples are uniformly distributed along the temporal dimension. A segmental structure is then used to aggregate information from the sampled snippets, allowing the model to capture long-range temporal structure across the entire video. The framework also incorporates a series of good practices for learning ConvNets on video data, including cross-modality pre-training, regularization, and enhanced data augmentation. The proposed method achieves state-of-the-art performance on the HMDB51 (69.4%) and UCF101 (94.2%) datasets. It also visualizes the learned ConvNet models, which qualitatively demonstrate the effectiveness of TSN and the proposed good practices. The framework is built upon the successful two-stream architecture, with modifications to address the challenges of modeling long-range temporal structures and learning with limited training samples. The method uses a sparse temporal sampling strategy to reduce computational cost while preserving relevant information, enabling end-to-end learning over long video sequences. The paper also explores various input modalities for two-stream ConvNets, including RGB images, stacked RGB difference, stacked optical flow fields, and stacked warped optical flow fields. The experiments show that combining these modalities improves performance. The method is tested on two challenging action recognition datasets, UCF101 and HMDB51, and outperforms existing methods. The results demonstrate the effectiveness of TSN in modeling long-term temporal structures, which is crucial for better understanding of actions in videos. The framework is also shown to be effective in reducing overfitting through the use of good practices such as cross-modality pre-training and regularization. The paper concludes that TSN provides a solid framework for video-based action recognition, with a reasonable computational cost and high performance.