GUI-WORLD is a new dataset designed to evaluate and enhance the capabilities of Multimodal Large Language Models (MLLMs) in understanding Graphical User Interface (GUI) content, particularly in dynamic and sequential tasks. The dataset includes over 12,000 GUI videos, covering six scenarios such as desktop and mobile applications, multi-window interactions, and extended reality (XR) environments. It features meticulously crafted Human-MLLM annotations, including detailed captions, keyframes, and diverse types of questions and answers. The dataset is used to assess the performance of current state-of-the-art MLLMs, including ImageLLMs and VideoLLMs, in understanding various GUI content, especially dynamic and sequential content. The findings reveal that ImageLLMs struggle with dynamic GUI content without manually annotated keyframes or operation history, while VideoLLMs fall short in all GUI-oriented tasks due to the sparse GUI video dataset. Based on GUI-WORLD, the paper introduces a new model, GUI-Vid, a GUI-oriented VideoLLM with enhanced capabilities to handle various and complex GUI tasks. However, due to the limitations in the performance of base LLMs, using VideoLLMs as GUI agents remains a significant challenge. The paper also discusses various factors critical to GUI understanding, including the integration of textual information, the number of keyframes, and image resolutions. Overall, the key contributions of this paper are a new dataset, a novel model, and comprehensive experiments and valuable insights. The dataset and code are publicly available at the project homepage.GUI-WORLD is a new dataset designed to evaluate and enhance the capabilities of Multimodal Large Language Models (MLLMs) in understanding Graphical User Interface (GUI) content, particularly in dynamic and sequential tasks. The dataset includes over 12,000 GUI videos, covering six scenarios such as desktop and mobile applications, multi-window interactions, and extended reality (XR) environments. It features meticulously crafted Human-MLLM annotations, including detailed captions, keyframes, and diverse types of questions and answers. The dataset is used to assess the performance of current state-of-the-art MLLMs, including ImageLLMs and VideoLLMs, in understanding various GUI content, especially dynamic and sequential content. The findings reveal that ImageLLMs struggle with dynamic GUI content without manually annotated keyframes or operation history, while VideoLLMs fall short in all GUI-oriented tasks due to the sparse GUI video dataset. Based on GUI-WORLD, the paper introduces a new model, GUI-Vid, a GUI-oriented VideoLLM with enhanced capabilities to handle various and complex GUI tasks. However, due to the limitations in the performance of base LLMs, using VideoLLMs as GUI agents remains a significant challenge. The paper also discusses various factors critical to GUI understanding, including the integration of textual information, the number of keyframes, and image resolutions. Overall, the key contributions of this paper are a new dataset, a novel model, and comprehensive experiments and valuable insights. The dataset and code are publicly available at the project homepage.