ViLT-CLIP is a video and language tuning CLIP model that integrates multimodal prompt learning and scenario-guided optimization. The model aims to adapt image-based CLIP to video tasks by balancing supervised and generalized performance. It introduces three key components: 1) independent prompts for vision and text branches to learn language and visual contexts; 2) inter-modal prompt mapping to ensure mutual synergy; and 3) reducing the discrepancy between hand-crafted prompts and learnable prompts to preserve essential video scenarios. The model achieves competitive performance in various video recognition and retrieval tasks with reduced computational cost. The paper discusses the challenges of adapting CLIP to video tasks, including limited video-text datasets, higher computational demands, and video noise. It highlights the trade-off between freezing encoders for zero-shot performance and fine-tuning for supervised performance. ViLT-CLIP addresses these issues by using a frozen CLIP backbone with lightweight prompt learning on both vision and language branches. The model introduces hierarchical prompts, cross-bidirectional linear layers, and a constrained branch to reduce the gap between learnable and hand-crafted prompts. ViLT-CLIP is evaluated on multiple benchmarks, including supervised, few-shot, zero-shot, and base-to-novel generalization settings. It outperforms existing methods in video recognition and retrieval tasks, achieving higher accuracy and better generalization. The model's effectiveness is demonstrated through extensive experiments, showing its ability to bridge the modality gap and adapt to video tasks efficiently. The results indicate that multimodal prompt learning and scenario-guided optimization are effective in improving CLIP's performance on video tasks.ViLT-CLIP is a video and language tuning CLIP model that integrates multimodal prompt learning and scenario-guided optimization. The model aims to adapt image-based CLIP to video tasks by balancing supervised and generalized performance. It introduces three key components: 1) independent prompts for vision and text branches to learn language and visual contexts; 2) inter-modal prompt mapping to ensure mutual synergy; and 3) reducing the discrepancy between hand-crafted prompts and learnable prompts to preserve essential video scenarios. The model achieves competitive performance in various video recognition and retrieval tasks with reduced computational cost. The paper discusses the challenges of adapting CLIP to video tasks, including limited video-text datasets, higher computational demands, and video noise. It highlights the trade-off between freezing encoders for zero-shot performance and fine-tuning for supervised performance. ViLT-CLIP addresses these issues by using a frozen CLIP backbone with lightweight prompt learning on both vision and language branches. The model introduces hierarchical prompts, cross-bidirectional linear layers, and a constrained branch to reduce the gap between learnable and hand-crafted prompts. ViLT-CLIP is evaluated on multiple benchmarks, including supervised, few-shot, zero-shot, and base-to-novel generalization settings. It outperforms existing methods in video recognition and retrieval tasks, achieving higher accuracy and better generalization. The model's effectiveness is demonstrated through extensive experiments, showing its ability to bridge the modality gap and adapt to video tasks efficiently. The results indicate that multimodal prompt learning and scenario-guided optimization are effective in improving CLIP's performance on video tasks.