The paper introduces OSCAR, a novel pre-training method for vision-language tasks that leverages object tags detected in images as anchor points to improve the learning of semantic alignments between images and text. Existing methods often concatenate image region features and text features, using self-attention to learn alignments, which can be noisy and ambiguous due to over-sampling and object overlap. OSCAR pre-trains models on a large corpus of 6.5 million text-image pairs, using a Word-Tag-Image triple representation. The model is pre-trained with a masked token loss and a contrastive loss, and fine-tuned on downstream tasks. Experiments show that OSCAR achieves state-of-the-art performance on six well-established vision-language understanding and generation tasks, demonstrating the effectiveness of using object tags as anchor points. The paper also includes ablation studies and qualitative analyses to validate the contributions of OSCAR's design choices.The paper introduces OSCAR, a novel pre-training method for vision-language tasks that leverages object tags detected in images as anchor points to improve the learning of semantic alignments between images and text. Existing methods often concatenate image region features and text features, using self-attention to learn alignments, which can be noisy and ambiguous due to over-sampling and object overlap. OSCAR pre-trains models on a large corpus of 6.5 million text-image pairs, using a Word-Tag-Image triple representation. The model is pre-trained with a masked token loss and a contrastive loss, and fine-tuned on downstream tasks. Experiments show that OSCAR achieves state-of-the-art performance on six well-established vision-language understanding and generation tasks, demonstrating the effectiveness of using object tags as anchor points. The paper also includes ablation studies and qualitative analyses to validate the contributions of OSCAR's design choices.