This study evaluates the effectiveness of in-context learning on multimodal large language models (LLMs) for classifying cancer pathology images. The researchers used the Generative Pretrained Transformer 4 with Vision capabilities (GPT-4V) to perform in-context learning on three critical histopathology tasks: tissue subtype classification in colorectal cancer, colon polyp subtyping, and breast tumor detection in lymph node sections. The results show that GPT-4V can achieve or even surpass the performance of specialized neural networks trained for these tasks, using only a minimal number of samples. This demonstrates that large vision language models trained on non-domain-specific data can be effectively applied to medical image processing tasks, democratizing access to generalist AI models for medical experts, especially in areas where annotated data is scarce. The study highlights the potential of in-context learning in improving the performance of foundation vision models and suggests that it may be a more efficient and scalable approach compared to traditional deep learning models.This study evaluates the effectiveness of in-context learning on multimodal large language models (LLMs) for classifying cancer pathology images. The researchers used the Generative Pretrained Transformer 4 with Vision capabilities (GPT-4V) to perform in-context learning on three critical histopathology tasks: tissue subtype classification in colorectal cancer, colon polyp subtyping, and breast tumor detection in lymph node sections. The results show that GPT-4V can achieve or even surpass the performance of specialized neural networks trained for these tasks, using only a minimal number of samples. This demonstrates that large vision language models trained on non-domain-specific data can be effectively applied to medical image processing tasks, democratizing access to generalist AI models for medical experts, especially in areas where annotated data is scarce. The study highlights the potential of in-context learning in improving the performance of foundation vision models and suggests that it may be a more efficient and scalable approach compared to traditional deep learning models.