Can Small Language Models be Good Reasoners for Sequential Recommendation?

Can Small Language Models be Good Reasoners for Sequential Recommendation?

May 13–17, 2024, Singapore | Yuling Wang, Changxin Tian, Binbin Hu, Yanhua Yu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Liang Pang, Xiao Wang
The paper "Can Small Language Models be Good Reasoners for Sequential Recommendation?" by Yuling Wang et al. explores the integration of large language models (LLMs) into sequential recommendation systems to enhance their reasoning capabilities. The authors address two main challenges: the complexity of user behavior patterns and the high resource requirements of LLMs. To overcome these issues, they propose SLIM (Step-by-step knowledge Distillation fraMework for recommendation), a novel framework that enables small language models to learn from larger teacher models through step-by-step reasoning. SLIM uses chain-of-thought (CoT) prompting to guide the teacher model in generating rationales based on user behavior sequences. These rationales are then used as labels to fine-tune a smaller student model, such as LLaMA2-7B, which acquires step-by-step reasoning capabilities. The student model's generated rationales are encoded into dense vectors, enabling it to provide meaningful recommendations in both ID-based and ID-agnostic scenarios. Experiments on three datasets (Video Games, Grocery and Gourmet Food, Home and Kitchen) demonstrate that SLIM significantly improves the performance of sequential recommendation backbones, outperforming state-of-the-art baselines. The method also shows promise in ID-agnostic scenarios, where it can generate high-quality recommendations without relying on any backbone models. Additionally, SLIM is shown to be more resource-efficient and cost-effective compared to existing LLM-based recommendations, making it suitable for real-world applications. The key contributions of the paper include: 1. Introducing the first knowledge distillation framework tailored for sequential recommendation. 2. Proposing SLIM, a novel step-by-step knowledge distillation framework that empowers sequential recommenders with CoT reasoning capabilities in a resource-efficient manner. 3. Demonstrating the effectiveness of SLIM through extensive experiments and further analysis showing its ability to generate meaningful recommendations at affordable costs.The paper "Can Small Language Models be Good Reasoners for Sequential Recommendation?" by Yuling Wang et al. explores the integration of large language models (LLMs) into sequential recommendation systems to enhance their reasoning capabilities. The authors address two main challenges: the complexity of user behavior patterns and the high resource requirements of LLMs. To overcome these issues, they propose SLIM (Step-by-step knowledge Distillation fraMework for recommendation), a novel framework that enables small language models to learn from larger teacher models through step-by-step reasoning. SLIM uses chain-of-thought (CoT) prompting to guide the teacher model in generating rationales based on user behavior sequences. These rationales are then used as labels to fine-tune a smaller student model, such as LLaMA2-7B, which acquires step-by-step reasoning capabilities. The student model's generated rationales are encoded into dense vectors, enabling it to provide meaningful recommendations in both ID-based and ID-agnostic scenarios. Experiments on three datasets (Video Games, Grocery and Gourmet Food, Home and Kitchen) demonstrate that SLIM significantly improves the performance of sequential recommendation backbones, outperforming state-of-the-art baselines. The method also shows promise in ID-agnostic scenarios, where it can generate high-quality recommendations without relying on any backbone models. Additionally, SLIM is shown to be more resource-efficient and cost-effective compared to existing LLM-based recommendations, making it suitable for real-world applications. The key contributions of the paper include: 1. Introducing the first knowledge distillation framework tailored for sequential recommendation. 2. Proposing SLIM, a novel step-by-step knowledge distillation framework that empowers sequential recommenders with CoT reasoning capabilities in a resource-efficient manner. 3. Demonstrating the effectiveness of SLIM through extensive experiments and further analysis showing its ability to generate meaningful recommendations at affordable costs.
Reach us at info@study.space
[slides] Can Small Language Models be Good Reasoners for Sequential Recommendation%3F | StudySpace