2024 | Bo Peng, Xinyi Ling, Ziru Chen, Huan Sun, Xia Ning
This paper introduces eCeLLM, a series of e-commerce large language models (LLMs) developed using a newly created benchmark instruction dataset, ECInstruct. ECInstruct is an open-sourced, large-scale, and high-quality dataset designed for e-commerce tasks, containing 116,528 samples across 10 diverse e-commerce tasks. The dataset includes a variety of tasks such as product understanding, user understanding, query product matching, and product question answering. The eCeLLM models are developed by instruction-tuning general-purpose LLMs on ECInstruct data.
The eCeLLM models are evaluated on both in-domain (IND) and out-of-domain (OOD) tasks. The results show that eCeLLM significantly outperforms baseline models, including the most advanced GPT-4 and state-of-the-art task-specific models, in IND evaluation. Additionally, eCeLLM demonstrates excellent generalizability to OOD settings, including unseen products and instructions, highlighting its superiority as a generalist e-commerce model. The eCeLLM models and ECInstruct dataset are publicly accessible through the provided link.
The paper also discusses the limitations of existing e-commerce models, such as their limited success in generalist modeling and poor performance on new users and products. The study highlights the potential of LLMs in e-commerce applications and the importance of using high-quality, real-world data for training. The results demonstrate that eCeLLM models, trained on diverse and high-quality instruction data, achieve strong performance across various e-commerce tasks, showing the effectiveness of instruction-tuning in improving LLMs for e-commerce applications. The study contributes to the development of more versatile and effective LLMs for e-commerce by providing a comprehensive benchmark dataset and state-of-the-art models.This paper introduces eCeLLM, a series of e-commerce large language models (LLMs) developed using a newly created benchmark instruction dataset, ECInstruct. ECInstruct is an open-sourced, large-scale, and high-quality dataset designed for e-commerce tasks, containing 116,528 samples across 10 diverse e-commerce tasks. The dataset includes a variety of tasks such as product understanding, user understanding, query product matching, and product question answering. The eCeLLM models are developed by instruction-tuning general-purpose LLMs on ECInstruct data.
The eCeLLM models are evaluated on both in-domain (IND) and out-of-domain (OOD) tasks. The results show that eCeLLM significantly outperforms baseline models, including the most advanced GPT-4 and state-of-the-art task-specific models, in IND evaluation. Additionally, eCeLLM demonstrates excellent generalizability to OOD settings, including unseen products and instructions, highlighting its superiority as a generalist e-commerce model. The eCeLLM models and ECInstruct dataset are publicly accessible through the provided link.
The paper also discusses the limitations of existing e-commerce models, such as their limited success in generalist modeling and poor performance on new users and products. The study highlights the potential of LLMs in e-commerce applications and the importance of using high-quality, real-world data for training. The results demonstrate that eCeLLM models, trained on diverse and high-quality instruction data, achieve strong performance across various e-commerce tasks, showing the effectiveness of instruction-tuning in improving LLMs for e-commerce applications. The study contributes to the development of more versatile and effective LLMs for e-commerce by providing a comprehensive benchmark dataset and state-of-the-art models.