WEAVER: Foundation Models for Creative Writing

WEAVER: Foundation Models for Creative Writing

30 Jan 2024 | Tiannan Wang, Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang, Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai, Yibin Liu, Jialong Wu, Shengwei Ding, Long Li, Zhiwei Huang, Xinle Deng, Teng Yu, Gangan Ma, Han Xiao, Zixin Chen, Danjun Xiang, Yunxia Wang, Yuanyuan Zhu, Yi Xiao, Jing Wang, Yiru Wang, Siran Ding, Jiayang Huang, Jiayi Xu, Yilihamu Tayier, Zhenyu Hu, Yuan Gao, Chengfeng Zheng, Yueshu Ye, Yihang Li, Lei Wan, Xinyue Jiang, Yujie Wang, Siyu Cheng, Zhule Song, Xiangru Tang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, Yuchen Eleanor Jiang*, Wangchunshu Zhou*
Weaver is a family of large language models (LLMs) designed specifically for content creation, particularly in the domain of writing. The models are pre-trained on a curated corpus focused on improving writing capabilities and then fine-tuned for creative and professional writing tasks. The pre-training and fine-tuning processes involve advanced methods for data synthesis and alignment, ensuring that the models can produce more human-like texts and follow diverse instructions. The Weaver family includes models of sizes MINI (1.8B), BASE (6B), Pro (14B), and ULTRA (34B), each suitable for different applications. Evaluation on the WRITEBENCH benchmark shows that all sizes of Weaver outperform generalist LLMs, with the ULTRA model surpassing GPT-4 in various writing scenarios. Weaver also supports retrieval-augmented generation (RAG) and function calling, enabling it to integrate external knowledge and tools. The report introduces WawaWRITER, an innovative human-AI collaborative writing platform that leverages Weaver's capabilities, offering features such as human-AI co-editing, personal knowledge bases, personalized writing assistance, and infinite long text generation. The platform aims to provide a next-generation AI-assisted writing experience that is more helpful and enjoyable.Weaver is a family of large language models (LLMs) designed specifically for content creation, particularly in the domain of writing. The models are pre-trained on a curated corpus focused on improving writing capabilities and then fine-tuned for creative and professional writing tasks. The pre-training and fine-tuning processes involve advanced methods for data synthesis and alignment, ensuring that the models can produce more human-like texts and follow diverse instructions. The Weaver family includes models of sizes MINI (1.8B), BASE (6B), Pro (14B), and ULTRA (34B), each suitable for different applications. Evaluation on the WRITEBENCH benchmark shows that all sizes of Weaver outperform generalist LLMs, with the ULTRA model surpassing GPT-4 in various writing scenarios. Weaver also supports retrieval-augmented generation (RAG) and function calling, enabling it to integrate external knowledge and tools. The report introduces WawaWRITER, an innovative human-AI collaborative writing platform that leverages Weaver's capabilities, offering features such as human-AI co-editing, personal knowledge bases, personalized writing assistance, and infinite long text generation. The platform aims to provide a next-generation AI-assisted writing experience that is more helpful and enjoyable.
Reach us at info@study.space