2 Jan 2024 | Shujie Li, Liang Li, Ruiying Geng, Min Yang, Binhua Li, Guanghu Yuan, Wanwei He, Shao Yuan, Can Ma, Fei Huang, Yongbin Li
This paper presents a unified data-to-text (D2T) pre-training method that transforms various structured data into a graph format and casts D2T generation tasks as graph-to-text generation. The proposed method, named UniD2T, integrates different types of structured data (e.g., tables, key-value pairs, knowledge graphs) into a unified graph format while retaining their structural information. To effectively capture the graph structure, UniD2T introduces a structure-enhanced Transformer with a new position matrix and attention matrix. The position matrix encodes relative positional information of connected nodes, while the attention matrix incorporates graph structures by considering explicit connectivity. Extensive experiments on six benchmark datasets demonstrate the effectiveness of UniD2T, showing significantly better performance compared to strong baselines. The model's ability to handle diverse structured data and its improved performance in various downstream tasks highlight the potential of this unified pre-training approach.This paper presents a unified data-to-text (D2T) pre-training method that transforms various structured data into a graph format and casts D2T generation tasks as graph-to-text generation. The proposed method, named UniD2T, integrates different types of structured data (e.g., tables, key-value pairs, knowledge graphs) into a unified graph format while retaining their structural information. To effectively capture the graph structure, UniD2T introduces a structure-enhanced Transformer with a new position matrix and attention matrix. The position matrix encodes relative positional information of connected nodes, while the attention matrix incorporates graph structures by considering explicit connectivity. Extensive experiments on six benchmark datasets demonstrate the effectiveness of UniD2T, showing significantly better performance compared to strong baselines. The model's ability to handle diverse structured data and its improved performance in various downstream tasks highlight the potential of this unified pre-training approach.