This paper introduces GraphToken, a parameter-efficient method for encoding structured data into sequential form for use in large language models (LLMs). Unlike previous approaches that focus on limited domains, GraphToken is designed for general encoding of structured data to support various reasoning tasks. The method learns an encoding function to extend prompts with explicit structured information, allowing LLMs to better understand and reason about graph-based data. The paper shows that explicitly representing graph structure significantly improves performance on graph reasoning tasks, achieving up to 73% improvements on the GraphQA benchmark. GraphToken uses a graph encoder that generates continuous representations for the LLM input, bypassing the need for text-based serialization. This approach allows the LLM to interpret the graph structure more effectively. The graph encoder is trained to align its output with the LLM embedding space, while keeping LLM parameters frozen. This reduces computational requirements and enables efficient training. The paper evaluates GraphToken on various graph reasoning tasks, including graph-level, node-level, and edge-level tasks. Results show that GraphToken outperforms existing methods, particularly in tasks requiring graph reasoning. The method is also shown to generalize well to unseen tasks and graphs. The paper also explores different graph encoder architectures and features, finding that certain designs perform better on specific tasks. It highlights the importance of choosing the right architecture for the specific graph reasoning problem. Additionally, the study shows that breaking equivariance in graph encoders can enhance graph reasoning capabilities when powerful decoders like LLMs are available. Overall, GraphToken provides a parameter-efficient way to encode structured data for LLMs, significantly improving their ability to reason about graphs. The method is generalizable and effective across a range of graph reasoning tasks, making it a valuable tool for integrating structured data into LLMs.This paper introduces GraphToken, a parameter-efficient method for encoding structured data into sequential form for use in large language models (LLMs). Unlike previous approaches that focus on limited domains, GraphToken is designed for general encoding of structured data to support various reasoning tasks. The method learns an encoding function to extend prompts with explicit structured information, allowing LLMs to better understand and reason about graph-based data. The paper shows that explicitly representing graph structure significantly improves performance on graph reasoning tasks, achieving up to 73% improvements on the GraphQA benchmark. GraphToken uses a graph encoder that generates continuous representations for the LLM input, bypassing the need for text-based serialization. This approach allows the LLM to interpret the graph structure more effectively. The graph encoder is trained to align its output with the LLM embedding space, while keeping LLM parameters frozen. This reduces computational requirements and enables efficient training. The paper evaluates GraphToken on various graph reasoning tasks, including graph-level, node-level, and edge-level tasks. Results show that GraphToken outperforms existing methods, particularly in tasks requiring graph reasoning. The method is also shown to generalize well to unseen tasks and graphs. The paper also explores different graph encoder architectures and features, finding that certain designs perform better on specific tasks. It highlights the importance of choosing the right architecture for the specific graph reasoning problem. Additionally, the study shows that breaking equivariance in graph encoders can enhance graph reasoning capabilities when powerful decoders like LLMs are available. Overall, GraphToken provides a parameter-efficient way to encode structured data for LLMs, significantly improving their ability to reason about graphs. The method is generalizable and effective across a range of graph reasoning tasks, making it a valuable tool for integrating structured data into LLMs.