This paper addresses the challenge of enhancing the graph understanding and reasoning capabilities of large language models (LLMs). To this end, the authors propose a benchmark named GraphInstruct, which includes 21 classical graph reasoning tasks with diverse graph generation pipelines and detailed reasoning steps. Based on GraphInstruct, they construct GraphLM through efficient instruction-tuning, demonstrating superior graph understanding capabilities. To further improve the graph reasoning capability, they introduce a step mask training strategy and develop GraphLM+, which shows even better performance. Extensive experiments validate the effectiveness of GraphLM and GraphLM+ over other LLMs. The paper also discusses the limitations and future directions, emphasizing the need for more realistic graph reasoning tasks and larger training datasets. The code for generating GraphInstruct is publicly available.This paper addresses the challenge of enhancing the graph understanding and reasoning capabilities of large language models (LLMs). To this end, the authors propose a benchmark named GraphInstruct, which includes 21 classical graph reasoning tasks with diverse graph generation pipelines and detailed reasoning steps. Based on GraphInstruct, they construct GraphLM through efficient instruction-tuning, demonstrating superior graph understanding capabilities. To further improve the graph reasoning capability, they introduce a step mask training strategy and develop GraphLM+, which shows even better performance. Extensive experiments validate the effectiveness of GraphLM and GraphLM+ over other LLMs. The paper also discusses the limitations and future directions, emphasizing the need for more realistic graph reasoning tasks and larger training datasets. The code for generating GraphInstruct is publicly available.