This paper proposes a novel method for continual knowledge graph embedding (CKGE), called IncDE, which effectively learns emerging knowledge while preserving old knowledge by leveraging the explicit graph structure of knowledge graphs (KGs). Traditional CKGE methods often ignore the graph structure, leading to inefficient learning and catastrophic forgetting. IncDE addresses these issues through three key components: hierarchical ordering, incremental distillation, and a two-stage training strategy. Hierarchical ordering organizes new triples into layers based on their proximity to the old graph structure, ensuring that entities and relations with higher importance are prioritized. Incremental distillation preserves old knowledge by distilling entity representations from previous layers to the current one, allowing the model to retain essential features of old knowledge. The two-stage training strategy first freezes the representations of old entities and relations, then trains all entities and relations in the second stage to prevent the disruption of old knowledge by under-trained new knowledge. Experiments on various datasets show that IncDE outperforms state-of-the-art baselines in terms of mean reciprocal rank (MRR), Hits@1, and Hits@10. IncDE achieves improvements of 0.2%-6.5% in MRR, demonstrating its effectiveness in preserving old knowledge while learning new knowledge. Ablation studies confirm the importance of incremental distillation, hierarchical ordering, and two-stage training in achieving these results. The method is also shown to be effective in real-world scenarios with varying knowledge growth, indicating its scalability and adaptability. The contributions of this paper include: (1) a novel CKGE framework that effectively learns and preserves knowledge using the explicit graph structure; (2) hierarchical ordering to determine an optimal learning sequence for emerging knowledge and incremental distillation and two-stage training to preserve old knowledge; and (3) the construction of three new datasets based on the scale changes of new knowledge, demonstrating that IncDE outperforms strong baselines. Notably, incremental distillation improves MRR by 0.2%-6.5%.This paper proposes a novel method for continual knowledge graph embedding (CKGE), called IncDE, which effectively learns emerging knowledge while preserving old knowledge by leveraging the explicit graph structure of knowledge graphs (KGs). Traditional CKGE methods often ignore the graph structure, leading to inefficient learning and catastrophic forgetting. IncDE addresses these issues through three key components: hierarchical ordering, incremental distillation, and a two-stage training strategy. Hierarchical ordering organizes new triples into layers based on their proximity to the old graph structure, ensuring that entities and relations with higher importance are prioritized. Incremental distillation preserves old knowledge by distilling entity representations from previous layers to the current one, allowing the model to retain essential features of old knowledge. The two-stage training strategy first freezes the representations of old entities and relations, then trains all entities and relations in the second stage to prevent the disruption of old knowledge by under-trained new knowledge. Experiments on various datasets show that IncDE outperforms state-of-the-art baselines in terms of mean reciprocal rank (MRR), Hits@1, and Hits@10. IncDE achieves improvements of 0.2%-6.5% in MRR, demonstrating its effectiveness in preserving old knowledge while learning new knowledge. Ablation studies confirm the importance of incremental distillation, hierarchical ordering, and two-stage training in achieving these results. The method is also shown to be effective in real-world scenarios with varying knowledge growth, indicating its scalability and adaptability. The contributions of this paper include: (1) a novel CKGE framework that effectively learns and preserves knowledge using the explicit graph structure; (2) hierarchical ordering to determine an optimal learning sequence for emerging knowledge and incremental distillation and two-stage training to preserve old knowledge; and (3) the construction of three new datasets based on the scale changes of new knowledge, demonstrating that IncDE outperforms strong baselines. Notably, incremental distillation improves MRR by 0.2%-6.5%.