The paper introduces a novel deep network architecture called Dynamically Expandable Network (DEN) for lifelong learning, which can dynamically adjust its network capacity as it trains on a sequence of tasks. DEN efficiently trains in an online manner by selectively retraining only the necessary parts of the network, dynamically expanding its capacity when needed, and preventing semantic drift through unit splitting, duplication, and timestamping. The method is validated on multiple public datasets under lifelong learning scenarios, showing superior performance compared to existing methods, achieving similar or better performance with significantly fewer parameters. Further fine-tuning of the learned network on all tasks results in even better performance, outperforming batch models by up to 4.8% in accuracy. The paper also discusses related work on lifelong learning, catastrophic forgetting, and dynamic network expansion, and provides experimental results to demonstrate the effectiveness of DEN.The paper introduces a novel deep network architecture called Dynamically Expandable Network (DEN) for lifelong learning, which can dynamically adjust its network capacity as it trains on a sequence of tasks. DEN efficiently trains in an online manner by selectively retraining only the necessary parts of the network, dynamically expanding its capacity when needed, and preventing semantic drift through unit splitting, duplication, and timestamping. The method is validated on multiple public datasets under lifelong learning scenarios, showing superior performance compared to existing methods, achieving similar or better performance with significantly fewer parameters. Further fine-tuning of the learned network on all tasks results in even better performance, outperforming batch models by up to 4.8% in accuracy. The paper also discusses related work on lifelong learning, catastrophic forgetting, and dynamic network expansion, and provides experimental results to demonstrate the effectiveness of DEN.