This paper proposes a deep reinforcement learning algorithm, GARTD3, for autonomous UAV navigation with adaptive control in 3D environments. The algorithm addresses the challenge of obstacle avoidance in low-altitude, complex environments by introducing a guide attention mechanism and a novel velocity-constrained loss function. The guide attention mechanism shifts the UAV's decision focus between navigation and obstacle avoidance based on environmental changes, while the velocity-constrained loss function improves the UAV's velocity control capability. The algorithm is tested in a 3D simulation environment with multiple obstacles, demonstrating superior performance compared to state-of-the-art algorithms. Simulation results show that GARTD3 achieves a 9.35% increase in average reward, a 14% increase in task success rate, and a 14% decrease in collision rate. The algorithm's effectiveness is validated through extensive experiments, showing that it can navigate complex environments more efficiently and safely than other methods. The paper also highlights the importance of historical trajectory data and the benefits of using guide attention and velocity-constrained loss functions in improving UAV navigation and obstacle avoidance capabilities.This paper proposes a deep reinforcement learning algorithm, GARTD3, for autonomous UAV navigation with adaptive control in 3D environments. The algorithm addresses the challenge of obstacle avoidance in low-altitude, complex environments by introducing a guide attention mechanism and a novel velocity-constrained loss function. The guide attention mechanism shifts the UAV's decision focus between navigation and obstacle avoidance based on environmental changes, while the velocity-constrained loss function improves the UAV's velocity control capability. The algorithm is tested in a 3D simulation environment with multiple obstacles, demonstrating superior performance compared to state-of-the-art algorithms. Simulation results show that GARTD3 achieves a 9.35% increase in average reward, a 14% increase in task success rate, and a 14% decrease in collision rate. The algorithm's effectiveness is validated through extensive experiments, showing that it can navigate complex environments more efficiently and safely than other methods. The paper also highlights the importance of historical trajectory data and the benefits of using guide attention and velocity-constrained loss functions in improving UAV navigation and obstacle avoidance capabilities.