This paper introduces a parameter-sharing off-policy multi-agent path planning and following approach for multiple unmanned aerial vehicles (UAVs). The method uses laser scan data as input, providing a more realistic simulation of real-world scenarios compared to traditional grid-based maps. The UAVs employ the Soft Actor-Critic (SAC) algorithm to plan paths, enabling end-to-end processing of laser scan data to avoid obstacles and reach goals. The planner incorporates paths generated by a sampling-based method as following points, which are continuously updated as the UAV progresses. Shared experiences among agents accelerate policy convergence. A reward function is designed to encourage UAV movement, addressing the issue of UAVs remaining stationary or being overly cautious near the goal. A multi-UAV simulation environment is established to train and validate the approach. Simulation results show that the proposed method achieves an 80% success rate in guiding three UAVs to their goal points. The method outperforms traditional approaches in terms of efficiency and performance, demonstrating its effectiveness in complex scenarios. The approach is based on multi-agent reinforcement learning, with a focus on parameter sharing and experience replay to enhance learning and convergence. The algorithm is designed to be lightweight and efficient, avoiding unnecessary communication mechanisms. The results highlight the potential of the proposed method for real-world applications in multi-UAV systems.This paper introduces a parameter-sharing off-policy multi-agent path planning and following approach for multiple unmanned aerial vehicles (UAVs). The method uses laser scan data as input, providing a more realistic simulation of real-world scenarios compared to traditional grid-based maps. The UAVs employ the Soft Actor-Critic (SAC) algorithm to plan paths, enabling end-to-end processing of laser scan data to avoid obstacles and reach goals. The planner incorporates paths generated by a sampling-based method as following points, which are continuously updated as the UAV progresses. Shared experiences among agents accelerate policy convergence. A reward function is designed to encourage UAV movement, addressing the issue of UAVs remaining stationary or being overly cautious near the goal. A multi-UAV simulation environment is established to train and validate the approach. Simulation results show that the proposed method achieves an 80% success rate in guiding three UAVs to their goal points. The method outperforms traditional approaches in terms of efficiency and performance, demonstrating its effectiveness in complex scenarios. The approach is based on multi-agent reinforcement learning, with a focus on parameter sharing and experience replay to enhance learning and convergence. The algorithm is designed to be lightweight and efficient, avoiding unnecessary communication mechanisms. The results highlight the potential of the proposed method for real-world applications in multi-UAV systems.