Deep deterministic policy gradient algorithm: A systematic review

Deep deterministic policy gradient algorithm: A systematic review

2024 | Ebrahim Hamid Sumiea, Said Jadid Abdulkadir, Hitham Seddig Alhussian, Safwan Mahmood Al-Selwi, Alawi Alqushaibi, Mohammed Gamal Ragab, Suliman Mohamed Fati
This systematic review and meta-analysis provide a comprehensive examination of the Deep Deterministic Policy Gradient (DDPG) algorithm, a popular Deep Reinforcement Learning (DRL) method. DDPG combines the strengths of value-based and policy-based reinforcement learning methods, using an actor-critic approach to solve continuous control problems. The review identifies 85 relevant studies published between 2018 and 2023, covering various applications of DDPG in fields such as autonomous driving, unmanned aerial vehicles, resource allocation, communications and the Internet of Things, robotics, and finance. The study highlights the algorithm's strengths, including its ability to handle high-dimensional state and action spaces, and its stability and convergence properties. However, it also identifies challenges such as overestimation bias, sensitivity to hyperparameters, and the exploration versus exploitation dilemma. The review discusses various extensions and modifications of DDPG, including TD3, SAC, and D4PG, which aim to improve the algorithm's performance and stability. The study also provides insights into the evaluation measures used to assess DDPG's performance, such as reward, convergence, exploration, and robustness. Finally, the review highlights the increasing intensity of publications related to DDPG, indicating its growing importance in the field of DRL. The findings of this review provide valuable insights for researchers and practitioners in the field of DRL and DDPG, offering a comprehensive overview of the algorithm's current state, challenges, and future directions.This systematic review and meta-analysis provide a comprehensive examination of the Deep Deterministic Policy Gradient (DDPG) algorithm, a popular Deep Reinforcement Learning (DRL) method. DDPG combines the strengths of value-based and policy-based reinforcement learning methods, using an actor-critic approach to solve continuous control problems. The review identifies 85 relevant studies published between 2018 and 2023, covering various applications of DDPG in fields such as autonomous driving, unmanned aerial vehicles, resource allocation, communications and the Internet of Things, robotics, and finance. The study highlights the algorithm's strengths, including its ability to handle high-dimensional state and action spaces, and its stability and convergence properties. However, it also identifies challenges such as overestimation bias, sensitivity to hyperparameters, and the exploration versus exploitation dilemma. The review discusses various extensions and modifications of DDPG, including TD3, SAC, and D4PG, which aim to improve the algorithm's performance and stability. The study also provides insights into the evaluation measures used to assess DDPG's performance, such as reward, convergence, exploration, and robustness. Finally, the review highlights the increasing intensity of publications related to DDPG, indicating its growing importance in the field of DRL. The findings of this review provide valuable insights for researchers and practitioners in the field of DRL and DDPG, offering a comprehensive overview of the algorithm's current state, challenges, and future directions.
Reach us at info@study.space
[slides and audio] Deep deterministic policy gradient algorithm%3A A systematic review