This paper introduces DQ-RTS, a decentralized Multi-Agent Reinforcement Learning (MARL) algorithm designed to address challenges in distributed environments with non-ideal communication and varying numbers of agents. DQ-RTS incorporates an optimized communication protocol to mitigate data loss between agents. A comparative analysis between DQ-RTS and its decentralized counterpart Q-RTS shows that DQ-RTS achieves faster convergence, with speed-up factors ranging from 1.6 to 2.7 in scenarios with non-ideal communication. DQ-RTS also demonstrates robustness by maintaining performance even when the number of agents fluctuates, making it suitable for applications requiring adaptable agent numbers. Extensive experiments on various benchmark tasks validate the scalability and effectiveness of DQ-RTS, establishing its potential as a practical solution for resilient MARL in dynamic environments. Reinforcement Learning (RL) is a machine learning technique used to train agents to perform tasks in environments by maximizing a reward signal. RL has applications in finance, robotics, natural language processing, and communications. Multi-Agent Reinforcement Learning (MARL) is a subfield of RL where multiple agents interact in the same environment. MARL is used in various applications, including autonomous driving, drone control, telecommunications, and smart grids. MARL is also useful in IoT applications where agents operate in a decentralized manner. In MARL, agents can interact in three settings: cooperative, competitive, and mixed. DQ-RTS is a novel fully decentralized MARL algorithm inspired by Q-RTS. It allows agents to operate independently, enabling knowledge distribution and resilience to data transmission failures and changes in the number of agents. DQ-RTS uses a local swarm knowledge Q-matrix computed by each agent, allowing for decentralized knowledge sharing. The algorithm operates in two phases: an update phase where agents interact with the environment and update their Q-tables, and a communication phase where agents share knowledge. DQ-RTS is designed to be hardware-friendly and suitable for implementation in digital circuits like FPGAs. Experiments show that DQ-RTS outperforms Q-RTS in scenarios with non-ideal communication, achieving faster convergence and better performance. The algorithm's decentralized structure reduces communication overhead and allows for efficient knowledge distribution. DQ-RTS is particularly effective in sparse communication networks, where it maintains performance despite reduced communication range. The algorithm's ability to handle varying numbers of agents and communication failures makes it suitable for dynamic environments. The results demonstrate that DQ-RTS is a robust and efficient solution for decentralized MARL in distributed environments.This paper introduces DQ-RTS, a decentralized Multi-Agent Reinforcement Learning (MARL) algorithm designed to address challenges in distributed environments with non-ideal communication and varying numbers of agents. DQ-RTS incorporates an optimized communication protocol to mitigate data loss between agents. A comparative analysis between DQ-RTS and its decentralized counterpart Q-RTS shows that DQ-RTS achieves faster convergence, with speed-up factors ranging from 1.6 to 2.7 in scenarios with non-ideal communication. DQ-RTS also demonstrates robustness by maintaining performance even when the number of agents fluctuates, making it suitable for applications requiring adaptable agent numbers. Extensive experiments on various benchmark tasks validate the scalability and effectiveness of DQ-RTS, establishing its potential as a practical solution for resilient MARL in dynamic environments. Reinforcement Learning (RL) is a machine learning technique used to train agents to perform tasks in environments by maximizing a reward signal. RL has applications in finance, robotics, natural language processing, and communications. Multi-Agent Reinforcement Learning (MARL) is a subfield of RL where multiple agents interact in the same environment. MARL is used in various applications, including autonomous driving, drone control, telecommunications, and smart grids. MARL is also useful in IoT applications where agents operate in a decentralized manner. In MARL, agents can interact in three settings: cooperative, competitive, and mixed. DQ-RTS is a novel fully decentralized MARL algorithm inspired by Q-RTS. It allows agents to operate independently, enabling knowledge distribution and resilience to data transmission failures and changes in the number of agents. DQ-RTS uses a local swarm knowledge Q-matrix computed by each agent, allowing for decentralized knowledge sharing. The algorithm operates in two phases: an update phase where agents interact with the environment and update their Q-tables, and a communication phase where agents share knowledge. DQ-RTS is designed to be hardware-friendly and suitable for implementation in digital circuits like FPGAs. Experiments show that DQ-RTS outperforms Q-RTS in scenarios with non-ideal communication, achieving faster convergence and better performance. The algorithm's decentralized structure reduces communication overhead and allows for efficient knowledge distribution. DQ-RTS is particularly effective in sparse communication networks, where it maintains performance despite reduced communication range. The algorithm's ability to handle varying numbers of agents and communication failures makes it suitable for dynamic environments. The results demonstrate that DQ-RTS is a robust and efficient solution for decentralized MARL in distributed environments.