This paper presents a rigorous proof of the convergence of stochastic iterative dynamic programming algorithms, specifically Q-learning and TD(λ). The authors show that these algorithms can be viewed as stochastic processes to which techniques from stochastic approximation theory apply. They provide a new convergence theorem that establishes a general class of convergent algorithms, including both Q-learning and TD(λ). The theorem is based on conditions that ensure convergence under certain assumptions about the learning process. The paper also discusses the convergence of TD(λ) for arbitrary λ and shows that the convergence proof extends to this case. The authors emphasize the importance of the temporal credit assignment problem in learning and the role of dynamic programming in solving it. They also highlight the similarities between Q-learning and TD(λ) and their applications in real-world learning problems. The paper concludes with a detailed proof of the convergence theorem, which is based on the properties of stochastic processes and the conditions required for convergence.This paper presents a rigorous proof of the convergence of stochastic iterative dynamic programming algorithms, specifically Q-learning and TD(λ). The authors show that these algorithms can be viewed as stochastic processes to which techniques from stochastic approximation theory apply. They provide a new convergence theorem that establishes a general class of convergent algorithms, including both Q-learning and TD(λ). The theorem is based on conditions that ensure convergence under certain assumptions about the learning process. The paper also discusses the convergence of TD(λ) for arbitrary λ and shows that the convergence proof extends to this case. The authors emphasize the importance of the temporal credit assignment problem in learning and the role of dynamic programming in solving it. They also highlight the similarities between Q-learning and TD(λ) and their applications in real-world learning problems. The paper concludes with a detailed proof of the convergence theorem, which is based on the properties of stochastic processes and the conditions required for convergence.