This paper introduces the Natural Actor-Critic (NAC) algorithm, a novel model-free reinforcement learning approach that improves policy parameters using natural policy gradients and estimates value functions via linear regression. The natural policy gradient is derived from the Fisher information metric, which ensures that policy improvements are independent of the coordinate frame of the policy representation and are more efficient than regular policy gradients. The critic uses a compatible function approximation to estimate the value function, enabling the algorithm to learn control on complex tasks like robotic arm manipulation. The NAC algorithm is shown to be a generalization of several existing reinforcement learning methods, including the original Actor-Critic and Bradtke's Linear Quadratic Q-Learning. The algorithm consists of two main components: the actor, which updates the policy using natural gradients, and the critic, which estimates the value function using linear regression. The actor improvement step is based on the natural gradient, which is derived from the Fisher information matrix, ensuring convergence to a local minimum. The paper also presents empirical evaluations demonstrating the effectiveness of the NAC algorithm in comparison to previous methods. The algorithm is applied to a robotic task involving a baseball bat hitting a T-ball, showing improved performance over other methods. The NAC algorithm is also shown to be applicable to episodic tasks, where the value function is estimated using a linear regression approach. The algorithm is compared to other methods in tasks such as cart-pole balancing and motor primitive learning, where it outperforms existing approaches. The paper concludes that the NAC algorithm provides a robust and efficient framework for reinforcement learning, with strong convergence guarantees and the ability to handle complex tasks. The algorithm is shown to be covariant, meaning that policy improvements are independent of the coordinate frame of the policy representation, and is applicable to a wide range of reinforcement learning problems.This paper introduces the Natural Actor-Critic (NAC) algorithm, a novel model-free reinforcement learning approach that improves policy parameters using natural policy gradients and estimates value functions via linear regression. The natural policy gradient is derived from the Fisher information metric, which ensures that policy improvements are independent of the coordinate frame of the policy representation and are more efficient than regular policy gradients. The critic uses a compatible function approximation to estimate the value function, enabling the algorithm to learn control on complex tasks like robotic arm manipulation. The NAC algorithm is shown to be a generalization of several existing reinforcement learning methods, including the original Actor-Critic and Bradtke's Linear Quadratic Q-Learning. The algorithm consists of two main components: the actor, which updates the policy using natural gradients, and the critic, which estimates the value function using linear regression. The actor improvement step is based on the natural gradient, which is derived from the Fisher information matrix, ensuring convergence to a local minimum. The paper also presents empirical evaluations demonstrating the effectiveness of the NAC algorithm in comparison to previous methods. The algorithm is applied to a robotic task involving a baseball bat hitting a T-ball, showing improved performance over other methods. The NAC algorithm is also shown to be applicable to episodic tasks, where the value function is estimated using a linear regression approach. The algorithm is compared to other methods in tasks such as cart-pole balancing and motor primitive learning, where it outperforms existing approaches. The paper concludes that the NAC algorithm provides a robust and efficient framework for reinforcement learning, with strong convergence guarantees and the ability to handle complex tasks. The algorithm is shown to be covariant, meaning that policy improvements are independent of the coordinate frame of the policy representation, and is applicable to a wide range of reinforcement learning problems.