The chapter introduces the fundamental concepts of least squares approximation, a key method in estimation theory. It begins with a practical example of curve fitting to illustrate the basic idea of minimizing the sum of squared residuals. The chapter then formally introduces Gauss's principle of linear least squares, which is central to solving a wide range of estimation problems. The method is applied to both batch and sequential estimation scenarios, including linear and nonlinear models. The chapter also discusses weighted least squares, which allows for different weights to be assigned to measurements based on their precision, and constrained least squares, which imposes equality constraints on the estimated parameters. Examples are provided to demonstrate the application of these methods, emphasizing the importance of proper model specification and the impact of measurement errors on estimation accuracy. The chapter concludes with a discussion on the convexity of the performance surface and the conditions for a unique solution.The chapter introduces the fundamental concepts of least squares approximation, a key method in estimation theory. It begins with a practical example of curve fitting to illustrate the basic idea of minimizing the sum of squared residuals. The chapter then formally introduces Gauss's principle of linear least squares, which is central to solving a wide range of estimation problems. The method is applied to both batch and sequential estimation scenarios, including linear and nonlinear models. The chapter also discusses weighted least squares, which allows for different weights to be assigned to measurements based on their precision, and constrained least squares, which imposes equality constraints on the estimated parameters. Examples are provided to demonstrate the application of these methods, emphasizing the importance of proper model specification and the impact of measurement errors on estimation accuracy. The chapter concludes with a discussion on the convexity of the performance surface and the conditions for a unique solution.