This paper introduces an online optimization algorithm for matrix factorization and sparse coding, which efficiently handles large-scale data sets with millions of samples. The algorithm is designed to adaptively learn a dictionary of basis elements for sparse representation of data, which is widely used in machine learning, neuroscience, signal processing, and statistics. The method is based on stochastic approximations and is capable of handling various matrix factorization formulations, including dictionary learning, non-negative matrix factorization, and sparse principal component analysis. The algorithm is shown to converge almost surely to a stationary point of the objective function and is demonstrated to achieve state-of-the-art performance in terms of speed and optimization for both small and large data sets. The algorithm processes data one at a time or in mini-batches, making it suitable for applications such as image and video processing. It is also shown to be effective in learning dictionaries for natural images and genomic data, with applications including image inpainting. The algorithm is parameter-free and does not require learning rate tuning, and it is efficient in terms of memory and computational cost. The paper also discusses the convergence analysis of the algorithm, showing that it converges almost surely to a stationary point of the objective function under certain assumptions. The algorithm is shown to be effective in a wide range of learning problems, including sparse coding, dictionary learning, and matrix factorization.This paper introduces an online optimization algorithm for matrix factorization and sparse coding, which efficiently handles large-scale data sets with millions of samples. The algorithm is designed to adaptively learn a dictionary of basis elements for sparse representation of data, which is widely used in machine learning, neuroscience, signal processing, and statistics. The method is based on stochastic approximations and is capable of handling various matrix factorization formulations, including dictionary learning, non-negative matrix factorization, and sparse principal component analysis. The algorithm is shown to converge almost surely to a stationary point of the objective function and is demonstrated to achieve state-of-the-art performance in terms of speed and optimization for both small and large data sets. The algorithm processes data one at a time or in mini-batches, making it suitable for applications such as image and video processing. It is also shown to be effective in learning dictionaries for natural images and genomic data, with applications including image inpainting. The algorithm is parameter-free and does not require learning rate tuning, and it is efficient in terms of memory and computational cost. The paper also discusses the convergence analysis of the algorithm, showing that it converges almost surely to a stationary point of the objective function under certain assumptions. The algorithm is shown to be effective in a wide range of learning problems, including sparse coding, dictionary learning, and matrix factorization.