Orthogonal Least Squares Learning Algorithm for Radial Basis Function Networks

Orthogonal Least Squares Learning Algorithm for Radial Basis Function Networks

March 1991 | S. Chen, C. F. N. Cowan, and P. M. Grant
This paper proposes an orthogonal least squares (OLS) learning algorithm for radial basis function (RBF) networks, which addresses the limitations of traditional methods that rely on random selection of RBF centers. The OLS algorithm selects RBF centers one by one in a rational way, maximizing the increment to the explained variance or energy of the desired output. This approach avoids numerical ill-conditioning and provides a simple and efficient means for fitting RBF networks. The algorithm is illustrated using examples from two signal processing applications: modeling time series data and communications channel equalization. The RBF network is a special two-layer network that is linear in the parameters by fixing the RBF centers and nonlinearities in the hidden layer. The output layer implements a linear combiner, and the weights of this combiner can be determined using the linear least squares method. The choice of the nonlinearity in the RBF network is not crucial for performance, but the selection of centers is critical. The OLS algorithm selects centers from the data points in a forward regression manner, maximizing the explained variance of the desired output at each step. This ensures that the selected centers are optimal for the task. In the time series modeling example, an RBF network is used to predict monthly unemployment figures in West Germany. The OLS algorithm selects 55 centers from the data points, resulting in a model that closely matches the actual data. The prediction error is small, and the model is validated using statistical tests. In the communications channel equalization example, an RBF network is used to reconstruct input signals in a noisy environment. The OLS algorithm selects 12 centers, resulting in a network that performs well compared to a randomly selected network. The performance of the OLS-based RBF network is compared to a randomly selected one, and it is shown to offer much better results. The OLS learning algorithm is a block-data algorithm that can be used in adaptive applications where filter parameters need to be updated as each sample is collected. The algorithm is efficient and provides a systematic approach to selecting RBF centers, which is far superior to random selection. The paper concludes that the OLS learning algorithm offers a powerful procedure for fitting adequate and parsimonious RBF networks in practical signal processing.This paper proposes an orthogonal least squares (OLS) learning algorithm for radial basis function (RBF) networks, which addresses the limitations of traditional methods that rely on random selection of RBF centers. The OLS algorithm selects RBF centers one by one in a rational way, maximizing the increment to the explained variance or energy of the desired output. This approach avoids numerical ill-conditioning and provides a simple and efficient means for fitting RBF networks. The algorithm is illustrated using examples from two signal processing applications: modeling time series data and communications channel equalization. The RBF network is a special two-layer network that is linear in the parameters by fixing the RBF centers and nonlinearities in the hidden layer. The output layer implements a linear combiner, and the weights of this combiner can be determined using the linear least squares method. The choice of the nonlinearity in the RBF network is not crucial for performance, but the selection of centers is critical. The OLS algorithm selects centers from the data points in a forward regression manner, maximizing the explained variance of the desired output at each step. This ensures that the selected centers are optimal for the task. In the time series modeling example, an RBF network is used to predict monthly unemployment figures in West Germany. The OLS algorithm selects 55 centers from the data points, resulting in a model that closely matches the actual data. The prediction error is small, and the model is validated using statistical tests. In the communications channel equalization example, an RBF network is used to reconstruct input signals in a noisy environment. The OLS algorithm selects 12 centers, resulting in a network that performs well compared to a randomly selected network. The performance of the OLS-based RBF network is compared to a randomly selected one, and it is shown to offer much better results. The OLS learning algorithm is a block-data algorithm that can be used in adaptive applications where filter parameters need to be updated as each sample is collected. The algorithm is efficient and provides a systematic approach to selecting RBF centers, which is far superior to random selection. The paper concludes that the OLS learning algorithm offers a powerful procedure for fitting adequate and parsimonious RBF networks in practical signal processing.
Reach us at info@study.space