The paper introduces a new algorithm for linear classification, combining Rosenblatt’s perceptron algorithm with Helmbold and Warmuth’s leave-one-out method. This algorithm, named the *voted-perceptron*, leverages data that are linearly separable with large margins, similar to Vapnik’s maximal-margin classifier. However, it is simpler to implement and more efficient in terms of computation time. The algorithm can also be used in high-dimensional spaces using kernel functions, making it applicable to problems like handwritten digit recognition. Experiments show that the voted-perceptron algorithm performs well, though not as well as maximal-margin classifiers, and offers significant improvements in computation time and programming effort. The paper also discusses the theoretical analysis of the algorithm, providing bounds on the expected generalization error and comparing them to those of SVMs.The paper introduces a new algorithm for linear classification, combining Rosenblatt’s perceptron algorithm with Helmbold and Warmuth’s leave-one-out method. This algorithm, named the *voted-perceptron*, leverages data that are linearly separable with large margins, similar to Vapnik’s maximal-margin classifier. However, it is simpler to implement and more efficient in terms of computation time. The algorithm can also be used in high-dimensional spaces using kernel functions, making it applicable to problems like handwritten digit recognition. Experiments show that the voted-perceptron algorithm performs well, though not as well as maximal-margin classifiers, and offers significant improvements in computation time and programming effort. The paper also discusses the theoretical analysis of the algorithm, providing bounds on the expected generalization error and comparing them to those of SVMs.