October 1989 | ANSELM BLUMER, ANDRZEJ EHRENFEUCHT, DAVID HAUSSLER AND MANFRED K. WARMUTH
The paper extends Valiant's learnability model to learning classes of concepts defined by regions in Euclidean space $ E^n $. It shows that the essential condition for distribution-free learnability is the finiteness of the Vapnik–Chervonenkis (VC) dimension, a simple combinatorial parameter of the concept class. The VC dimension determines the complexity and closure properties of learnable classes, and provides necessary and sufficient conditions for feasible learnability. The paper analyzes the sample complexity of learning functions and shows that any function from samples into the concept class that always gives hypotheses consistent with the sample is a learning function with sample complexity bounded by $ m(\epsilon, \delta) = \max(4/\epsilon \log(2/\delta), 8d/\epsilon \log(13/\epsilon)) $, where $ d $ is the VC dimension of the class. The paper also discusses computational feasibility of learning algorithms, distinguishing between polynomial learnability with respect to domain dimension and polynomial learnability with respect to target complexity. It shows that concept classes with finite VC dimension are polynomially learnable, and provides examples of such classes. The paper also discusses the relationship between learning and data compression, and provides bounds on the sample complexity of learning algorithms for various concept classes.The paper extends Valiant's learnability model to learning classes of concepts defined by regions in Euclidean space $ E^n $. It shows that the essential condition for distribution-free learnability is the finiteness of the Vapnik–Chervonenkis (VC) dimension, a simple combinatorial parameter of the concept class. The VC dimension determines the complexity and closure properties of learnable classes, and provides necessary and sufficient conditions for feasible learnability. The paper analyzes the sample complexity of learning functions and shows that any function from samples into the concept class that always gives hypotheses consistent with the sample is a learning function with sample complexity bounded by $ m(\epsilon, \delta) = \max(4/\epsilon \log(2/\delta), 8d/\epsilon \log(13/\epsilon)) $, where $ d $ is the VC dimension of the class. The paper also discusses computational feasibility of learning algorithms, distinguishing between polynomial learnability with respect to domain dimension and polynomial learnability with respect to target complexity. It shows that concept classes with finite VC dimension are polynomially learnable, and provides examples of such classes. The paper also discusses the relationship between learning and data compression, and provides bounds on the sample complexity of learning algorithms for various concept classes.