This paper proposes MetaOptNet, a meta-learning approach for few-shot learning that uses differentiable convex optimization. The method employs linear classifiers as base learners to learn feature embeddings that generalize well under a linear classification rule for novel categories. The key idea is to exploit the convex nature of linear classifiers, allowing efficient optimization through implicit differentiation of optimality conditions and the dual formulation of the optimization problem. This enables the use of high-dimensional embeddings with improved generalization at a modest computational cost. The approach achieves state-of-the-art performance on several few-shot learning benchmarks, including miniImageNet, tieredImageNet, CIFAR-FS, and FC100.
The method is based on a meta-learning framework where the goal is to learn an embedding model that minimizes generalization error across a distribution of tasks. The base learner is a linear classifier, such as a support vector machine (SVM) or ridge regression, which is optimized using a differentiable quadratic programming (QP) solver. This allows end-to-end learning of the embedding model with various linear classifiers. The dual formulation of the optimization problem is used to reduce the number of optimization variables, making the approach computationally efficient.
The paper evaluates the performance of different base learners on several few-shot learning benchmarks. It shows that linear classifiers, such as SVMs, outperform nearest-neighbor classifiers in the few-shot regime, especially when high-dimensional feature embeddings are available. The method is also shown to be efficient, with the QP solver's computational cost being comparable to that of simpler base learners. The results demonstrate that regularized linear models allow for higher embedding dimensions with reduced overfitting, making them well-suited for few-shot learning tasks. The approach is also shown to be effective in reducing meta-overfitting through data augmentation and regularization techniques. Overall, the method provides a practical and efficient solution for few-shot learning, achieving state-of-the-art performance on several benchmarks.This paper proposes MetaOptNet, a meta-learning approach for few-shot learning that uses differentiable convex optimization. The method employs linear classifiers as base learners to learn feature embeddings that generalize well under a linear classification rule for novel categories. The key idea is to exploit the convex nature of linear classifiers, allowing efficient optimization through implicit differentiation of optimality conditions and the dual formulation of the optimization problem. This enables the use of high-dimensional embeddings with improved generalization at a modest computational cost. The approach achieves state-of-the-art performance on several few-shot learning benchmarks, including miniImageNet, tieredImageNet, CIFAR-FS, and FC100.
The method is based on a meta-learning framework where the goal is to learn an embedding model that minimizes generalization error across a distribution of tasks. The base learner is a linear classifier, such as a support vector machine (SVM) or ridge regression, which is optimized using a differentiable quadratic programming (QP) solver. This allows end-to-end learning of the embedding model with various linear classifiers. The dual formulation of the optimization problem is used to reduce the number of optimization variables, making the approach computationally efficient.
The paper evaluates the performance of different base learners on several few-shot learning benchmarks. It shows that linear classifiers, such as SVMs, outperform nearest-neighbor classifiers in the few-shot regime, especially when high-dimensional feature embeddings are available. The method is also shown to be efficient, with the QP solver's computational cost being comparable to that of simpler base learners. The results demonstrate that regularized linear models allow for higher embedding dimensions with reduced overfitting, making them well-suited for few-shot learning tasks. The approach is also shown to be effective in reducing meta-overfitting through data augmentation and regularization techniques. Overall, the method provides a practical and efficient solution for few-shot learning, achieving state-of-the-art performance on several benchmarks.