Generalization error bounds for multiclass sparse linear classifiers
Authors: Tomer Levy, Felix Abramovich; Published in: Journal of Machine Learning Research, 24(151):1−35, 2023.
Abstract
This study focuses on high-dimensional multiclass classification using sparse multinomial logistic regression. Unlike binary classification, multiclass classification involves various notions of sparsity associated with different structural assumptions on the regression coefficients matrix. The authors propose a computationally feasible feature selection procedure based on penalized maximum likelihood with convex penalties that capture specific types of sparsity. They consider global row-wise sparsity, double row-wise sparsity, and low-rank sparsity. By choosing appropriate tuning parameters, the derived plug-in classifiers achieve minimax generalization error bounds in terms of misclassification excess risk within the corresponding classes of multiclass sparse linear classifiers. The developed approach is general and can be adapted to other types of sparsity as well.
[abs]