An Inertial Block Majorization Minimization Framework for Nonsmooth Nonconvex Optimization
Le Thi Khanh Hien, Duy Nhat Phan, Nicolas Gillis; 24(18):1−41, 2023.
Abstract
This paper presents TITAN, a new framework called inerTIal block majorizaTion minimizAtioN ramework, designed for solving nonsmooth nonconvex optimization problems. TITAN is the first framework to incorporate the majorization-minimization approach in block-coordinate update methods while also incorporating an inertial force in each step of the block updates. The inertial force is generated using an extrapolation operator that encompasses heavy-ball and Nesterov-type accelerations for block proximal gradient methods. By utilizing various surrogate functions, such as proximal, Lipschitz gradient, Bregman, quadratic, and composite surrogate functions, and by modifying the extrapolation operator, TITAN offers a wide range of inertial block-coordinate update methods. The convergence properties of the TITAN sequence, including sub-sequential convergence and global convergence, are analyzed. Additionally, the effectiveness of TITAN is demonstrated through its application to two important machine learning problems: sparse non-negative matrix factorization and matrix completion.
[abs]