Issue |
ESAIM: COCV
Volume 26, 2020
|
|
---|---|---|
Article Number | 28 | |
Number of page(s) | 20 | |
DOI | https://doi.org/10.1051/cocv/2019011 | |
Published online | 18 March 2020 |
Thresholding gradient methods in Hilbert spaces: support identification and linear convergence
1
CNRS, École Normale Supérieure (DMA),
75005
Paris, France.
2
LCSL, Istituto Italiano di Tecnologia and Massachusetts Institute of Technology,
Cambridge,
MA 02139, USA.
3
DIBRIS, Università degli Studi di Genova,
16146
Genova, Italy.
4
Dipartimento di Matematica, Università degli Studi di Genova,
16146
Genova, Italy.
* Corresponding author: guillaume.garrigos@ens.fr
Received:
3
December
2017
Accepted:
6
March
2019
We study the ℓ1 regularized least squares optimization problem in a separable Hilbert space. We show that the iterative soft-thresholding algorithm (ISTA) converges linearly, without making any assumption on the linear operator into play or on the problem. The result is obtained combining two key concepts: the notion of extended support, a finite set containing the support, and the notion of conditioning over finite-dimensional sets. We prove that ISTA identifies the solution extended support after a finite number of iterations, and we derive linear convergence from the conditioning property, which is always satisfied for ℓ1 regularized least squares problems. Our analysis extends to the entire class of thresholding gradient algorithms, for which we provide a conceptually new proof of strong convergence, as well as convergence rates.
Mathematics Subject Classification: 49M29 / 65J10 / 65J15 / 65J20 / 65J22 / 65K15 / 90C25 / 90C46
Key words: Forward–Backward method / support identification / conditioning / convergence rates
© The authors. Published by EDP Sciences, SMAI 2020
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.