Volume 27, 2021Regular articles published in advance of the transition of the journal to Subscribe to Open (S2O). Free supplement sponsored by the Fonds National pour la Science Ouverte
|Number of page(s)||22|
|Published online||01 March 2021|
A consensus-based global optimization method for high dimensional machine learning problems
Mathematical Institute, University of Oxford,
OX2 6GG, UK.
2 School of Mathematical Sciences, Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai 200240, PR China.
3 Department of Mathematics, Stanford University, California 94305, USA.
* Corresponding author: firstname.lastname@example.org
Accepted: 15 July 2020
We improve recently introduced consensus-based optimization method, proposed in [R. Pinnau, C. Totzeck, O. Tse, S. Martin, Math. Models Methods Appl. Sci. 27 (2017) 183–204], which is a gradient-free optimization method for general non-convex functions. We first replace the isotropic geometric Brownian motion by the component-wise one, thus removing the dimensionality dependence of the drift rate, making the method more competitive for high dimensional optimization problems. Secondly, we utilize the random mini-batch ideas to reduce the computational cost of calculating the weighted average which the individual particles tend to relax toward. For its mean-field limit – a nonlinear Fokker-Planck equation – we prove, in both time continuous and semi-discrete settings, that the convergence of the method, which is exponential in time, is guaranteed with parameter constraints independent of the dimensionality. We also conduct numerical tests to high dimensional problems to check the success rate of the method.
Mathematics Subject Classification: 60H35 / 70F10
Key words: Global optimization / high dimensional optimization / consensus-based optimization / random batch method
© EDP Sciences, SMAI 2021
Initial download of the metrics may take a while.