PROXIMAL-PROXIMAL-GRADIENT METHOD

• 来源期刊: 计算数学（英文版）
• 关键词: Proximal-gradient ADMM Finito MISO SAGA Operator splitting First-order methods Distributed optimization Stochastic optimization Almost sure convergence linear convergence.
• 年，卷（期）: 2019,（6）
• 页码范围: 778-812
• 页数: 35页
• 语种: 英文
• DOI: 10.4208/jcm.1906-m2018-0282

摘要

In this paper,we present the proximal-proximal-gradient method (PPG),a novel optimization method that is simple to implement and simple to parallelize.PPG generalizes the proximal-gradient method and ADMM and is applicable to minimization problems written as a sum of many differentiable and many non-differentiable convex functions.The non-differentiable functions can be coupled.We furthermore present a related stochastic variation,which we call stochastic PPG (S-PPG).S-PPG can be interpreted as a generalization of Finito and MISO over to the sum of many coupled non-differentiable convex functions.We present many applications that can benefit from PPG and S-PPG and prove convergence for both methods.We demonstrate the empirical effectiveness of both methods through experiments on a CUDA GPU.A key strength of PPG and S-PPG is,compared to existing methods,their ability to directly handle a large sum of non-differentiable nonseparable functions with a constant stepsize independent of the number of functions.Such non-diminishing stepsizes allows them to be fast.