Details

Autor(en) / Beteiligte

• Bieker, Katharina
• Gebken, Bennet
• Peitz, Sebastian

Titel

On the Treatment of Optimization Problems With L1 Penalty Terms via Multiobjective Continuation

Ist Teil von

• IEEE transactions on pattern analysis and machine intelligence, 2022-11, Vol.44 (11), p.7797-7808

Ort / Verlag

IEEE

Erscheinungsjahr

2022

Link zum Volltext

Quelle

IEEE Xplore

Beschreibungen/Notizen

• We present a novel algorithm that allows us to gain detailed insight into the effects of sparsity in linear and nonlinear optimization. Sparsity is of great importance in many scientific areas such as image and signal processing, medical imaging, compressed sensing, and machine learning, as it ensures robustness against noisy data and yields models that are easier to interpret due to the small number of relevant terms. It is common practice to enforce sparsity by adding the <inline-formula><tex-math notation="LaTeX">\ell _1</tex-math> <mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="bieker-ieq1-3114962.gif"/> </inline-formula>-norm as a penalty term. In order to gain a better understanding and to allow for an informed model selection, we directly solve the corresponding multiobjective optimization problem (MOP) that arises when minimizing the main objective and the <inline-formula><tex-math notation="LaTeX">\ell _1</tex-math> <mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="bieker-ieq2-3114962.gif"/> </inline-formula>-norm simultaneously. As this MOP is in general non-convex for nonlinear objectives, the penalty method will fail to provide all optimal compromises. To avoid this issue, we present a continuation method specifically tailored to MOPs with two objective functions one of which is the <inline-formula><tex-math notation="LaTeX">\ell _1</tex-math> <mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="bieker-ieq3-3114962.gif"/> </inline-formula>-norm. Our method can be seen as a generalization of homotopy methods for linear regression problems to the nonlinear case. Several numerical examples - including neural network training - demonstrate our theoretical findings and the additional insight gained by this multiobjective approach.