Details

Autor(en) / Beteiligte

• Hasannasab, Marzieh
• Hertrich, Johannes
• Neumayer, Sebastian
• Plonka, Gerlind
• Setzer, Simon
• Steidl, Gabriele

Titel

Parseval Proximal Neural Networks

Ist Teil von

• The Journal of fourier analysis and applications, 2020-08, Vol.26 (4), Article 59

Ort / Verlag

New York: Springer US

Erscheinungsjahr

2020

Link zum Volltext

Quelle

SpringerLink

Beschreibungen/Notizen

• The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable tight frame proximal neural networks. Let H and K be real Hilbert spaces, b ∈ K and T ∈ B ( H , K ) a linear operator with closed range and Moore–Penrose inverse T † . Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator Prox : K → K the operator T † Prox ( T · + b ) is a proximity operator on H equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator Prox = S λ : ℓ 2 → ℓ 2 and any frame analysis operator T : H → ℓ 2 that the frame shrinkage operator T † S λ T is a proximity operator on a suitable Hilbert space. The concatenation of proximity operators on R d equipped with different norms establishes a PNN. If the network arises from tight frame analysis or synthesis operators, then it forms an averaged operator. In particular, it has Lipschitz constant 1 and belongs to the class of so-called Lipschitz networks, which were recently applied to defend against adversarial attacks. Moreover, due to its averaging property, PNNs can be used within so-called Plug-and-Play algorithms with convergence guarantee. In case of Parseval frames, we call the networks Parseval proximal neural networks (PPNNs). Then, the involved linear operators are in a Stiefel manifold and corresponding minimization methods can be applied for training of such networks. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.