Details

Autor(en) / Beteiligte

• Kirkpatrick, James
• Pascanu, Razvan
• Rabinowitz, Neil
• Veness, Joel
• Desjardins, Guillaume
• Rusu, Andrei A.
• Milan, Kieran
• Quan, John
• Ramalho, Tiago
• Grabska-Barwinska, Agnieszka
• Hassabis, Demis
• Clopath, Claudia
• Kumaran, Dharshan
• Hadsell, Raia

Titel

REPLY TO HUSZÁR: The elastic weight consolidation penalty is empirically valid

Ist Teil von

• Proceedings of the National Academy of Sciences - PNAS, 2018-03, Vol.115 (11), p.E2498-E2498

Ort / Verlag

United States: National Academy of Sciences

Erscheinungsjahr

2018

Quelle

EZB Electronic Journals Library

Beschreibungen/Notizen

• In their recent work on elastic weight consolidation (EWC), Kirkpatrick et al show that forgetting in neural networks can be alleviated by using a quadratic penalty whose derivation was inspired by Bayesian evidence accumulation. In his letter, Dr Huszar provides an alternative form for this penalty by following the standard work on expectation propagation using the Laplace approximation. He correctly argues that in cases when more than two tasks are undertaken the two forms of the penalty are different. Dr. Huszar also shows that for a toy linear regression problem his expression appears to be better. Here, they appreciate Huszar for pointing out the discrepancy between the standard expectation propagation and EWC in the multitask case.