Details

Autor(en) / Beteiligte

• Wang, Zhongyuan
• Yi, Peng
• Jiang, Kui
• Jiang, Junjun
• Han, Zhen
• Lu, Tao
• Ma, Jiayi

Titel

Multi-Memory Convolutional Neural Network for Video Super-Resolution

Ist Teil von

• IEEE transactions on image processing, 2019-05, Vol.28 (5), p.2530-2544

Ort / Verlag

United States: IEEE

Erscheinungsjahr

2019

Quelle

IEEE/IET Electronic Library (IEL)

Beschreibungen/Notizen

• Video super-resolution (SR) is focused on reconstructing high-resolution frames from consecutive low-resolution (LR) frames. Most previous video SR methods based on convolutional neural networks (CNN) use a direct connection and single-memory module within the network, and thus, they fail to make full use of spatio-temporal complementary information from LR observed frames. To fully exploit spatio-temporal correlations between adjacent LR frames and reveal more realistic details, this paper proposes a multi-memory CNN (MMCNN) for video SR, cascading an optical flow network and an image-reconstruction network. A series of residual blocks engaged in utilizing intra-frame spatial correlations is proposed for feature extraction and reconstruction. Particularly, instead of using a single-memory module, we embed convolutional long short-term memory into the residual block, thus forming a multi-memory residual block to progressively extract and retain inter-frame temporal correlations between the consecutive LR frames. We conduct extensive experiments on numerous testing datasets with respect to different scaling factors. Our proposed MMCNN shows superiority over the state-of-the-art methods in terms of PSNR and visual quality and surpasses the best counterpart method by 1 dB at most.