Details

Autor(en) / Beteiligte

• Saito, Masaki
• Saito, Shunta
• Koyama, Masanori
• Kobayashi, Sosuke

Titel

Train Sparsely, Generate Densely: Memory-Efficient Unsupervised Training of High-Resolution Temporal GAN

Ist Teil von

• International journal of computer vision, 2020-11, Vol.128 (10-11), p.2586-2606

Ort / Verlag

New York: Springer US

Erscheinungsjahr

2020

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• Training of generative adversarial network (GAN) on a video dataset is a challenge because of the sheer size of the dataset and the complexity of each observation. In general, the computational cost of training GAN scales exponentially with the resolution. In this study, we present a novel memory efficient method of unsupervised learning of high-resolution video dataset whose computational cost scales only linearly with the resolution. We achieve this by designing the generator model as a stack of small sub-generators and training the model in a specific way. We train each sub-generator with its own specific discriminator. At the time of the training, we introduce between each pair of consecutive sub-generators an auxiliary subsampling layer that reduces the frame-rate by a certain ratio. This procedure can allow each sub-generator to learn the distribution of the video at different levels of resolution. We also need only a few GPUs to train a highly complex generator that far outperforms the predecessor in terms of inception scores.