Details

Autor(en) / Beteiligte

• Lin, Mingbao
• Cao, Liujuan
• Li, Shaojie
• Ye, Qixiang
• Tian, Yonghong
• Liu, Jianzhuang
• Tian, Qi
• Ji, Rongrong

Titel

Filter Sketch for Network Pruning

Ist Teil von

• IEEE transaction on neural networks and learning systems, 2022-12, Vol.33 (12), p.7091-7100

Ort / Verlag

Piscataway: IEEE

Erscheinungsjahr

2022

Quelle

IEEE Electronic Library (IEL)

Beschreibungen/Notizen

• We propose a novel network pruning approach by information preserving of pretrained network weights (filters). Network pruning with the information preserving is formulated as a matrix sketch problem, which is efficiently solved by the off-the-shelf frequent direction method. Our approach, referred to as FilterSketch, encodes the second-order information of pretrained weights, which enables the representation capacity of pruned networks to be recovered with a simple fine-tuning procedure. FilterSketch requires neither training from scratch nor data-driven iterative optimization, leading to a several-orders-of-magnitude reduction of time cost in the optimization of pruning. Experiments on CIFAR-10 show that FilterSketch reduces 63.3% of floating-point operations (FLOPs) and prunes 59.9% of network parameters with negligible accuracy cost for ResNet-110. On ILSVRC-2012, it reduces 45.5% of FLOPs and removes 43.0% of parameters with only 0.69% accuracy drop for ResNet-50. Our code and pruned models can be found at https://github.com/lmbxmu/FilterSketch .