L1FilterPruner on NNI

Introduction

L1FilterPruner is a general structured pruning algorithm for pruning filters in the convolutional layers.

In ‘PRUNING FILTERS FOR EFFICIENT CONVNETS’, authors Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet and Hans Peter Graf.

L1Filter Pruner prunes filters in the convolution layers

The procedure of pruning m filters from the ith convolutional layer is as follows:

1. For each filter , calculate the sum of its absolute kernel weights
2. Sort the filters by .
3. Prune filters with the smallest sum values and their corresponding feature maps. The kernels in the next convolutional layer corresponding to the pruned feature maps are also removed.
4. A new kernel matrix is created for both the th and th layers, and the remaining kernel weights are copied to the new model.

Experiment

We implemented one of the experiments in ‘PRUNING FILTERS FOR EFFICIENT CONVNETS’ with L1FilterPruner, we pruned VGG-16 for CIFAR-10 to VGG-16-pruned-A in the paper, in which $64%$ parameters are pruned. Our experiments results are as follows:

Model	Error(paper/ours)	Parameters	Pruned
VGG-16	6.75/6.49	1.5x10^7
VGG-16-pruned-A	6.60/6.47	5.4x10^6	64.0%

The experiments code can be found at examples/model_compress