Pruner in NNI

NNI implements the main part of the pruning algorithm as pruner. All pruners are implemented as close as possible to what is described in the paper (if it has). The following table provides a brief introduction to the pruners implemented in nni, click the link in table to view a more detailed introduction and use cases.

There are two kinds of pruners in NNI, please refer to basic pruner and scheduled pruner for details.


Brief Introduction of Algorithm

Level Pruner

Pruning the specified ratio on each weight element based on absolute value of weight element

L1 Norm Pruner

Pruning output channels with the smallest L1 norm of weights (Pruning Filters for Efficient Convnets) Reference Paper

L2 Norm Pruner

Pruning output channels with the smallest L2 norm of weights

FPGM Pruner

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration Reference Paper

Slim Pruner

Pruning output channels by pruning scaling factors in BN layers(Learning Efficient Convolutional Networks through Network Slimming) Reference Paper

Activation APoZ Rank Pruner

Pruning output channels based on the metric APoZ (average percentage of zeros) which measures the percentage of zeros in activations of (convolutional) layers. Reference Paper

Activation Mean Rank Pruner

Pruning output channels based on the metric that calculates the smallest mean value of output activations

Taylor FO Weight Pruner

Pruning filters based on the first order taylor expansion on weights(Importance Estimation for Neural Network Pruning) Reference Paper

ADMM Pruner

Pruning based on ADMM optimization technique Reference Paper

Linear Pruner

Sparsity ratio increases linearly during each pruning rounds, in each round, using a basic pruner to prune the model.

AGP Pruner

Automated gradual pruning (To prune, or not to prune: exploring the efficacy of pruning for model compression) Reference Paper

Lottery Ticket Pruner

The pruning process used by “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”. It prunes a model iteratively. Reference Paper

Simulated Annealing Pruner

Automatic pruning with a guided heuristic search method, Simulated Annealing algorithm Reference Paper

Auto Compress Pruner

Automatic pruning by iteratively call SimulatedAnnealing Pruner and ADMM Pruner Reference Paper

AMC Pruner

AMC: AutoML for Model Compression and Acceleration on Mobile Devices Reference Paper

Movement Pruner

Movement Pruning: Adaptive Sparsity by Fine-Tuning Reference Paper