Pruner in NNI

NNI implements the main part of the pruning algorithm as pruner. All pruners are implemented as close as possible to what is described in the paper (if it has). The following table provides a brief introduction to the pruners implemented in nni, click the link in table to view a more detailed introduction and use cases.

There are two kinds of pruners in NNI, please refer to basic pruner and scheduled pruner for details.

Name	Brief Introduction of Algorithm
Level Pruner	Pruning the specified ratio on each weight element based on absolute value of weight element
L1 Norm Pruner	Pruning output channels with the smallest L1 norm of weights (Pruning Filters for Efficient Convnets) Reference Paper
L2 Norm Pruner	Pruning output channels with the smallest L2 norm of weights
FPGM Pruner	Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration Reference Paper
Slim Pruner	Pruning output channels by pruning scaling factors in BN layers(Learning Efficient Convolutional Networks through Network Slimming) Reference Paper
Activation APoZ Rank Pruner	Pruning output channels based on the metric APoZ (average percentage of zeros) which measures the percentage of zeros in activations of (convolutional) layers. Reference Paper
Activation Mean Rank Pruner	Pruning output channels based on the metric that calculates the smallest mean value of output activations
Taylor FO Weight Pruner	Pruning filters based on the first order taylor expansion on weights(Importance Estimation for Neural Network Pruning) Reference Paper
ADMM Pruner	Pruning based on ADMM optimization technique Reference Paper
Linear Pruner	Sparsity ratio increases linearly during each pruning rounds, in each round, using a basic pruner to prune the model.
AGP Pruner	Automated gradual pruning (To prune, or not to prune: exploring the efficacy of pruning for model compression) Reference Paper
Lottery Ticket Pruner	The pruning process used by “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”. It prunes a model iteratively. Reference Paper
Simulated Annealing Pruner	Automatic pruning with a guided heuristic search method, Simulated Annealing algorithm Reference Paper
Auto Compress Pruner	Automatic pruning by iteratively call SimulatedAnnealing Pruner and ADMM Pruner Reference Paper
AMC Pruner	AMC: AutoML for Model Compression and Acceleration on Mobile Devices Reference Paper
Movement Pruner	Movement Pruning: Adaptive Sparsity by Fine-Tuning Reference Paper