Automatic Model Pruning using NNI Tuners

It’s convenient to implement auto model pruning with NNI compression and NNI tuners

First, model compression with NNI

You can easily compress a model with NNI compression. Take pruning for example, you can prune a pretrained model with L2FilterPruner like this

from nni.algorithms.compression.pytorch.pruning import L2FilterPruner
config_list = [{ 'sparsity': 0.5, 'op_types': ['Conv2d'] }]
pruner = L2FilterPruner(model, config_list)
pruner.compress()

The ‘Conv2d’ op_type stands for the module types defined in default_layers.py for pytorch.

Therefore { 'sparsity': 0.5, 'op_types': ['Conv2d'] }means that all layers with specified op_types will be compressed with the same 0.5 sparsity. When pruner.compress() called, the model is compressed with masks and after that you can normally fine tune this model and pruned weights won’t be updated which have been masked.

Then, make this automatic

The previous example manually chose L2FilterPruner and pruned with a specified sparsity. Different sparsity and different pruners may have different effects on different models. This process can be done with NNI tuners.

Firstly, modify our codes for few lines

import nni
from nni.algorithms.compression.pytorch.pruning import *

params = nni.get_parameters()
sparsity = params['sparsity']
pruner_name = params['pruner']
model_name = params['model']

model, pruner = get_model_pruner(model_name, pruner_name, sparsity)
pruner.compress()

train(model)  # your code for fine-tuning the model
acc = test(model)  # test the fine-tuned model
nni.report_final_results(acc)

Then, define a config file in YAML to automatically tuning model, pruning algorithm and sparsity.

searchSpace:
sparsity:
  _type: choice
  _value: [0.25, 0.5, 0.75]
pruner:
  _type: choice
  _value: ['slim', 'l2filter', 'fpgm', 'apoz']
model:
  _type: choice
  _value: ['vgg16', 'vgg19']
trainingService:
platform: local
trialCodeDirectory: .
trialCommand: python3 basic_pruners_torch.py --nni
trialConcurrency: 1
trialGpuNumber: 0
tuner:
  name: grid

The full example can be found here

Finally, start the searching via

nnictl create -c config.yml