Skip to content

Latest commit

 

History

History
81 lines (67 loc) · 3.52 KB

README_usage.md

File metadata and controls

81 lines (67 loc) · 3.52 KB

Module 1: Squeeze with Class BN (Google Drive)

To obtain network (i.e., ResNet) with class-wise batch normalization statistics, a modified ResNet is being forwarded for one epoch.

  • modified ResNet: it is exactly same as ResNet but with Normal BatchNorm2d replaced by ClassAwareBatchNorm2d. It additionally tracks per-class BN statistics and does not have any influence on the performance.
  • modified ResNet is at models/resnet_class.py

To obtain model with class BN, run the following scripts or download from google drive:

cd recover
bash scripts/imagenet1k_forward.sh
bash scripts/tiny_forward.sh
  • Training model on ImageNet-21K is based on this repo.

Module 2: Recover with Class BN (Google Drive)

To recover LPLD-distilled images, run the following script or download from google drive:

cd recover
bash scripts/imagenet1k_recover.sh
bash scripts/tiny_recover.sh
bash scripts/in21k_recover.sh
  • This training script requires the class-wise BN stats from Module 1.

Module 3: Relabel, Prune, and Validate (Google Drive)

Module 3.1: Relabel and Prune

Basic Usage:

cd relabel_and_validate
python generate_soft_label_pruning.py \
    --cfg_yaml [config file]  \
    --train_dir [image path] \
    --fkd_path [label path to save] \
    --pruning_ratio [optional: label pruning ratio]
  • the [config file] should contain information about batch size, augmentation, and etc.
    • the val_dir in [config file] should be replaced with the your own path to the dataset test set.
  • if pruning_ratio is put, full labels will NOT be generated, and this speeds up label generation process.
    • since we perform random pruning, it is equivalent to only generate a few labels.
  • labels are provided in google drive

Script files are provided and integrated in validation.

Module 3.2: Validate

Basic Usage:

python train_FKD_label_pruning_batch.py \
    --cfg_yaml [config file] \
    --model [model] \
    --prune_ratio [pruning ratio] \
    --granularity [epoch/batch] \
    --prune_metric [order/random] \
    --train_dir [image path] \
    --fkd_path [label path to load]
  • validation should share the same [config file], [image path], and [label path] with relabel.
  • granularity is where to prune with epochs or batch (improved label pool).
  • prune_metric is where to prune in a random order.

For convenience, run the following scripts to reproduce results of the main table in the paper:

cd relabel_and_validate
bash scripts/reproduce/main_table_in1k.sh
bash scripts/reproduce/main_table_tiny.sh
bash scripts/reproduce/main_table_in21k.sh

In our paper's implementation, we sample labels from full labels to adpat different pruning ratios. For a specific pruning ratio, you do not need to generate full labels:

cd relabel_and_validate
bash scripts/reproduce/less_label_in1k.sh
bash scripts/reproduce/less_label_tiny.sh
bash scripts/reproduce/less_label_in21k.sh