Self-Supervision Can Be a Good Few-Shot Learner

This is a MindSpore implementation of the ECCV2022 paper Self-Supervision Can Be a Good Few-Shot Learner (UniSiam).

Contents

• Contents
  • UniSiam Description
  • Dataset
  • Environment Requirements
  • Script description

UniSiam Description

Existing few-shot learning (FSL) methods rely on training with a large labeled dataset, which prevents them from leveraging abundant unlabeled data. From an information-theoretic perspective, we propose an effective unsupervised FSL method, learning representations with self-supervision. Following the InfoMax principle, our method learns comprehensive representations by capturing the intrinsic structure of the data. Specifically, we maximize the mutual information (MI) of instances and their representations with a low-bias MI estimator to perform self-supervised pre-training. Rather than supervised pre-training focusing on the discriminable features of the seen classes, our self-supervised model has less bias toward the seen classes, resulting in better generalization for unseen classes. We explain that supervised pre-training and selfsupervised pre-training are actually maximizing different MI objectives. Extensive experiments are further conducted to analyze their FSL performance with various training settings. Surprisingly, the results show that self-supervised pre-training can outperform supervised pre-training under the appropriate conditions. Compared with state-of-the-art FSL methods, our approach achieves comparable performance on widely used FSL benchmarks without any labels of the base classes.

@inproceedings{Lu2022Self,
    title={Self-Supervision Can Be a Good Few-Shot Learner},
    author={Lu, Yuning and Wen, Liangjian and Liu, Jianzhuang and Liu, Yajing and Tian, Xinmei},
    booktitle={European Conference on Computer Vision (ECCV)},
    year={2022}
}

Dataset

• mini-ImageNet
  • download the mini-ImageNet dataset from google drive and unzip it.
  • download the split files of mini-ImageNet which created by Ravi and Larochelle.
  • move the split files to the folder ./split/miniImageNet

Environment Requirements

• Hardware(GPU)
  • Prepare hardware environment with GPU.
• Framework
  • MindSpore 1.7
• For more information, please check the resources below£º
  • MindSpore Tutorials
  • MindSpore Python API

Script description

Run

python ./train.py --data_path [your DATA FOLDER] --dataset [DATASET NAME] --backbone [BACKBONE] [--OPTIONARG]

For example, to train UniSiam model with ResNet-18 backbone and strong data augmentations on mini-ImageNet (V100):

python train.py \
  --dataset miniImageNet \
  --backbone resnet18 \
  --lrd_step \
  --data_path [your mini-imagenet-folder] \
  --save_path [your save-folder]