Tools for mini-ImageNet Dataset

This repo provides python source code for creating mini-ImageNet dataset from ImageNet and the utils for generating batches during training. This repo is related to our work on few-shot learning: Meta-Transfer Learning.

Summary

• About mini-ImageNet
• Requirements
• Installation
• Usage
• Performance
• Acknowledgement

About mini-ImageNet

The mini-ImageNet dataset was proposed by Vinyals et al. for few-shot learning evaluation. Its complexity is high due to the use of ImageNet images but requires fewer resources and infrastructure than running on the full ImageNet dataset. In total, there are 100 classes with 600 samples of 84×84 color images per class. These 100 classes are divided into 64, 16, and 20 classes respectively for sampling tasks for meta-training, meta-validation, and meta-test.

Please note that the split files in csv_files folder are created by Ravi and Larochelle (GitHub link). Vinyals et al. didn't include their split files for mini-ImageNet when they first released their paper, so Ravi and Larochelle created their own splits. Additional split files are provided here.

Requirements

• Python 2.7 or 3.x
• numpy
• tqdm
• opencv-python
• Pillow

Installation

Install via Gitee:

git clone https://gitee.com/devilmaycry812839668/mini-imagenet-tools.git

Usage

First, you need to download the image source files from ImageNet website. If you already have it, you may use it directly. Some people report the ImageNet website is not working. Here is an alternative download link. Please carefully read the terms for ImageNet before you download it.

Filename: ILSVRC2012_img_train.tar
Size: 138 GB
MD5: 1d675b47d978889d74fa0da5fadfb00e

Then clone the repo:

git clone https://gitee.com/devilmaycry812839668/mini-imagenet-tools.git
cd mini-imagenet-tools

To generate mini-ImageNet dataset from tar file:

python mini_imagenet_generator.py --tar_path [your_path_of_the_ILSVRC2012_img_train.tar] --target_dir [your_dir_for_the_miniImageNet_Dataset_processed_images]

If you want to resize the images to the specified resolution:

python mini_imagenet_generator.py --tar_path [your_path_of_the_ILSVRC2012_img_train.tar] --target_dir [your_dir_for_the_miniImageNet_Dataset_processed_images] --image_resize 100

Examples:

python mini_imagenet_generator.py --tar_path ~/Download/ILSVRC2012_img_train.tar --target_dir ~/project --image_resize 100

Then you will get the miniImageNetDataset which is called ~/project/processed_images The directory of processed_images is the miniImageNetDataset. The details of processed_images directory, below:

processed_images
    train
        n12351235342
            n1235123534200000001.jpg
            n1235123534200000002.jpg
            n1235123534200000003.jpg
            ...
        n32451237390
        n41051297730
        ...
    val
    test

P.S. In default settings, the images will be resized to 84 × 84.

If you don't want to resize the images, you may set --image_resize 0.

To use the MiniImageNetDataLoader class:

from mini_imagenet_dataloader import MiniImageNetDataSet

minidataset = MiniImageNetDataSet(5,5,15,20000,600,800,'~/processed_images/')

train_images, train_labels, test_images, test_labels = minidataset.get_train_batch(100)

print(train_images, train_labels, test_images, test_labels)
print(train_images.shape)
print(train_labels.shape)
print(test_images.shape)
print(test_labels.shape)

for i in range(20000):
    train_images, train_labels, test_images, test_labels = minidataset.get_train_batch(i)
    print(i)

python ~/mini-imagenet-tools/mini_imagenet_dataloader.py --dataset_dir ~/processed_images --way_num 5 --shot_num 5 --episode_test_sample_num 15 --train_episodes_num 20000 --val_episodes_num 600 --test_episodes_num 800

For yout attention

This project is revised by devilmaycry81289668 If you want to look at the origin version, please click: https://github.com/y2l/mini-imagenet-tools/

Performance

Check the SOTA results for mini-ImageNet on this page.

Download Processed Images

Download jpg files (Thanks for the contribution by @vainaijr)

Download tar files

Acknowledgement

Model-Agnostic Meta-Learning

Optimization as a Model for Few-Shot Learning

Meta-Learning for Semi-Supervised Few-Shot Classification

@ChristopherDaw