Image Classification Experiments on Caltech-256 dataset

Results

Trained ternary quantization

Trained ternary quantization [2] is a method of reducing size of a neural network. It replaces all weights in a convolutional layer with +1, 0, -1 and two float scaling factors (one for +1 and one for -1).

Binary-Weights-Networks

Stochastic depth

Entropy regularization

Delegating classifiers

SmoothGrad

SmoothGrad is a method of computing nice sensitivity maps. Sensitivity maps show which image pixels influence class predictions. They require to compute gradients with respect to an input image. Here are a few examples:

Implementation details

Notes

• I often use cosine annealing [6] of the learning rate. I believe that this reduces training time a lot.

Requirements

• Python 3.5
• pytorch 0.2
• torchvision, Pillow
• numpy, pandas, sklearn, matplotlib, tqdm

References

[1] Regularizing Neural Networks by Penalizing Confident Output Distributions

[2] Trained Ternary Quantization

[3] Deep Networks with Stochastic Depth

[4] SmoothGrad: removing noise by adding noise

[5] XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

[6] Snapshot Ensembles: Train 1, get M for free