ResMLP

ResMLP: Feedforward networks for image classification with data-efficient training, arxiv

PaddlePaddle training/validation code and pretrained models for ResMLP.

The official and 3rd party pytorch implementation are here and here.

This implementation is developed by PPViT.

ResMLP Model Overview

Update

• Update (2022-03-31): Code is refactored.
• Update (2020-09-27): Model FLOPs and # params are uploaded.
• Update (2020-09-24): Update new ResMLP weights.
• Update (2020-09-23): Add new ResMLP weights.
• Update (2020-08-11): Code is released and ported weights are uploaded.

Models Zoo

Original:

Model	Acc@1	Acc@5	#Params	FLOPs	Image Size	Crop_pct	Interpolation	Link
resmlp_24_224	79.38	94.55	30.0M	6.0G	224	0.875	bicubic	google/baidu
resmlp_36_224	79.77	94.89	44.7M	9.0G	224	0.875	bicubic	google/baidu
resmlp_big_24_224	81.04	95.02	129.1M	100.7G	224	0.875	bicubic	google/baidu
resmlp_12_distilled_224	77.95	93.56	15.3M	3.0G	224	0.875	bicubic	google/baidu
resmlp_24_distilled_224	80.76	95.22	30.0M	6.0G	224	0.875	bicubic	google/baidu
resmlp_36_distilled_224	81.15	95.48	44.7M	9.0G	224	0.875	bicubic	google/baidu
resmlp_big_24_distilled_224	83.59	96.65	129.1M	100.7G	224	0.875	bicubic	google/baidu
resmlp_big_24_22k_224	84.40	97.11	129.1M	100.7G	224	0.875	bicubic	google/baidu

*The results are evaluated on ImageNet2012 validation set.

Note: ResMLP weights are ported from timm and facebookresearch

Data Preparation

ImageNet2012 dataset is used in the following file structure:

│imagenet/
├──train_list.txt
├──val_list.txt
├──train/
│  ├── n01440764
│  │   ├── n01440764_10026.JPEG
│  │   ├── n01440764_10027.JPEG
│  │   ├── ......
│  ├── ......
├──val/
│  ├── n01440764
│  │   ├── ILSVRC2012_val_00000293.JPEG
│  │   ├── ILSVRC2012_val_00002138.JPEG
│  │   ├── ......
│  ├── ......

• train_list.txt: list of relative paths and labels of training images. You can download it from: google/baidu
• val_list.txt: list of relative paths and labels of validation images. You can download it from: google/baidu

Usage

To use the model with pretrained weights, download the .pdparam weight file and change related file paths in the following python scripts. The model config files are located in ./configs/.

For example, assume weight file is downloaded in ./resmlp_12_224.pdparams, to use the resmlp_12_224 model in python:

from config import get_config
from resmlp import build_resmlp as build_model
# config files in ./configs/
config = get_config('./configs/resmlp_12_224.yaml')
# build model
model = build_model(config)
# load pretrained weights
model_state_dict = paddle.load('./resmlp_12_224.pdparams')
model.set_state_dict(model_state_dict)

Evaluation

To evaluate model performance on ImageNet2012, run the following script using command line:

sh run_eval_multi.sh

or

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
python main_multi_gpu.py \
-cfg='./configs/resmlp_12_224.yaml' \
-dataset='imagenet2012' \
-batch_size=256 \
-data_path='/dataset/imagenet' \
-eval \
-pretrained='./resmlp_12_224.pdparams' \
-amp

Note: if you have only 1 GPU, change device number to CUDA_VISIBLE_DEVICES=0 would run the evaluation on single GPU.

Training

To train the model on ImageNet2012, run the following script using command line:

sh run_train_multi.sh

or

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
python main_multi_gpu.py \
-cfg='./configs/resmlp_12_224.yaml' \
-dataset='imagenet2012' \
-batch_size=256 \
-data_path='/dataset/imagenet' \
-amp

Note: it is highly recommanded to run the training using multiple GPUs / multi-node GPUs.

Reference

@article{touvron2021resmlp,
  title={Resmlp: Feedforward networks for image classification with data-efficient training},
  author={Touvron, Hugo and Bojanowski, Piotr and Caron, Mathilde and Cord, Matthieu and El-Nouby, Alaaeldin and Grave, Edouard and Joulin, Armand and Synnaeve, Gabriel and Verbeek, Jakob and J{\'e}gou, Herv{\'e}},
  journal={arXiv preprint arXiv:2105.03404},
  year={2021}
}