HAR-DVS

This project is a part of our work, "Dynamic Vision Sensors for Human Activity Recognition" - Stefanie Anna Baby, Bimal Vinod, Chaitanya Chinni, Kaushik Mitra, accepted at the 4th IAPR Asian Conference on Pattern Recognition (ACPR) 2017.

The links to our work can be found at IEEE, arXiv and webpage.

IITM DVS128 Gesture Dataset

The IITM DVS128 Gesture Dataset contains 10 hand gestures by 12 subjects with 10 sets each totalling to a 1200 hand gestures. These gestures are captured using a DVS128 camera.

The aedat dataset and the corresponding converted avi dataset can be downloaded from IITM_DVS_10.

Folder structure

.
├── lib
│   ├── dense_trajectory_release_v1.2
│   │   ├── ...
│   │   └── ...
│   ├── bov_encode.m
│   ├── generate_codebook.m
│   ├── generate_motion_maps.m
│   ├── groupfile_indices.m
│   ├── groupfile_indices_mm.m
│   ├── hidden_indices.m
│   ├── normalize_image.m
│   ├── parsave.m
│   ├── svm_loo.m
│   └── run_dense.sh
├── README.md
├── extract_features.m
├── run_dvs.m
└── startup.m

The lib folder contains a bunch of utility snippets used by the main code files.

The lib/dense_trajectory_release_v1.2 code is taken from http://lear.inrialpes.fr/people/wang/dense_trajectories which was from the work by "Action Recognition by Dense Trajectories" by Wang et al.. More details can be found in the README file associated with it.

Adding the datasets

The data folder should contain all the data in the root directory with the following structure. The extracted DVS data should be put in data/<dataset_name>/original_data with individual folders for each class.

The code expects .avi files and not aedat. All the features (motion-maps and dense-trajectories) are extracted by extract_features.m into the data/<dataset_name>/features_extracted folder.

The run_dvs.m program automatically segregates the test and train data into data/<dataset_name>/encoded_data for K-fold cross-validation.

The same structure is followed for any other dataset. The <dataset_name> can be IITM_DVS_10 or UCF11_DVS etc.

.
├── data
│   ├── IITM_DVS_10
│   │   ├── original_data
│   │   │   ├── comeHere
│   │   │   │   └── *.avi
│   │   │   ├── left_swipe
│   │   │   │   └── *.avi
│   │   │   └── ...
│   │   ├── features_extracted
│   │   │   └── ...
│   │   └── encoded_data
│   │   │   ├── test
│   │   │   └── train
│   ├── UCF11_DVS
│   │   ├── ...
│   │   └── ...
│   └── ...
└── ...

Citation

If you find our work helpful in your publications or projects, please consider citing our paper.

S. A. Baby and B. Vinod and C. Chinni and K. Mitra, "Dynamic Vision Sensors for Human Activity Recognition," 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), Nanjing, China, 2017.

@inproceedings{sababy:hardvs:2017,
    author={S. A. {Baby} and B. {Vinod} and C. {Chinni} and K. {Mitra}},
    booktitle={2017 4th IAPR Asian Conference on Pattern Recognition (ACPR)},
    title={Dynamic Vision Sensors for Human Activity Recognition},
    year={2017},
    pages={316-321},
    doi={10.1109/ACPR.2017.136},
    ISSN={2327-0985},
    month={Nov}
}

Credits

Thanks to Heng Wang for the dense-trajectories code - http://lear.inrialpes.fr/people/wang/dense_trajectories