EMNLP '23: Learning Co-Speech Gesture for Multimodal Aphasia Type Detection

This codebase contains the python scripts for the model for the EMNLP 2023. https://arxiv.org/pdf/2310.11710.pdf

Environment & Installation Steps

Python 3.8 & Pytorch 1.12

Run

Execute the following steps in the same environment:

cd Multimodal-Aphasia-Type-Detection_EMNLP_2023 & python main.py

Dataset

• dataset_chunk50_sample.json: Due to ethical concerns, we can only provide a sample dataframe consisting of ASR transcripts and labels.

Ethical Concerns

We sourced the dataset from AphasiaBank with Institutional Review Board (IRB) approval (SKKU2022-11-039) and strictly follow the data sharing guidelines provided by TalkBank, including the Ground Rules for all TalkBank databases based on American Psychological Association Code of Ethics (Association et al., 2002). Additionally, our research adheres to the five core issues outlined by TalkBank in accordance with the General Data Protection Regulation (GDPR) (Regulation, 2018). These issues include addressing commercial purposes, handling scientific data, obtaining informed consent, ensuring deidentification, and maintaining a code of conduct. Considering ethical concerns, we did not use any photographs and personal information of the participants from the AphasiaBank.

---

If our work was helpful in your research, please kindly cite this work:

BIBTEX
will be updated

Acknowledgments

This research was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation (NRF) of Korea (NRF2022S1A5A8054322) and the National Research Foundation of Korea grant funded by the Korea government (MSIT) (No. 2023R1A2C2007625).

Our Lab Site

Data Science & Artificial Intelligence Laboratory (DSAIL) @ Sungkyunkwan University