# affective-computing

**Repository Path**: tom_ud/affective-computing

## Basic Information

- **Project Name**: affective-computing
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-10-23
- **Last Updated**: 2024-10-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README


Our survey 'Recent Trends of Multimodal Affective Computing: A Survey from NLP Perspective' has released
link: https://arxiv.org/abs/2409.07388


# Multimodal Affective Computing Papers #

## We summarize the related work for the following tasks:
* Multimodal Sentiment Analysis
* Multimodal Emotion Recognition in Conversation
* Multimodal Aspect-based Sentiment Analysis
* Multimodal Multi-label Emotion Recognition
* Multimodal Emotion-cause Pair Extraction in Conversation
  
**We will release a review of multimodal affective computing soon**

----------------

>#### _Multimodal Sentiment Analysis (MSA)_ ####
>> ### Research ###
* [1] Huisheng Mao, Baozheng Zhang, Hua Xu, Ziqi Yuan, Yihe Liu. [[_Robust-MSA: Understanding the Impact of Modality Noise
on Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2211.13484)      

* [2] S Afzal, HA Khan, IU Khan, MJ Piran…. [[_A Comprehensive Survey on Affective Computing_ Challenges, Trends, Applications, and Future Directions_]](https://arxiv.org/abs/2305.07665)      

* [3] J Peng, T Wu, W Zhang, F Cheng, S Tan, F Yi…. [[_A fine-grained modal label-based multi-stage network for multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S0957417423002221)      

* [4] Y Zhi, J Li, H Wang, J Chen…. [[_A Fine-Grained Tri-Modal Interaction Model for Multimodal Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/10447872/)      

* [5] J LIU, H SONG, DP CHEN, B WANG, ZW ZHANG. [[_A Multimodal Sentiment Analysis Model Enhanced with Non-verbal Information and Contrastive Learning_]](https://jeit.ac.cn/en/article/doi/10.11999/JEIT231274)      

* [6] X Sun, X Ren, X Xie. [[_A NOVEL MULTIMODAL SENTIMENT ANALYSIS MODEL BASED ON GATED FUSION AND MULTI-TASK LEARNING_]](https://ieeexplore.ieee.org/abstract/document/10446040/)      

* [7] MM Amin, R Mao, E Cambria, BW Schuller. [[_A Wide Evaluation of ChatGPT on Affective Computing Tasks_]](https://arxiv.org/abs/2308.13911)      

* [8] AAB Zadeh, PP Liang, S Poria, E Cambria…. [[_CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph_]](https://aclanthology.org/P18-1208/)      

* [9] W Yu, H Xu, F Meng, Y Zhu, Y Ma, J Wu…. [[_A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotations of Modality_]](https://aclanthology.org/2020.acl-main.343/)      

* [10] K Kim, S Park. [[_All-modalities-in-One BERT for multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S1566253522002329)      

* [11] Hui Ma, Jian Wang, Hongfei Lin, Bo Zhang, Yijia Zhang, Bo Xu. [[_A_Transformer-Based_Model_With_Self-Distillation_for_Multimodal_Emotion_Recognition_in_Conversations_]](https://ieeexplore.ieee.org/document/10109845)      

* [12] J Tang, D Liu, X Jin, Y Peng, Q Zhao…. [[_Bi-Direction Attention Based Fusion Network for Multimodal Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/9932611/)      

* [13] Z Li, B Xu, C Zhu, T Zhao. [[_CLMLF-A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection_]](https://arxiv.org/abs/2204.05515)      

* [14] J Yang, Y Yu, D Niu, W Guo, Y Xu. [[_Contrastive Feature Decomposition for Multimodal Sentiment Analysis_]](https://aclanthology.org/2023.acl-long.421/)      

* [15] Y Yu, M Zhao, S Qi, F Sun, B Wang, W Guo…. [[_Contrastive Knowledge Injection for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2306.15796)      

* [16] DS Chauhan, MS Akhtar, A Ekbal…. [[_Context-aware Interactive Attention for Multi-modal Sentiment and Emotion Analysis_]](https://aclanthology.org/D19-1566/)      

* [17] M Huang, C Qing, J Tan, X Xu. [[_Context-Based Adaptive Multimodal Fusion Network for Continuous Frame-Level Sentiment Prediction_]](https://ieeexplore.ieee.org/abstract/document/10271721/)      

* [18] Y Zeng, W Yan, S Mai, H Hu. [[_Disentanglement Translation Network for multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S1566253523003470)      

* [19] S Mai, H Hu, S Xing. [[_Hierarchical Feature Fusion Network with Local and Global Perspectives for Multimodal Affective Computing_]](https://aclanthology.org/P19-1046/)      

* [20] Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih. [[_Multimodal Phased Transformer for Sentiment Analysis_]](https://aclanthology.org/2021.emnlp-main.189/)      

* [21] A Martins, R Astudillo. [[_A Sparse Model of Attention and Multi-Label Classification_]](https://proceedings.mlr.press/v48/martins16)      

* [22] H Liang, W Xie, X He, S Song…. [[_GUIDED CIRCULAR DECOMPOSITION AND CROSS-MODAL RECOMBINATION FOR MULTIMODAL SENTIMENT ANALYSIS_]](https://ieeexplore.ieee.org/abstract/document/10446166/)      

* [23] S Mai, Y Zeng, S Zheng, H Hu. [[_Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/9767560/)      

* [24] X Zhao, Y Chen, W Li, L Gao…. [[_An Extended Multimodal Adaptation Gate for Multimodal Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/9746536/)      

* [25] D Hazarika, S Poria, R Mihalcea…. [[_Interactive Conversational Memory Network for Multimodal Emotion Detection_]](https://aclanthology.org/D18-1280/)      

* [26] C Busso, M Bulut, CC Lee, A Kazemzadeh…. [[_Interactive emotional dyadic motion capture database_]](https://link.springer.com/article/10.1007/s10579-008-9076-6)      

* [27] W Han, H Chen, S Poria. [[_Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2109.00412)      

* [28] W Rahman, MK Hasan, S Lee, A Zadeh…. [[_Integrating Multimodal Information in Large Pretrained Transformers_]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8005298/)      

* [29] C Chen, H Hong, J Guo, B Song. [[_Inter-Intra Modal Representation Augmentation With Trimodal Collaborative Disentanglement Network for Multimodal Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/10089492/)      

* [30] VW Liang, Y Zhang, Y Kwon…. [[_Understanding the Modality Gap in Multi-modal Contrastive Representation Learning_]](https://proceedings.neurips.cc/paper_files/paper/2022/hash/702f4db7543a7432431df588d57bc7c9-Abstract-Conference.html)      

* [31] D Hazarika, R Zimmermann, S Poria. [[_Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis_]](https://dl.acm.org/doi/abs/10.1145/3394171.3413678)      

* [32] J Zeng, J Zhou, T Liu. [[_Mitigating Inconsistencies in Multimodal Sentiment Analysis under Uncertain Missing Modalities_]](https://aclanthology.org/2022.emnlp-main.189/)      

* [33] K Yang, H Xu, K Gao. [[_Cross-Modal BERT for Text-Audio Sentiment Analysis_]](https://dl.acm.org/doi/abs/10.1145/3394171.3413690)      

* [34] S Mai, H Hu, S Xing. [[_Modality to Modality Translation An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion_]](https://aaai.org/ojs/index.php/AAAI/article/view/5347)      

* [35] Z Liu, B Zhou, D Chu, Y Sun, L Meng. [[_Modality translation-based multimodal sentiment analysis under uncertain missing modalities_]](https://www.sciencedirect.com/science/article/pii/S1566253523002890)      

* [36] Z Lin, B Liang, Y Long, Y Dang…. [[_Hierarchical Graph Contrastive Learning for Multimodal Sentiment Analysis_]](https://repository.essex.ac.uk/34855/)      

* [37] A Zadeh, R Zellers, E Pincus, LP Morency. [[_Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos_]](https://arxiv.org/abs/1606.06259)      

* [38] J Yang, Y Wang, R Yi, Y Zhu, A Rehman…. [[_Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences_]](https://arxiv.org/abs/2010.11985)      

* [39] L Xiao, X Wu, W Wu, J Yang…. [[_MULTI-CHANNEL ATTENTIVE GRAPH CONVOLUTIONAL NETWORK WITH SENTIMENT FUSION FOR MULTIMODAL SENTIMENT ANALYSIS_]](https://ieeexplore.ieee.org/abstract/document/9747542/)      

* [40] J Zheng, S Zhang, Z Wang, X Wang…. [[_Multi-Channel Weight-Sharing Autoencoder Based on Cascade Multi-Head Attention for Multimodal Emotion Recognition_]](https://ieeexplore.ieee.org/abstract/document/9693238/)      

* [41] L Fang, G Liu, R Zhang. [[_MULTI-GRAINED MULTIMODAL INTERACTION NETWORK FOR SENTIMENT ANALYSIS_]](https://ieeexplore.ieee.org/abstract/document/10446351/)      

* [42] S Poria, E Cambria, D Hazarika…. [[_Multi-level Multiple Attentions for Contextual Multimodal Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/8215597/)      

* [43] MS Akhtar, DS Chauhan, D Ghosal, S Poria…. [[_Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis_]](https://arxiv.org/abs/1905.05812)      

* [44] R Lin, H Hu. [[_Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2210.14556)      

* [45] PP Liang, Z Liu, A Zadeh, LP Morency. [[_Multimodal Language Analysis with Recurrent Multistage Fusion_]](https://arxiv.org/abs/1808.03920)      

* [46] Z Wu, Z Gong, J Koo, J Hirschberg. [[_Multimodal Multi-loss Fusion Network for Sentiment Analysis_]](https://aclanthology.org/2024.naacl-long.197/)      

* [47] J Cheng, I Fostiropoulos, B Boehm…. [[_Multimodal Phased Transformer for Sentiment Analysis_]](https://aclanthology.org/2021.emnlp-main.189/)      

* [48] J Zheng, S Zhang, X Wang, Z Zeng. [[_Multimodal Representations Learning Based on Mutual Information Maximization and Minimization and Identity Embedding for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2201.03969)      

* [49] Z Li, Q Guo, C Feng, L Deng, Q Zhang…. [[_Multimodal Sentiment Analysis Based on Interactive Transformer and Soft Mapping_]](https://onlinelibrary.wiley.com/doi/abs/10.1155/2022/6243347) 

* [50] M Chen, S Wang, PP Liang, T Baltrušaitis…. [[_Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning_]](https://dl.acm.org/doi/abs/10.1145/3136755.3136801)      

* [51] R Kaur, S Kautish. [[_Multimodal Sentiment Analysis: A Survey and Comparison_]](https://www.igi-global.com/chapter/multimodal-sentiment-analysis/308579)      

* [52] A Gandhi, K Adhvaryu, S Poria, E Cambria, A Hussain. [[_A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions_]](https://www.sciencedirect.com/science/article/pii/S1566253522001634)      

* [53] X Yang, S Feng, Y Zhang, D Wang. [[_Multimodal Sentiment Detection Based on Multi-channel Graph Neural Networks_]](https://aclanthology.org/2021.acl-long.28/)      

* [54] Huan Deng, Zhenguo Yang, Tianyong Hao, Qing Li, Wenyin Liu. [[_Multimodal_Affective_Computing_With_Dense_Fusion_Transformer_for_Inter-_and_Intra-Modality_Interactions_]](Multimodal_Affective_Computing_With_Dense_Fusion_Transformer_for_Inter-_and_Intra-Modality_Interactions)      

* [55] MS Akhtar, DS Chauhan, D Ghosal, S Poria…. [[_NAACL19_Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis(1)_]](https://arxiv.org/abs/1905.05812)      

* [56] F Wang, S Tian, L Yu, J Liu, J Wang, K Li…. [[_TEDT: Transformer-Based Encoding–Decoding Translation Network for Multimodal Sentiment Analysis_]](https://link.springer.com/article/10.1007/s12559-022-10073-9)      

* [57] H Luo, L Ji, Y Huang, B Wang, S Ji, T Li. [[_Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptors_]](https://arxiv.org/abs/2112.01368)      

* [58] H Yang, X Gao, J Wu, T Gan, N Ding…. [[_Self-adaptive Context and Modal-interaction Modeling For Multimodal Emotion Recognition_]](https://aclanthology.org/2023.findings-acl.390/)      

* [59] F Qian, J Han, Y He, T Zheng…. [[_Sentiment Knowledge Enhanced Self-supervised Learning for Multimodal Sentiment Analysis_]](https://aclanthology.org/2023.findings-acl.821/)      

* [60] Y Wu, Y Zhao, H Yang, S Chen, B Qin, X Cao…. [[_Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors_]](https://arxiv.org/abs/2203.00257)      

* [61] S Lai, J Li, G Guo, X Hu, Y Li, Y Tan, Z Song…. [[_Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning_]](https://arxiv.org/abs/2305.08473)      

* [62] C Zhu, M Chen, S Zhang, C Sun, H Liang, Y Liu…. [[_Sentiment Knowledge Enhanced Attention Fusion Network for multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S1566253523002749)      

* [63] M Chen, X Li. [[_Sentimental Words Aware Fusion Network for Multimodal Sentiment Analysis_]](https://aclanthology.org/2020.coling-main.93/)      

* [64] Y Wei, S Yuan, R Yang, L Shen, Z Li…. [[_Tackling Modality Heterogeneity with Multi-View Calibration Network for Multimodal Sentiment Detection_]](https://aclanthology.org/2023.acl-long.287/)      

* [65] J Zeng, T Liu, J Zhou. [[_Tag-assisted Multimodal Sentiment Analysis under Uncertain_]](https://dl.acm.org/doi/abs/10.1145/3477495.3532064)      

* [66] J Yu, J Wang, R Xia, J Li. [[_Targeted Multimodal Sentiment Classifcation Based on Coarse-to-Fine Grained Image-Target Matching_]](https://www.ijcai.org/proceedings/2022/0622.pdf)      

* [67] M Zhou, W Quan, Z Zhou, K Wang, T Wang…. [[_Text-oriented Cross Attention Network for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2404.04545)      

* [68] C Huang, J Zhang, X Wu, Y Wang, M Li…. [[_Text-centered fusion network with crossmodal attention for multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S0950705123002526)      

* [69] A Zadeh, M Chen, S Poria, E Cambria…. [[_Tensor Fusion Network for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/1707.07250)      

* [70] D Wang, X Guo, Y Tian, J Liu, LH He, X Luo. [[_A text enhanced transformer fusion network for multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S0031320322007385)      

* [71] L Stappen, A Baird, L Schumann…. [[_The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements_]](https://ieeexplore.ieee.org/abstract/document/9484711/)      

* [72] X Zhao, Y Chen, S Liu, X Zang, Y Xiang…. [[_A New Token Mixup Multimodal Data Augmentation for Multimodal Sentiment Analysis_]](https://dl.acm.org/doi/abs/10.1145/3543507.3583406)      

* [73] Y Li, W Weng, C Liu. [[_two-stage contrastive learning and feature hierarchical fusion network for multimodal sentiment analysis_]](https://link.springer.com/article/10.1007/s00521-024-09634-w)      

* [74] Y Zeng, S Mai, H Hu. [[_Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2111.08451)      

* [75] W Yu, H Xu, Z Yuan, J Wu. [[__AAAI21_Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis_]](https://ojs.aaai.org/index.php/AAAI/article/view/17289)      

* [76] Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency, Ruslan Salakhutdinov. [[__Multimodal Transformer for Unaligned Multimodal Language Sequences_]](https://arxiv.org/abs/1906.00295)      

* [77] J Tang, K Li, X Jin, A Cichocki, Q Zhao…. [[_Hierarchical Learning for Multimodal Sentiment Analysis Using Coupled-Translation Fusion Network_]](https://aclanthology.org/2021.acl-long.412/)      

* [78] A Zadeh, YS Cao, S Hessner, PP Liang…. [[_A Multimodal Language Dataset for Spanish, Portuguese, German and French_]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8106386/)      

* [79] W Han, H Chen, S Poria. [[__EMNLP21_Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis_]](https://arxiv.org/abs/2109.00412)   

* [80] Z Yuan, W Li, H Xu, W Yu. [[__MM21_Transformer-based Feature Reconstruction Network for Robust Multimodal Sentiment Analysis_]](https://dl.acm.org/doi/abs/10.1145/3474085.3475585)


>#### _Multimodal Emotion Recognition in Conversation (MERC)_ ####
>> ### Research ###

* [1] S Harata, T Sakuma, S Kato. [[_Audio-Visual Shared Emotion Representation for Robust Emotion Recognition on Modality Missing Using Hemi-hyperspherical Embedding and Latent Space Unification_]](https://link.springer.com/chapter/10.1007/978-3-031-06388-6_18)      

* [2] FA Rahman, G Lu. [[_A Contextualized Real-Time Multimodal Emotion Recognition for Conversational Agents using Graph Convolutional Networks in Reinforcement Learning_]](https://arxiv.org/abs/2310.18363)
  
* [3] W Zheng, J Yu, R Xia, S Wang. [[_A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations_]](https://aclanthology.org/2023.acl-long.861/)      

* [4] Y Zhang, J Wang, Y Liu, L Rong, Q Zheng, D Song…. [[_A Multitask learning model for multimodal sarcasm, sentiment and emotion recognition in conversations_]](https://www.sciencedirect.com/science/article/pii/S1566253523000040)      

* [5] M Sharafi, M Yazdchi, R Rasti, F Nasimi. [[_A novel spatio-temporal convolutional neural framework for multimodal emotion recognition_]](https://www.sciencedirect.com/science/article/pii/S1746809422004694)      

* [6] B Pan, K Hirota, Z Jia, Y Dai. [[_A review of multimodal emotion recognition from datasets, preprocessing, features, and fusion methods_]](https://www.sciencedirect.com/science/article/pii/S092523122300989X)      

* [7] DS Chauhan, GV Singh, A Arora, A Ekbal…. [[_A Sentiment and Emotion aware Multimodal Multiparty Humor Recognition in Multilingual Conversational Setting_]](https://aclanthology.org/2022.coling-1.587/)      
    
* [8] X Li, J Song, Z Zhao, C Wang…. [[_A SUPERVISED INFORMATION ENHANCED MULTI-GRANULARITY CONTRASTIVE LEARNING FRAMEWORK FOR EEG BASED EMOTION RECOGNITION_]](https://ieeexplore.ieee.org/abstract/document/10447740/)      

* [9] H Ma, J Wang, H Lin, B Zhang…. [[_A Transformer-based Model with Self-distillation for Multimodal Emotion Recognition in Conversations_]](https://ieeexplore.ieee.org/abstract/document/10109845/)      

* [10] K Ali, CE Hughes. [[_A Unified Transformer-based Network for Multimodal Emotion Recognition_]](https://arxiv.org/abs/2308.14160)      
    
* [11] J Vazquez-Rodriguez, G Lefebvre…. [[_ACCOMMODATING MISSING MODALITIES IN TIME-CONTINUOUS MULTIMODAL EMOTION RECOGNITION_]](https://ieeexplore.ieee.org/abstract/document/10388079/)      

* [12] S Poria, D Hazarika, N Majumder, G Naik…. [[_A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations_]](https://arxiv.org/abs/1810.02508)      

* [13] JA Miranda-Correa, MK Abadi…. [[_A Dataset for Affect, Personality and Mood Research on Individuals and Groups_]](https://ieeexplore.ieee.org/abstract/document/8554112/)      

* [14] NK Devulapally, S Anand, SD Bhattacharjee…. [[_Adaptive Multimodal Analysis for Speaker Emotion Recognition in Group Conversations_]](https://arxiv.org/abs/2401.15164)      

* [15] Fei Ma, Shao-Lun Huang, Lin Zhang. [[_An_Efficient_Approach_for_Audio-Visual_Emotion_Recognition_With_Missing_Labels_And_Missing_Modalities_]](https://ieeexplore.ieee.org/document/9428219)      

* [16] Hui Ma, Jian Wang, Hongfei Lin, Bo Zhang, Yijia Zhang, Bo Xu. [[_A_Transformer-Based_Model_With_Self-Distillation_for_Multimodal_Emotion_Recognition_in_Conversations_]](https://ieeexplore.ieee.org/document/10109845)      

* [17] A Jia, Y He, Y Zhang, S Uprety, D Song…. [[_A Multi-Modal Dataset for Human Desire Understanding_]](https://aclanthology.org/2022.naacl-main.108/)      

* [18] X Zhang, Y Li. [[_A Cross-Modality Context Fusion and Semantic Refinement Network for Emotion Recognition in Conversation_]](https://aclanthology.org/2023.acl-long.732/)      

* [19] A Joshi, A Bhat, A Jain, A Singh…. [[_COntextualized GNN based Multimodal Emotion recognitioN_]](https://aclanthology.org/2022.naacl-main.306/)      

* [20] Yahui Fu, Shogo Okada, Longbiao Wang, Lili Guo, Yaodong Song. [[_Context-_and_Knowledge-Aware_Graph_Convolutional_Network_for_Multimodal_Emotion_Recognition_]](Context-_and_Knowledge-Aware_Graph_Convolutional_Network_for_Multimodal_Emotion_Recognition)      

* [21] WY Choi, KY Song, CW Lee. [[_Convolutional Attention Networks for Multimodal Emotion Recognition from Speech and Text Data_]](https://aclanthology.org/W18-3304/)      

* [22] Y Wang, Y Li, P Bell, C Lai - researchgate.net. [[_Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition_]](https://www.researchgate.net/profile/Yuanchao-Li-5/publication/370981780_Cross-Attention_is_Not_Enough_Incongruity-Aware_Multimodal_Sentiment_Analysis_and_Emotion_Recognition/links/64a48b888de7ed28ba74a8aa/Cross-Attention-is-Not-Enough-Incongruity-Aware-Multimodal-Sentiment-Analysis-and-Emotion-Recognition.pdf)      

* [23] SAM Zaidi, S Latif, J Qadi. [[_Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers_]](https://arxiv.org/abs/2306.13804)      

* [24] Y Zhang, H Liu, D Wang, D Zhang, T Lou…. [[_Cross-modal credibility modelling for EEG-based multimodal emotion recognition_]](https://iopscience.iop.org/article/10.1088/1741-2552/ad3987/meta)      

* [25] L Xiao, X Wu, S Yang, J Xu, J Zhou, L He. [[_Cross-modal fine-grained alignment and fusion network for multimodal aspect-based sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S0306457323002455)      

* [26] Y Li, Y Wang, Z Cui. [[_Decoupled Multimodal Distilling for Emotion Recognition_]](http://openaccess.thecvf.com/content/CVPR2023/html/Li_Decoupled_Multimodal_Distilling_for_Emotion_Recognition_CVPR_2023_paper.html)      

* [27] F Zhang, XC Li, CP Lim, Q Hua, CR Dong, JH Zhai. [[_Deep Emotional Arousal Network for Multimodal Sentiment Analysis and Emotion Recognition_]](https://www.sciencedirect.com/science/article/pii/S1566253522000653)      

* [28] AI Middya, B Nag, S Roy. [[_Deep learning based multimodal emotion recognition using model-level fusion of audio–visual modalities_]](https://www.sciencedirect.com/science/article/pii/S0950705122002593)
  
* [29] G Arora, M Sabharwal, P Kapila…. [[_Deep Residual Adaptive Neural Network Based Feature Extraction for Cognitive Computing with Multimodal Sentiment Sensing and Emotion Recognition Process_]](https://search.proquest.com/openview/5b82ac981d0318b3a57fe3bd2e737a49/1?pq-origsite=gscholar&cbl=52057)      

* [30] Y Mao, Q Sun, G Liu, X Wang, W Gao, X Li…. [[_Exploring Multi-Modal Emotion Dynamics in Conversations_]](https://arxiv.org/abs/2010.07637)      

* [31] Y Guo, C Tang, H Wu, B Chen. [[_EEG EMOTION RECOGNITION BASED ON DYNAMICAL GRAPH ATTENTION NETWORK_]](https://ieeexplore.ieee.org/abstract/document/10447925/)      
    
* [32] GV Singh, M Firdaus, A Ekbal…. [[_A Multimodal Transformer for Identifying Emotions and Intents in Social Conversations_]](https://ieeexplore.ieee.org/abstract/document/9961847/)      

* [33] K Ezzameli, H Mahersia. [[_Emotion recognition from unimodal to multimodal analysis: A review_]](https://www.sciencedirect.com/science/article/pii/S156625352300163X)      

* [34] J Vazquez-Rodriguez, G Lefebvre…. [[_Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals_]](https://ieeexplore.ieee.org/abstract/document/9953852/)      

* [35] D Andreoletti, F Cardoso, A Arzillo, L Luceri… - 2021 - ceur-ws.org. [[_Emotion_Recognition_and_Detection_Methods_A_Compre_]](https://ceur-ws.org/Vol-3116/Paper_2.pdf)      

* [36] Lili Guo, Longbiao Wang, Jianwu Dang, Yahui Fu, Jiaxing Liu. [[_Emotion_Recognition_With_Multimodal_Transformer_Fusion_Framework_Based_on_Acoustic_and_Lexical_Information_]](https://ieeexplore.ieee.org/document/9740502)      

* [37] MM Islam, S Nooruddin, F Karray…. [[_A deep learning based model-level fusion approach_]](https://www.sciencedirect.com/science/article/pii/S1746809424002994)      

* [38] H Zuo, R Liu, J Zhao, G Gao, H Li. [[_EXPLOITING MODALITY-INVARIANT FEATURE FOR ROBUST MULTIMODAL EMOTION RECOGNITION WITH MISSING MODALITIES_]](https://ieeexplore.ieee.org/abstract/document/10095836/)      

* [39] Haolin Zuo, Rui Liu, Jinming Zhao, Guanglai Gao, Haizhou Li. [[_Exploiting_Modality-Invariant_Feature_for_Robust_Multimodal_Emotion_Recognition_with_Missing_Modalities_]](Exploiting_Modality-Invariant_Feature_for_Robust_Multimodal_Emotion_Recognition_with_Missing_Modalities)      

* [40] Z Fang, A He, Q Yu, B Gao, W Ding, T Zhang…. [[_A novel multimodal emotion recognition approach integrating face, body and text_]](https://arxiv.org/abs/2211.15425)      

* [41] X Yang, S Feng, D Wang, S Qi, W Wu, Y Zhang…. [[_Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt_]](https://arxiv.org/abs/2305.10169)      

* [42] YP Ruan, S Han, T Li, Y Wu. [[_FUSING MODALITY-SPECIFIC REPRESENTATIONS AND DECISIONS FOR MULTIMODAL EMOTION RECOGNITION_]](https://ieeexplore.ieee.org/abstract/document/10447035/)      

* [43] M Jin, J Li. [[_Learning Deep Representations for Multimodal Emotion Recognition_]](https://dl.acm.org/doi/abs/10.1145/3581783.3612074)      

* [44] J Li, X Wang, G Lv, Z Zeng. [[_A graph network based multimodal fusion technique for emotion recognition in conversation_]](https://www.sciencedirect.com/science/article/pii/S0925231223005507)      

* [45] M Sun, X Zhang, J Ma, S Xie, Y Liu…. [[_A Knowledge-guided Dual-inconsistency Network for Multi-modal Rumor Detection_]](https://ieeexplore.ieee.org/abstract/document/10123962/)      

* [46] Haiyang Xu, Hui Zhang, Kun Han, Yun Wang, Yiping Peng, Xiangang Li. [[_Interspeech19_Learning Alignment for Multimodal Emotion Recognition from Speech_]](Interspeech19_Learning Alignment for Multimodal Emotion Recognition from Speech)      

* [47] C Ziems, W Held, O Shaikh, J Chen, Z Zhang…. [[_Is GPT a Computational Model of Emotion__]](https://direct.mit.edu/coli/article/doi/10.1162/coli_a_00502/118498)      

* [48] D Li, Y Wang, K Funakoshi, M Okumura. [[_Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognition_]](https://arxiv.org/abs/2311.11009)      

* [49] J Sun, S Han, YP Ruan, X Zhang…. [[_Layer-wise Fusion with Modality Independence Modeling for Multi-modal Emotion Recognition_]](https://aclanthology.org/2023.acl-long.39/)      

* [50] Z Tang, Q Xiao, X Zhou, Y Li, C Chen, K Li. [[_Learning discriminative multi-relation representations for multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S0020025523007107)      

* [51] P Wang, S Zeng, J Chen, L Fan, M Chen, Y Wu…. [[_Leveraging Label Information for Multimodal Emotion Recognition_]](https://arxiv.org/abs/2309.02106)      

* [52] V Chudasama, P Kar, A Gudmalwar…. [[_Multi-modal Fusion Network for Emotion Recognition in Conversation_]](http://openaccess.thecvf.com/content/CVPR2022W/MULA/html/Chudasama_M2FNet_Multi-Modal_Fusion_Network_for_Emotion_Recognition_in_Conversation_CVPRW_2022_paper.html)      

* [53] N Wang, H Cao, J Zhao, R Chen…. [[_Missing-Modality Robust Emotion Recognition Framework With Iterative Data Augmentation_]](https://ieeexplore.ieee.org/abstract/document/9868120/)      

* [54] Y Zhang, A Jia, B Wang, P Zhang, D Zhao, P Li…. [[_A Multi-modal, Multi-task Interactive Graph Attention Network for Conversational Sentiment Analysis and Emotion Recognition_]](https://dl.acm.org/doi/abs/10.1145/3593583)      

* [55] M Ren, X Huang, J Liu, M Liu, X Li…. [[_Multimodal Adversarial Learning Network for Conversational Emotion Recognition_]](https://ieeexplore.ieee.org/abstract/document/10121331/)      

* [56] M Firdaus, H Chauhan, A Ekbal…. [[_A Multimodal Multi-Label Emotion, Intensity and Sentiment Dialogue Dataset for Emotion Recognition and Sentiment Analysis in Conversations_]](https://aclanthology.org/2020.coling-main.393/)      

* [57] Z Lian, H Sun, L Sun, Z Wen, S Zhang, S Chen…. [[_Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition_]](https://arxiv.org/abs/2404.17113)      

* [58] J Zhao, R Li, Q Jin. [[_Missing Modality Imagination Network for Emotion Recognition with Uncertain Missing Modalities_]](https://aclanthology.org/2021.acl-long.203/)      

* [59] X Chen. [[_RULE-BASED NETWORK FOR MULTIMODAL EMOTION RECOGNITION_]](https://ieeexplore.ieee.org/abstract/document/10447930/)      

* [60] J Zheng, S Zhang, Z Wang, X Wang…. [[_Multi-Channel Weight-Sharing Autoencoder Based on Cascade Multi-Head Attention for Multimodal Emotion Recognition_]](https://ieeexplore.ieee.org/abstract/document/9693238/)      

* [61] T Shi, SL Huang. [[_An Attention-Based Correlation-Aware Multimodal Fusion Framework for Emotion Recognition in Conversations_]](https://aclanthology.org/2023.acl-long.824/)      

* [62] Q Li, Y Gao, C Wang, Y Deng, J Xue…. [[_Multilevel_Transformer_for_Multimodal_Emotion_Recognition_]](https://ieeexplore.ieee.org/abstract/document/10446812/)      

* [63] G Aguilar, V Rozgić, W Wang, C Wang. [[_Multimodal and Multi-view Models for Emotion Recognition_]](https://arxiv.org/abs/1906.10198)      

* [64] J Cheng, I Fostiropoulos, B Boehm…. [[_Multimodal Phased Transformer for Sentiment Analysis_]](https://aclanthology.org/2021.emnlp-main.189/)      

* [65] S Zou, X Huang, X Shen. [[_Multimodal Prompt Transformer with Hybrid Contrastive Learning for Emotion Recognition in Conversation_]](https://dl.acm.org/doi/abs/10.1145/3581783.3611805)   

* [66] X Sun, H He, H Tang, K Zeng…. [[_Multimodal rough set transformer for sentiment analysis and emotion recognition_]](https://ieeexplore.ieee.org/abstract/document/10263177/)      

* [67] L Sun, B Liu, J Tao, Z Lian. [[_Multimodal Speech Emotion Recognition using Cross Attnention with Aligned Audio and Text_]](https://ieeexplore.ieee.org/abstract/document/9414654/)      

* [68] F Chen, J Shao, S Zhu…. [[_Rethinking Graph Neural Networks for Emotion Recognition in Conversation_]](http://openaccess.thecvf.com/content/CVPR2023/html/Chen_Multivariate_Multi-Frequency_and_Multimodal_Rethinking_Graph_Neural_Networks_for_Emotion_CVPR_2023_paper.html)      

* [69] A Joshi, A Bhat, A Jain, A Singh…. [[_COntextualized GNN based Multimodal Emotion recognitioN_]](https://aclanthology.org/2022.naacl-main.306/)      

* [70] Z Li, Y Zhou, Y Liu, F Zhu, C Yang…. [[_A Quantum-Inspired Adaptive-Priority-Learning Model for Multimodal Emotion Recognition_]](https://aclanthology.org/2023.findings-acl.772/)      

* [71] C Zhang, Y Zhang, B Cheng. [[_A REINFORCEMENT LEARNING FRAMEWORK FOR MULTIMODAL EMOTION RECOGNITION_]](https://ieeexplore.ieee.org/abstract/document/10446459/)      

* [72] H Yang, X Gao, J Wu, T Gan, N Ding…. [[_Self-adaptive Context and Modal-interaction Modeling For Multimodal Emotion Recognition_]](https://aclanthology.org/2023.findings-acl.390/)      

* [73] H Chen, C Guo, Y Li, P Zhang, D Jiang. [[_Semi-Supervised Multimodal Emotion Recognition with Class-Balanced Pseudo-Labeling_]](https://dl.acm.org/doi/abs/10.1145/3581783.3612864)      

* [74] U Chinta, J Kalita, A Atyabi. [[_Facial Images and EEG_]](https://ieeexplore.ieee.org/abstract/document/10099070/)      

* [75] B Yao, W Shi. [[_SPEAKER-CENTRIC MULTIMODAL FUSION NETWORKS FOR EMOTION RECOGNITION IN CONVERSATIONS_]](https://ieeexplore.ieee.org/abstract/document/10447720/)      

* [76] D Zhang, F Chen, J Chang, X Chen…. [[_Structure Aware Multi-Graph Network for Multi-Modal Emotion Recognition in Conversations_]](https://ieeexplore.ieee.org/abstract/document/10219015/) 

* [77] X Song, L Huang, H Xue, S Hu. [[_Supervised Prototypical Contrastive Learning for Emotion Recognition in Conversation_]](https://arxiv.org/abs/2210.08713)      

* [78] X Li. [[_Token-channel compounded Cross Attention for Multimodal Emotion Recognition_]](https://arxiv.org/abs/2306.13592)      

* [79] Z Zhao, Y Wang, Y Xu, J Zhang. [[_Transformer-Based Deep-Scale Fusion Network for Multimodal Emotion Recognition_]](https://ieeexplore.ieee.org/abstract/document/10254334/)      

* [80] L Stappen, A Baird, L Christ, L Schumann…. [[_Sentiment, Emotion, Physiological-Emotion, and Stress_]](https://dl.acm.org/doi/abs/10.1145/3475957.3484450)      

* [81] S Qiu, N Sekhar, P Singhal. [[_Topic and Style-aware Transformer for Multimodal Emotion Recognition_]](https://aclanthology.org/2023.findings-acl.130/)      

* [82] G Hu, TE Lin, Y Zhao, G Lu, Y Wu, Y Li. [[_Towards Unified Multimodal Sentiment Analysis and Emotion Recognition_]](https://arxiv.org/abs/2211.11256)      

* [83] D Sun, Y He, J Han. [[_USING AUXILIARY TASKS IN MULTIMODAL FUSION OF WAV2VEC 2.0 AND BERT FOR MULTIMODAL EMOTION RECOGNITION_]](https://ieeexplore.ieee.org/abstract/document/10096586/)   

* [84] J Hu, Y Liu, J Zhao, Q Jin. [[_Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation_]](https://arxiv.org/abs/2107.06779)      

* [85] L Stappen, A Baird, L Christ, L Schumann…. [[_Sentiment, Emotion, Physiological-Emotion, and Stress_]](https://dl.acm.org/doi/abs/10.1145/3475957.3484450) 



>#### _Multimodal Aspect-based Sentiment Analysis (MABSA)_ ####
>> ### Research ###

* [1] H Zhao, M Yang, X Bai, H Liu. [[_A Survey on Multimodal Aspect-Based Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/10401113/)      
    
* [2] C Wang, Y Luo, C Meng, F Yuan. [[_An Adaptive Dual Graph Convolution Fusion Network for Aspect-Based Sentiment Analysis_]](https://dl.acm.org/doi/abs/10.1145/3659579)      

* [3] R Zhou, W Guo, X Liu, S Yu, Y Zhang…. [[_AoM Detecting Aspect-oriented Information for Multimodal_]](https://arxiv.org/abs/2306.01004)      

* [4] L Xiao, X Wu, S Yang, J Xu, J Zhou, L He. [[_Cross-modal fine-grained alignment and fusion network for multimodal aspect-based sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S0306457323002455)      

* [5] L Yang, JC Na, J Yu. [[_Cross-Modal Multitask Transformer for End-to-End Multimodal Aspect-Based Sentiment Analysis_]](https://www.sciencedirect.com/science/article/pii/S0306457322001479) 

* [6] Z Yu, J Wang, LC Yu, X Zhang. [[_Dual-Encoder Transformers with Cross-modal Alignment for Multimodal Aspect-based Sentiment Analysis_]](https://aclanthology.org/2022.aacl-main.32/)      

* [7] Jianfei Yu, Jing Jiang, Rui Xia. [[_Entity-Sensitive_Attention_and_Fusion_Network_for_Entity-Level_Multimodal_Sentiment_Classification_]]((https://ieeexplore.ieee.org/document/8926404))   

* [8] X Yang, S Feng, D Wang, S Qi, W Wu, Y Zhang…. [[_Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt_]](https://arxiv.org/abs/2305.10169)      

* [9] X Yang, S Feng, D Wang, S Qi, W Wu, Y Zhang…. [[_Few-shot joint multimodal aspect-sentiment analysis based on generative_]](https://arxiv.org/abs/2305.10169)      

* [10] Jun Zhao, Fuping Yang. [[_Fusion_with_GCN_and_SE-ResNeXt_Network_for_Aspect_Based_Multimodal_Sentiment_Analysis_]](https://ieeexplore.ieee.org/document/10082618)      

* [11] Junjie Xu, Shuwen Yang, Luwei Xiao, Zhichao Fu, Xingjiao Wu, Tianlong Ma,... . [[_Graph_Convolution_over_the_Semantic-syntactic_Hybrid_Graph_Enhanced_by_Affective_Knowledge_for_Aspect-level_Sentiment_Classification_]](https://ieeexplore.ieee.org/document/9892027)      

* [12] Jianfei Yu, Kai Chen, Rui Xia. [[_Hierarchical_Interactive_Multimodal_Transformer_for_Aspect-Based_Multimodal_Sentiment_Analysis_]](https://ieeexplore.ieee.org/document/9765342)      

* [13] K Zhang, K Zhang, M Zhang, H Zhao, Q Liu…. [[_Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis_]](https://arxiv.org/abs/2203.16369)     

* [14] X Ju, D Zhang, R Xiao, J Li, S Li…. [[_Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal Relation Detection_]](https://aclanthology.org/2021.emnlp-main.360/)      

* [15] X Ju, D Zhang, R Xiao, J Li, S Li…. [[_Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal_]](https://aclanthology.org/2021.emnlp-main.360/)      

* [16] F Zhao, C Li, Z Wu, Y Ouyang, J Zhang…. [[_M2DF Multi-grained Multi-curriculum Denoising Framework for_]](https://arxiv.org/abs/2310.14605)      

* [17] F Zhao, C Li, Z Wu, Y Ouyang, J Zhang…. [[_M2DF- Multi-grained Multi-curriculum Denoising Framework for Multimodal Aspect-based Sentiment Analysis_]](https://arxiv.org/abs/2310.14605)    

* [18] J Zhou, J Zhao, JX Huang, QV Hu, L He. [[_MASAL A large-scale dataset for multimodal aspect-based sentiment_]](https://www.sciencedirect.com/science/article/pii/S0925231221007931)      

* [19]  PictureCarol Xu, PictureXuan Luo, PictureDan Wang. [[_MMCPR: A Chinese Product Review Dataset for Multimodal Aspect-Based Sentiment Analysis_]]((https://dl.acm.org/doi/abs/10.1007/978-3-031-23585-6_8))      

* [20] Jie Mu, Feiping Nie, Wei Wang, Jian Xu, Jing Zhang, Han Liu. [[_MOCOLNet_A_Momentum_Contrastive_Learning_Network_for_Multimodal_Aspect-Level_Sentiment_Analysis_]](MOCOLNet_A_Momentum_Contrastive_Learning_Network_for_Multimodal_Aspect-Level_Sentiment_Analysis)      

* [21] Z Zhang, Z Wang, X Li, N Liu, B Guo, Z Yu. [[_ModalNet_an Aspect-level sentiment classification model by exploring multimodal data with fusion discriminant attentional networks_]](https://link.springer.com/article/10.1007/s11280-021-00955-7)      

* [22] H Jin, J Tan, L Liu, L Qiu, S Yao, X Chen…. [[_MSRA- A Multi-Aspect Semantic Relevance Approach for E-Commerce via Multimodal Pre-Training_]](https://dl.acm.org/doi/abs/10.1145/3583780.3615224)      

* [23] J Yang, Y Xiao, X Du. [[_Multi-grained fusion network with self-distillation for aspect-based multimodal sentiment analysis_]](https://www.sciencedirect.com/science/article/pii/S0950705124003599)      

* [24] N Xu, W Mao, G Chen. [[_Multi-Interactive Memory Network for Aspect Based Multimodal Sentiment Analysis_]](https://ojs.aaai.org/index.php/AAAI/article/view/3807)      
    
* [25] N Xu, W Mao, G Chen. [[_Multi_Interactive Memory Network for Aspect Based Multimodal Sentiment Analysis_]](https://ojs.aaai.org/index.php/AAAI/article/view/3807)      

* [26] M Anschütz, T Eder, G Groh. [[_Retrieving Users’ Opinions on Social Media with Multimodal Aspect-Based Sentiment Analysis_]](https://ieeexplore.ieee.org/abstract/document/10066699/)      

* [27] J Yu, J Wang, R Xia, J Li. [[_Targeted Multimodal Sentiment Classification Based on_]](https://www.ijcai.org/proceedings/2022/0622.pdf)      

* [28] Donghong Gu, Jiaqian Wang, Shaohua Cai, Chi Yang, Zhengxin Song. [[_Targeted_Aspect-Based_Multimodal_Sentiment_Analysis_An_Attention_Capsule_Extraction_and_Multi-Head_Fusion_Network_]](https://ieeexplore.ieee.org/document/9606882)      

* [29] Y Ling, J Yu, R Xia. [[_Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis 2_]](https://arxiv.org/abs/2204.07955)      

* [30] Y Ling, J Yu, R Xia. [[_Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis_]](https://arxiv.org/abs/2204.07955)      

* [31] QT Truong, HW Lauw. [[_VistaNet_Visual Aspect Attention Network for Multimodal Sentiment Analysis_]](https://ojs.aaai.org/index.php/AAAI/article/view/3799)      

* [32] D Lu, L Neves, V Carvalho, N Zhang…. [[_Visual Attention Model for Name Tagging in Multimodal Social Media_]](https://aclanthology.org/P18-1185/)  
     

>#### _Multimodal Multi-label Emotion Recognition (MMER)_ ####
>> ### Research ###

* [1] P Yang, F Luo, S Ma, J Lin, X Sun. [[_A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification_]](https://aclanthology.org/P19-1518/)      

* [2] D Zhou, Y Xiang, L Zhang, C Ye…. [[_A Divide-And-Conquer Approach for Multi-label Multi-hop Relation Detection in Knowledge Base Question Answering_]](https://aclanthology.org/2021.findings-emnlp.412/)      

* [3] C Wu, L Cao, Y Ge, Y Liu, M Zhang, J Su. [[_A Label Dependence-Aware Sequence Generation Model for Multi-Level Implicit Discourse Relation Recognition_]](https://ojs.aaai.org/index.php/AAAI/article/view/21401)      

* [4] R You, Z Guo, L Cui, X Long, Y Bao…. [[_Cross-Modality Attention with Semantic Graph Embedding for Multi Label Classification_]](https://ojs.aaai.org/index.php/AAAI/article/view/6964)    

* [5] S Qian, D Xue, H Zhang, Q Fang, C Xu. [[_Dual Adversarial Graph Neural Networks for Multi-label Cross-modal Retrieval_]](https://ojs.aaai.org/index.php/AAAI/article/view/16345)      

* [6] Y Wang, Y Xie, Y Liu, K Zhou, X Li. [[_Fast Graph Convolution Network Based Multi-label Image Recognition via Cross-modal Fusion_]](https://dl.acm.org/doi/abs/10.1145/3340531.3411880)   

* [7] L Qiu, Y Yang, CC Cao, Y Zheng, H Ngai…. [[_Generating Perturbation-based Explanations with Robustness to Out-of-Distribution Data_]](https://dl.acm.org/doi/abs/10.1145/3485447.3512254)   

* [8] H Lian, C Lu, S Li, Y Zhao, C Tang, Y Zong…. [[_Label Distribution Adaptation for Multimodal Emotion Recognition with Multi-label Learning_]](https://dl.acm.org/doi/abs/10.1145/3607865.3613183)      

* [9] L Xiao, X Huang, B Chen, L Jing. [[_Label-Specific Document Representation for Multi-Label Text Classification_]](https://aclanthology.org/D19-1044/)      

* [10] Q Ma, C Yuan, W Zhou, S Hu. [[_Label-Specific Dual Graph Neural Network for Multi-Label Text Classification_]](https://aclanthology.org/2021.acl-long.298/)      

* [11] S Ge, Z Jiang, Z Cheng, C Wang, Y Yin…. [[_Learning Robust Multi-Modal Representation for Multi-Label Emotion Recognition via Adversarial Masking and Perturbation_]](https://dl.acm.org/doi/abs/10.1145/3543507.3583258)      

* [12] T Chen, M Xu, X Hui, H Wu…. [[_Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition_]](http://openaccess.thecvf.com/content_ICCV_2019/html/Chen_Learning_Semantic-Specific_Graph_Representation_for_Multi-Label_Image_Recognition_ICCV_2019_paper.html)      

* [13] J Zhao, T Zhang, J Hu, Y Liu, Q Jin, X Wang…. [[_M3ED- Multi-modal Multi-scene Multi-label Emotional Dialogue Database_]](https://arxiv.org/abs/2205.10237)      

* [14] J Zhao, Y Zhao, J Li. [[_M3TR- Multi-modal Multi-label Recognition with Transformer_]](https://dl.acm.org/doi/abs/10.1145/3474085.3475191)      

* [15] ZM Chen, XS Wei, P Wang…. [[_Multi-Label Image Recognition with Graph Convolutional Networks∗_]](http://openaccess.thecvf.com/content_CVPR_2019/html/Chen_Multi-Label_Image_Recognition_With_Graph_Convolutional_Networks_CVPR_2019_paper.html)      

* [16] HD Le, GS Lee, SH Kim, S Kim, HJ Yang. [[_Multi-Label Multimodal Emotion Recognition With Transformer-Based Fusion and Emotion-Level Representation Learning_]](https://ieeexplore.ieee.org/abstract/document/10042438/)      

* [17] D Zhang, X Ju, J Li, S Li, Q Zhu…. [[_Multi-modal Multi-label Emotion Detection with Modality and Label Dependence_]](https://aclanthology.org/2020.emnlp-main.291/)      

* [18] D Zhang, X Ju, W Zhang, J Li, S Li, Q Zhu…. [[_Multi-modal Multi-label Emotion Recognition with Heterogeneous Hierarchical Message Passing_]](https://ojs.aaai.org/index.php/AAAI/article/view/17686)      

* [19] A Illendula, A Sheth. [[_Multimodal Emotion Classification_]](https://dl.acm.org/doi/abs/10.1145/3308560.3316549)      

* [20] M Wołczyk, M Proszewska, Ł Maziarka…. [[_PluGeN Multi-Label Conditional Generation from Pre-trained Models_]](https://ojs.aaai.org/index.php/AAAI/article/view/20843)      

* [21] K Zhu, J Wu. [[_Residual Attention- A Simple but Effective Method for Multi-Label Recognition_]](http://openaccess.thecvf.com/content/ICCV2021/html/Zhu_Residual_Attention_A_Simple_but_Effective_Method_for_Multi-Label_Recognition_ICCV_2021_paper.html)      

* [22] P Yang, X Sun, W Li, S Ma, W Wu, H Wang. [[_SGM- Sequence Generation Model for Multi-Label Classification_]](https://arxiv.org/abs/1806.04822)      

* [23] A Ando, R Masumura, H Kamiyama…. [[_Speech Emotion Recognition based on Multi-Label Emotion Existence Model_]](https://www.isca-archive.org/interspeech_2019/ando19_interspeech.pdf)      

* [24] Y Zhang, M Chen, J Shen, C Wang. [[_Tailor Versatile Multi-Modal Learning for Multi-Label Emotion Recognition_]](https://ojs.aaai.org/index.php/AAAI/article/view/20895)      

* [25] J Shen, W Qiu, Y Meng, J Shang, X Ren…. [[_TaxoClass- Hierarchical Multi-Label Text Classification Using Only Class Names_]](https://par.nsf.gov/biblio/10311077)      

* [26] X Ju, D Zhang, J Li, G Zhou. [[_Transformer-based Label Set Generation for Multi-modal Multi-label Emotion Detection_]](https://dl.acm.org/doi/abs/10.1145/3394171.3413577)


>#### _Multimodal Emotion-cause Pair Extraction (MECPE)_###
>> ### Research ###

* [1] W Li, Y Li, V Pandelea, M Ge, L Zhu…. [[_Cause Pair Extraction in Conversations_]](https://ieeexplore.ieee.org/abstract/document/9926166/)      

* [2] SV Mathur, AR Jindal, H Mittal…. [[_Exploring Multimodal Emotion Cause Pair Extraction as Sequence Labelling Task_]](https://arxiv.org/abs/2404.02088)      

* [3] F Wang, Z Ding, R Xia, Z Li, J Yu. [[_Cause Pair Extraction in Conversations_]](https://ieeexplore.ieee.org/abstract/document/9969873/)      

   

>### _Multimodal Affective Computing with Missing_modality_ ###

* [1] T Zhou, S Canu, P Vera, S Ruan. [[_modality fusion based deep neural network for brain tumor segmentation with missing MR modalities_]](https://www.sciencedirect.com/science/article/pii/S0925231221013904)      

* [2] J Zhao, R Li, Q Jin. [[_Missing Modality Imagination Network for Emotion Recognition with Uncertain Missing Modalities_]](https://aclanthology.org/2021.acl-long.203/)      

* [3] W Luo, M Xu, H Lai. [[_Multimodal Reconstruct and Align Net for Missing Modality Problem in Sentiment Analysis_]](https://link.springer.com/chapter/10.1007/978-3-031-27818-1_34)      

* [4] Q Zhang, L Shi, P Liu, Z Zhu, L Xu - 2023 - Springer. [[_integrating consistency and difference networks by transformer for multimodal sentiment analysis_]](https://link.springer.com/article/10.1007/s10489-023-04869-x)      

* [5] J Zeng, T Liu, J Zhou. [[_assisted Multimodal Sentiment Analysis under Uncertain Missing Modalities_]](https://dl.acm.org/doi/abs/10.1145/3477495.3532064)      

* [6] S Parthasarathy, S Sundaram. [[_Visual Expression Recognition_]](https://dl.acm.org/doi/abs/10.1145/3395035.3425202)      

* [7] C Shang, A Palmer, J Sun, KS Chen…. [[_Missing View Imputation with Generative Adversarial Networks_]](https://ieeexplore.ieee.org/abstract/document/8257992/)      


>### _Probing_ ###
* [1] T Le Scao, A Fan, C Akiki, E Pavlick, S Ilić, D Hesslow… - 2023 - inria.hal.science. [[_A 176B-Parameter Open-Access Multilingual Language Model_]](https://inria.hal.science/hal-03850124/)      

* [2] Z Elyoseph, D Hadar-Shoval, K Asraf…. [[_ChatGPT outperforms humans in emotional awareness evaluations_]](https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023.1199058/full)      

* [3] K Schaaff, C Reinig, T Schlippe. [[_Exploring ChatGPT’s Empathic Abilities_]](https://ieeexplore.ieee.org/abstract/document/10388208/)      

* [4] W Zhao, Y Zhao, X Lu, S Wang, Y Tong…. [[_Is ChatGPT Equipped with Emotional Dialogue Capabilities__]](https://arxiv.org/abs/2304.09582)      

* [5] H Touvron, L Martin, K Stone, P Albert…. [[_Open Foundation and Fine-Tuned Chat Models_]](https://arxiv.org/abs/2307.09288)

>### _Survey_###
* [1] Liu, H., Lou, T., Zhang, Y., Wu, Y., Xiao, Y., Jense…. [[EEG-Based Multimodal Emotion Recognition: A Machine Learning Perspective_]](https://ieeexplore.ieee.org/abstract/document/10443946)

>### _Multimodal Affective Computing With ChatGPT_ ###
* [1] [[Exploring aspect‐based sentiment quadruple extraction with implicit aspects, opinions, and ChatGPT: a comprehensive survey_]](https://ieeexplore.ieee.org/abstract/document/10443946)
* [2] [[Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis_]](https://arxiv.org/abs/2306.17177)
* [3] [[Sentiment Analysis of Using ChatGPT in Education_]](https://ieeexplore.ieee.org/document/10215977)