Talk Kumano (Lecture series)

Talk Kumano (Lecture series)

03.02.2017 11:00 – 12:15

Computational Model for Inferring How an Individual Perceive Others’ Affective States

To fully understand how people emotionally communicate each other, a computer-mediated conversation system needs to read both each participant’s felt affective state and how it is perceived by each of the others. However, the primary target of previous studies has been the former. In this talk, I will introduce our computational model that explains how an individual will recognize others' affective states.
Among the various factors, we focus on perceiver effect and scene effect, i.e. the effect of observable behavioural cues in target scene to be judged, and separately model them by assuming their independence.
Perceiver model explains perceiver's cognitive bias or tendency, and doesn't include any scene information. Scene model on the contrary explains how a target scene will be recognized by an unspecific perceiver. We train both as stochastic models, and combine them according to the Bayes' rule. The perceived affect is predicted as the state that maximizes the probability given perceiver and target scene.
To evaluate the proposed framework, we collected perceived emotional congruency, we call perceived empathy, on a five-point scale from 100 subjects for 97 dyadic conversation scenes. Firstly, by calculating the two probability distributions directly from the whole rating data in a leave-one-out cross validation scheme, we obtained the normalized accuracy of .98, where one and zero mean the test-retest reliability and chance level, respectively. This strongly verifies the proposed Bayesian modelling and no interaction between the two factors surprisingly.
However, this calculation is inapplicable to unknown perceiver or scene.
Thus, as a solution, we secondly combined our perceiver model [Kumano et al. ACII 2013], which explains perceiver's cognitive tendency from his/her personality traits and gender, and our scene model [Kumano et al. TAC 2015], which relates expressed behaviours with affect perceived by a crowd of people. The resulting accuracy was around .5. However, we consider this is promising as an initial trial.

Shiro Kumano received the Ph.D. degree in information science and technology from the University of Tokyo, Japan, in 2009. He then joined Nippon Telegraph and Telephone (NTT) Corporation. He is
currently a Research Scientist with the NTT Communication Science Laboratories, and an honorary Research Associate with the University College London. His research interests include computer vision and affective computing, especially in relation to facial expression recognition and the automatic understanding of empathy. Dr. Kumano is a Member of the IEEE and IEICE. He has served as an Organizing Committee Member of the IAPR International Conference on Machine Vision Applications (MVA2013, MVA2015) and serve as a Senior Program Committee Member of the International Conference on Affective Computing and Intelligent Interaction (ACII2017).


Bâtiment: Campus Biotech

Room 144.165

Organisé par

Centre interfacultaire en sciences affectives (CISA)


Shiro Kumano, NTT Communication Science Laboratories - University College London

entrée libre


Catégorie: Séminaire

Mots clés: CISA

Plus d'infos

Contact: missing email

Fichiers joints

Talk Kumano.pdf153.5 Kb