Pith. sign in

REVIEW 4 major objections 4 minor 92 references

MVRS: The Multimodal Virtual Reality Stimuli-based Emotion Recognition Dataset

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces MVRS, a multimodal emotion-recognition dataset built by showing 13 people aged 12 to 60 short VR videos intended to evoke relaxation, fear, stress, sadness, and joy while recording four synchronized streams: one-eye vid

desk verdict Genuinely new but tiny VR multimodal dataset; the reported 99% accuracies are likely participant-leakage artifacts and the dataset-quality claim needs re-evaluation. read the letter →

arxiv 2509.05330 v1 pith:7G45VMM7 submitted 2025-08-31 cs.AI

Seyed Muhammad Hossein Mousavi , Atiye Ilanloo This is my paper
classification cs.AI
keywords emotionrecognitionmultimodaldatasetvirtualrealityeyetrackingbodymotionphysiologicalsignalsfusionaffectivecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces MVRS, a multimodal emotion-recognition dataset built by showing 13 people aged 12 to 60 short VR videos intended to evoke relaxation, fear, stress, sadness, and joy while recording four synchronized streams: one-eye video from a webcam inside a low-cost headset, skeletal body motion from a Kinect v2, and EMG and GSR signals from an Arduino. The authors want to establish that such synchronized multimodal VR data is a usable, separable resource for automatic emotion recognition, not merely a raw collection. To demonstrate this, they extract per-modality features, fuse them early and late, and classify with Random Forest, Decision Tree, and XGBoost. Reported accuracies reach 99.38 percent in the autoencoder-based early-fusion setting, with XGBoost the strongest classifier overall. The claimed value is that MVRS fills a gap: few emotion datasets combine body motion with physiological signals under immersive stimuli.

What carries the argument

The load-bearing mechanism is timestamp synchronization across three separate capture systems: a webcam-in-headset eye tracker, a Kinect v2 skeletal tracker, and an Arduino reading EMG and GSR analog pins. Each row in each modality is labeled from the stimulus timeline, so the intended emotion during a clip becomes the ground-truth label. The evaluation hinges on early fusion (concatenating selected features or an autoencoder's learned embedding) versus late fusion (majority voting per modality), with XGBoost as the strongest classifier on the fused feature spaces.

What would settle it

A concrete test: hold out entire participants, not random rows, and retrain. If accuracy drops sharply, the separability is driven by participant-specific patterns rather than emotion. A second test: collect self-reported felt emotion per clip; if a substantial fraction of reports disagree with the assigned labels, the ground truth is stimulus intent rather than felt emotion.

Watch

Extended reading notes

Core claim

MVRS's central claim is that synchronized multimodal recording under VR emotional videos yields an emotion-recognition dataset with clear class structure. Thirteen participants watched a fixed 11-minute 39-second sequence of VR clips (relaxation bookends, fear scenes from a horror game and film, stress scenes from Saw and a roller coaster, sadness from A Quiet Place, joy from Mr. Bean), while a low-cost webcam in the headset recorded the right eye, a Kinect v2 recorded skeletal joints, and an Arduino recorded forearm EMG and GSR, all timestamp-aligned. Feature extraction produced statistical, kinematic, spectral, and gaze descriptors; early fusion by feature selection or autoencoder and late

Load-bearing premise

The whole evaluation assumes that each participant actually felt the emotion the video was chosen to elicit, and that the timestamps from the two laptops and Arduino line up well enough that labels apply to all modalities; neither assumption is directly measured in the paper.

Editorial extensions

If this is right

  • Early fusion of body motion, physiological, and eye-tracking features consistently outperforms any single modality, so MVRS supports models that exploit cross-modal emotion cues.
  • Body motion is the strongest single channel (97.66 percent accuracy with XGBoost), suggesting skeletal tracking alone carries much of the emotion signal in VR settings.
  • Stress and fear are the most separable classes while relaxation is the hardest, so downstream users should expect class imbalance and use balancing strategies.
  • The timestamp-aligned, multi-format release (MP4, XEF, CSV/TXT/JSON) is designed for reproducibility of multimodal fusion pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported near-99 percent figures come from five-fold cross-validation on rows, and the paper removes participant identity before splitting; the same participant's segments may appear in both training and test folds. Leave-one-participant-out evaluation is the natural follow-up and may yield lower, more honest accuracy.
  • The low-cost webcam-in-headset modification (one eye tracked through a commercial HMD) could lower the barrier for VR emotion datasets, but gaze features may be sensitive to headset slippage and right-eye-only geometry; tests comparing headset-aligned frames against per-frame iris registration would clarify this.
  • The 12-year-old participant watching horror and torture clips raises consent and distress considerations the paper touches only lightly; a future extension could add post-session emotional state checks or age-segmented stimuli.
  • Because labels come from the stimulus timeline rather than participant self-report, the dataset is best read as 'responses to intended emotion induction' rather than 'felt emotions'; this distinction matters for clinical or diagnostic uses.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces MVRS, a multimodal VR-based emotion recognition dataset recorded from 13 participants (ages 12–60) under five emotion-eliciting VR video conditions: relaxation, fear, stress, sadness, and joy. Data consist of eye tracking from a webcam mounted inside a modified VR HMD, body motion from Kinect v2, and EMG/GSR from an Arduino-based setup. Features are extracted per modality, fused using two early-fusion schemes and one late-fusion scheme, and evaluated with Random Forest, Decision Tree, and XGBoost classifiers under 5-fold cross-validation. The paper reports very high classification accuracies, up to 99.38%, and concludes that the dataset is a reliable, comprehensive, and high-quality resource for the community.

Significance. If the dataset is delivered as described and if the classification results are valid, MVRS would fill a genuine gap: few publicly available datasets combine body motion, physiological signals, and eye tracking under immersive VR elicitation. The authors provide public code and dataset links on Kaggle, Zenodo, Figshare, and IEEE Data Port, and the protocol is described in unusual detail. However, the central quality claim rests on classification results that currently appear to be inflated by participant-level data leakage. The evaluation protocol does not separate participants between training and test folds, and synchronization accuracy is acknowledged as a challenge but never measured. The dataset may still be a useful contribution after a corrected evaluation, but the present results do not establish emotion separability or 'dataset quality' as claimed.

major comments (4)
  1. [Data Processing / Classification and Metrics] The evaluation protocol is not participant-independent. The Data Processing section states that 'all participants' data' are stacked into a single file per modality, and the Classification section says the pipeline 'remove[s] the emotion and participant columns' before 5-fold cross-validation with 80/20 splits. With only 13 participants, random sample-level splits will place the same participant's temporally contiguous, sensor-specific samples in both training and test folds. The model can therefore exploit participant-identity signatures rather than emotion-specific patterns. The reported accuracies in Tables 1–6, including the 99.38% headline in Table 5, cannot be interpreted as evidence of emotion separability. Please re-run the evaluation with participant-blocked cross-validation (e.g., leave-one-participant-out or grouped folds) and report per-participant results; if the dataset con
  2. [Construction / Challenges and Multimodal Fusion] The central multimodal claim depends on accurate timestamp synchronization across two separate laptops, one for eye tracking and one for Kinect plus Arduino. The Challenges section explicitly admits 'synchronization issues, especially with ensuring consistent timestamps across devices,' but the paper provides no measurement of alignment error, no offset statistics, and no validation using a known event (e.g., the countdown beep). Without an alignment-error analysis, the statement that all modalities are 'timestamp aligned' is unsupported. Please include a quantitative synchronization check and describe how residual misalignment was handled before feature extraction and fusion.
  3. [Data Collection Protocol and Labeling] Labels are assigned automatically from stimulus timestamps: 'we labeled the data for each row of modalities' raw data accordingly.' The questionnaires are mentioned as validating ground truth, but no self-report or post-stimulus emotion ratings are reported or used. A stimulus label is not necessarily the emotion the participant felt; this is particularly important because the paper's own acknowledgments note that responses depend on participant state, age, gender, culture, and other factors. Please either report the questionnaire results against the stimulus labels or explicitly restrict the dataset claim to 'stimulus-elicited conditions' rather than 'felt emotions.'
  4. [Classification and Metrics / Figure 21] The experimental reporting is internally inconsistent and insufficient for reproducibility. The text says '5-fold cross-validation' with '80 percent train and 20 percent test,' but Figure 21 is described as showing 'classification metrics across 30 folds.' It is not clear whether the tables report the aggregate of five folds, repeated five times, or something else. In addition, the paper does not report the number of feature vectors per participant/class, class sizes after stacking, or confidence intervals. Please clarify the exact splitting protocol and provide sample counts and per-fold variability for all tables.
minor comments (4)
  1. [References] Reference [69] is cited as 'Retracted: Human emotion recognition based on galvanic skin response signal feature selection and svm.' Citing a retracted paper as support for a feature set is problematic; please cite the original peer-reviewed version or replace the reference.
  2. [Introduction / VR HMD Setup] Typo: 'VaR's VR Video Player app' should likely be 'Var's VR Video Player' or the correct app name. Please verify.
  3. [Data Analysis] The text refers to 'relaxation and neutral states' when discussing eye-tracking classification, but the dataset has no neutral class. Please correct the wording.
  4. [Dataset Details] The demographic description says '13 subjects or participant stimuli' and gives only an age range; no table reports sex, age distribution, or per-participant session details. Adding such a table would improve usability.

Circularity Check

1 steps flagged · score 6.0 of 10

Emotion labels are assigned from timestamps while the same timestamps (window_start/window_end) are used as dominant input features, so the reported classification accuracies partly reduce to reading the label off the input by construction.

  1. self definitional [Data Processing (Section 4.1) and Data Analysis / Figure 15 discussion; used in Classification and Metrics]
    "The second step is labeling based on the time stamps. As we have the exact duration of each stimulus and also the exact duration of each participant’s experiment, we labeled the data for each row of modalities' raw data accordingly. ... For body motion, temporal markers ( window_start and window_end) dominate, followed by kinematic features such as positional and spectral energy measures from the hands, ankles, and spine, indicating that temporal alignment and limb dynamics play key roles in differentiating emotional states."

    The ground-truth emotion label for every row is generated by mapping timestamps to the fixed stimulus schedule. The body-motion feature set then includes the same timestamps as features (window_start and window_end), and the paper's own feature-importance analysis reports that these temporal markers dominate the classifier. Consequently, a model can achieve high 'emotion recognition' accuracy by learning the deterministic timestamp-to-stimulus mapping, without relying on genuine emotion-dependent signals from body motion, physiology, or gaze. The claimed confirmation of 'dataset quality and emotion separability' (e.g., up to 99.38% accuracy) is therefore partly forced by construction: the target label is a function of an input feature. This is a self-definitional shortcut, not independent

full rationale

The paper's central claim that MVRS is a reliable, high-quality resource rests primarily on the classification results in Tables 1-6. The evaluation chain is: timestamps -> emotion labels; timestamps -> window_start/window_end features; features -> classifier accuracy -> 'quality and emotion separability'. Because the label assignment and a dominant input feature both derive from the same timestamps, the high accuracies are in part an artifact of the experimental protocol being encoded as a feature. This is a concrete circular reduction that the paper itself exposes when it reports that temporal markers dominate feature importance. I also note two non-circular validity risks: (1) random 5-fold CV after stacking all participants can leak participant identity across folds, inflating scores; (2) the paper acknowledges cross-device timestamp synchronization issues but provides no alignment-error measurement. These are correctness concerns, not circularity. Self-citations appear frequently but are not load-bearing for the dataset-quality conclusion. Overall, the central claimed evidence is partially circular, so the score is 6 rather than 0-2.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper does not introduce new theoretical entities. Its results depend on domain assumptions about stimulus validity, synchronization accuracy, and sensor reliability, none of which are empirically validated. The only hand-chosen parameters are the PCA component cap and autoencoder latent size, both standard but arbitrarily set.

free parameters (2)
  • PCA components cap = 60 or min(training samples, features)
    Chosen by the authors in the classification pipeline to cap dimensionality; no justification or tuning procedure is given.
  • Autoencoder latent dimension = up to 100
    Set as a hyperparameter for the autoencoder-based early fusion; not tuned and no ablation is provided.
assumptions (4)
  • domain assumption Participants experienced the intended emotions during VR stimuli.
    Labels are assigned based on the stimulus video, not on participant self-reported emotion, as described in Data Collection Protocol. No self-report data are used in the classification.
  • domain assumption Timestamp synchronization across devices is accurate.
    Two separate laptops were used; the paper says timestamps fix synchronization, but the Challenges section admits synchronization issues and no measurement of alignment error is provided.
  • domain assumption Webcam-based eye tracking in a modified VR headset provides valid gaze and pupil features.
    The VR HMD Setup section describes a manual modification with a webcam and LED, but no calibration, accuracy validation, or comparison to commercial eye trackers is given.
  • domain assumption Kinect v2 skeletal tracking is reliable for seated participants wearing a VR headset.
    The Kinect is placed in front of the participant, but occlusion by the headset and seating posture may affect tracking quality; no validation of skeletal accuracy is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MVRS: The Multimodal Virtual Reality Stimuli-based Emotion Recognition Dataset." pith.science (2026). https://pith.science/paper/7G45VMM7

@misc{pith2026250905330,
  author       = {Pith},
  title        = {Pith review of: MVRS: The Multimodal Virtual Reality Stimuli-based Emotion Recognition Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7G45VMM7}},
  note         = {Machine review of arXiv:2509.05330}
}
read the original abstract

Automatic emotion recognition has become increasingly important with the rise of AI, especially in fields like healthcare, education, and automotive systems. However, there is a lack of multimodal datasets, particularly involving body motion and physiological signals, which limits progress in the field. To address this, the MVRS dataset is introduced, featuring synchronized recordings from 13 participants aged 12 to 60 exposed to VR based emotional stimuli (relaxation, fear, stress, sadness, joy). Data were collected using eye tracking (via webcam in a VR headset), body motion (Kinect v2), and EMG and GSR signals (Arduino UNO), all timestamp aligned. Participants followed a unified protocol with consent and questionnaires. Features from each modality were extracted, fused using early and late fusion techniques, and evaluated with classifiers to confirm the datasets quality and emotion separability, making MVRS a valuable contribution to multimodal affective computing.

Figures

Figures reproduced from arXiv: 2509.05330 by the authors.

Figure 1
Figure 1. The Conceptual View of Multimodal Fusion [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An eye-tracking frame sample of our MVRS dataset and some basic processing as annotations (sadness) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A depth sample view of our MVRS dataset in depth gray and colored formats, having some basic body joint annotations [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: Portion of the time-series data belonging to one of the participants in our MVRS dataset during the stress stimuli video, with the [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: A bit modified Russell’s circumplex model of emotion based on our collected emotional data [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Snapshots and the order of our stimulus VR videos during the data collection [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 8
Figure 8. Figure 8: Our modified VR HMD for the data collection [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Our Microsoft Kinect V.2 setup for body motion modality data collection [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Our Arduino setup for EMG and GSR data collection [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Our data collection protocol for the MVRS dataset [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Summary of our feature extraction and multimodal fusion steps [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Feature distribution across modalities of our MVRS dataset [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: The correlation analysis of the top ten features of our selected categories [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Feature importance analysis of our selected portions for our MVRS dataset [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Cross-modality similarity analysis using Euclidean distance, Pearson correlation, and cosine similarity [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: PCA visualization of feature distributions across modalities (0 → Fear, 1 → Joy, 2 → Relaxation, 3 → Sadness, 4 → Stress) [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: t-SNE visualization of body motion, EMG+GSR, eye tracking, and fused early fusion features of our MVRS dataset, showing [PITH_FULL_IMAGE:figures/full_fig_p024_18.png]
Figure 19
Figure 19. Figure 19: UMAP projections of unimodal and fused features, showing fragmented clusters in single modalities and clearer structure in [PITH_FULL_IMAGE:figures/full_fig_p025_19.png]
Figure 20
Figure 20. Figure 20: Log-SNR distributions across modalities, with moderate spread in eye tracking, wide variability in body motion, stable but low SNR in EMG+GSR, and a broader spread in the fused features. The log-SNR distribution highlights substantial variability in signal quality acr…
Figure 21
Figure 21. Figure 21: Violin plots of classification metrics across 30 folds for the second early-stage fusion (autoencoder + XGBoost), showing [PITH_FULL_IMAGE:figures/full_fig_p026_21.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

92 extracted references · 72 canonical work pages

  1. [1]

    A review of affective computing: From unimodal analysis to multimodal fusion

    Poria, Soujanya, et al. "A review of affective computing: From unimodal analysis to multimodal fusion." Information fusion 37 (2017): 98-125

    2017

  2. [2]

    Deap: A database for emotion analysis; using physiological signals

    Koelstra, Sander, et al. "Deap: A database for emotion analysis; using physiological signals." IEEE transactions on affective computing 3.1 (2011): 18-31

    2011

  3. [3]

    Vreed: Virtual reality emotion recognition dataset using eye tracking & physiological measures

    Tabbaa, Luma, et al. "Vreed: Virtual reality emotion recognition dataset using eye tracking & physiological measures." Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 5.4 (2021): 1-20

    2021

  4. [4]

    Real-time Automatic Emotion Recognition from Body Gestures

    Piana, Stefano, et al. "Real-time automatic emotion recognition from body gestures." arXiv preprint arXiv:1402.5047 (2014)

    work page Pith review arXiv 2014

  5. [5]

    A systematic review on affective computing: Emotion models, databases, and recent advances

    Wang, Yan, et al. "A systematic review on affective computing: Emotion models, databases, and recent advances." Information Fusion 83 (2022): 19-52

    2022

  6. [6]

    EEVR: A Dataset of Paired Physiological Signals and Textual Descriptions for Joint Emotion Representation Learning

    Singh, Pragya, et al. "EEVR: A Dataset of Paired Physiological Signals and Textual Descriptions for Joint Emotion Representation Learning." Advances in Neural Information Processing Systems 37 (2024): 15765-15778

    2024

  7. [7]

    Emotion classification with multi‐modal physiological signals using multi‐attention‐based neural network

    Zou, Chengsheng, et al. "Emotion classification with multi‐modal physiological signals using multi‐attention‐based neural network." Cognitive Computation and Systems 6.1-3 (2024): 1-11

    2024

  8. [8]

    Emotion recognition based on multi-modal physiological signals and transfer learning

    Fu, Zhongzheng, et al. "Emotion recognition based on multi-modal physiological signals and transfer learning." Frontiers in Neuroscience 16 (2022): 1000716

    2022

Show all 92 references
  1. [9]

    Advancing emotion recognition with Virtual Reality: A multimodal approach using physiological signals and machine learning

    Polo, Edoardo Maria, et al. "Advancing emotion recognition with Virtual Reality: A multimodal approach using physiological signals and machine learning." Computers in Biology and Medicine 193 (2025): 110310

    2025

  2. [10]

    Multimodal Emotion Recognition using visual, vocal and Physiological Signals: a review

    Udahemuka, Gustave, Karim Djouani, and Anish M. Kurien. "Multimodal Emotion Recognition using visual, vocal and Physiological Signals: a review." Applied Sciences 14.17 (2024): 8071

    2024

  3. [11]

    Emotion recognition with multi-modal peripheral physiological signals

    Gohumpu, Jennifer, Mengru Xue, and Yanchi Bao. "Emotion recognition with multi-modal peripheral physiological signals." Frontiers in Computer Science 5 (2023): 1264713

    2023

  4. [12]

    Virtual reality for emotion elicitation–a review

    Somarathna, Rukshani, Tomasz Bednarz, and Gelareh Mohammadi. "Virtual reality for emotion elicitation–a review." IEEE Transactions on Affective Computing 14.4 (2022): 2626-2645

    2022

  5. [13]

    Virtual reality technology

    Burdea, Grigore C., and Philippe Coiffet. Virtual reality technology. John Wiley & Sons, 2003

    2003

  6. [14]

    Is clinical virtual reality ready for primetime?

    Rizzo, Albert, and Sebastian Thomas Koenig. "Is clinical virtual reality ready for primetime?." Neuropsychology 31.8 (2017): 877

    2017

  7. [15]

    A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda

    Radianti, Jaziar, et al. "A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda." Computers & education 147 (2020): 103778

    2020

  8. [16]

    Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors

    Marín-Morales, Javier, et al. "Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors." Scientific reports 8.1 (2018): 13657

    2018

  9. [17]

    Virtual reality in the assessment, understanding, and treatment of mental health disorders

    Freeman, Daniel, et al. "Virtual reality in the assessment, understanding, and treatment of mental health disorders." Psychological medicine 47.14 (2017): 2393-2400

    2017

  10. [18]

    A discussion of cybersickness in virtual environments

    LaViola Jr, Joseph J. "A discussion of cybersickness in virtual environments." ACM Sigchi Bulletin 32.1 (2000): 47-56

    2000

  11. [19]

    Multimodal machine learning: A survey and taxonomy

    Baltrušaitis, Tadas, Chaitanya Ahuja, and Louis-Philippe Morency. "Multimodal machine learning: A survey and taxonomy." IEEE transactions on pattern analysis and machine intelligence 41.2 (2018): 423-443

    2018

  12. [20]

    Multimodal emotion recognition: A comprehensive review, trends, and challenges

    Ramaswamy, Manju Priya Arthanarisamy, and Suja Palaniswamy. "Multimodal emotion recognition: A comprehensive review, trends, and challenges." Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 14.6 (2024): e1563

    2024

  13. [21]

    A survey of affect recognition methods: audio, visual and spontaneous expressions

    Zeng, Zhihong, et al. "A survey of affect recognition methods: audio, visual and spontaneous expressions." Proceedings of the 9th international conference on Multimodal interfaces. 2007

    2007

  14. [22]

    Affect detection: An interdisciplinary review of models, methods, and their applications

    Calvo, Rafael A., and Sidney D'Mello. "Affect detection: An interdisciplinary review of models, methods, and their applications." IEEE Transactions on affective computing 1.1 (2010): 18-37

    2010

  15. [23]

    The Magic XRoom: A Flexible VR Platform for Controlled Emotion Elicitation and Recognition

    Mousavi, Seyed Muhammad Hossein, et al. "The Magic XRoom: A Flexible VR Platform for Controlled Emotion Elicitation and Recognition." Proceedings of the 25th International Conference on Mobile Human-Computer Interaction. 2023

    2023

  16. [24]

    Seyed Muhammad Hossein Mousavi. (2025). Synthetic Data Generation of Body Motion Data by Neural Gas Network for Emotion Recognition. Qeios. doi:10.32388/H3YWEX.2

    2025 doi

  17. [25]

    Synthetic Data Generation by Supervised Neural Gas Network for Physiological Emotion Recognition Data

    Mousavi, S. "Synthetic Data Generation by Supervised Neural Gas Network for Physiological Emotion Recognition Data." arXiv preprint arXiv:2501.16353 (2025)

    2025 arXiv

  18. [26]

    Introduction to Facial Micro Expressions Analysis Using Color and Depth Images: A Matlab Coding Approach (2023)

    Mousavi, Seyed Muhammad Hossein. "Introduction to Facial Micro Expressions Analysis Using Color and Depth Images: A Matlab Coding Approach (2023)." arXiv preprint arXiv:2307.06396 (2023)

    2023 arXiv

  19. [27]

    Emotion Recognition in Adaptive Virtual Reality Settings: Challenges and Opportunities

    Mousavi, Seyed Muhammad Hossein, et al. "Emotion Recognition in Adaptive Virtual Reality Settings: Challenges and Opportunities." WAMWB@ MobileHCI (2023): 1-20

    2023

  20. [28]

    Bees Local Phase Quantisation Feature Selection for RGB-D Facial Expression Recognition

    Mousavi, Seyed Muhammad Hossein, and Atiye Ilanloo. "Bees Local Phase Quantisation Feature Selection for RGB-D Facial Expression Recognition." Intelligent Engineering Optimisation with the Bees Algorithm. Cham: Springer Nature Switzerland,

  21. [29]

    Pupil size variation as an indication of affective processing

    Partala, Timo, and Veikko Surakka. "Pupil size variation as an indication of affective processing." International journal of human- computer studies 59.1-2 (2003): 185-198

    2003

  22. [30]

    Affective body expression perception and recognition: A survey

    Kleinsmith, Andrea, and Nadia Bianchi-Berthouze. "Affective body expression perception and recognition: A survey." IEEE Transactions on Affective Computing 4.1 (2012): 15-33. 32

    2012

  23. [31]

    Mousavi, S. M. H. (2025). Graph-Based Parallel Multi-Objective Optimization of Skeletal Body Motion Data for Emotion Recognition with Knowledge-Distilled Classifier (https://github.com/SeyedMuhammadHosseinMousavi/Graph-Based-Parallel- Multi-Objective-Optimization-of-Skeletal-B...

    2025 doi

  24. [32]

    Toward machine emotional intelligence: Analysis of affective physiological state

    Picard, Rosalind W., Elias Vyzas, and Jennifer Healey. "Toward machine emotional intelligence: Analysis of affective physiological state." IEEE transactions on pattern analysis and machine intelligence 23.10 (2001): 1175-1191

    2001

  25. [33]

    Emotion assessment: Arousal evaluation using EEG’s and peripheral physiological signals

    Chanel, Guillaume, et al. "Emotion assessment: Arousal evaluation using EEG’s and peripheral physiological signals." International workshop on multimedia content representation, classification and security. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006

    2006

  26. [34]

    Challenges in representation learning: A report on three machine learning contests

    Goodfellow, Ian J., et al. "Challenges in representation learning: A report on three machine learning contests." Neural information processing: 20th international conference, ICONIP 2013, daegu, korea, november 3-7, 2013. Proceedings, Part III 20. Springer berlin heidelberg, 2013

    2013

  27. [35]

    Affectnet: A database for facial expression, valence, and arousal computing in the wild

    Mollahosseini, Ali, Behzad Hasani, and Mohammad H. Mahoor. "Affectnet: A database for facial expression, valence, and arousal computing in the wild." IEEE Transactions on Affective Computing 10.1 (2017): 18-31

    2017

  28. [36]

    Dynamic texture recognition using local binary patterns with an application to facial expressions

    Zhao, Guoying, and Matti Pietikainen. "Dynamic texture recognition using local binary patterns with an application to facial expressions." IEEE transactions on pattern analysis and machine intelligence 29.6 (2007): 915-928

    2007

  29. [37]

    A 3D facial expression database for facial behavior research

    Yin, Lijun, et al. "A 3D facial expression database for facial behavior research." 7th international conference on automatic face and gesture recognition (FGR06). IEEE, 2006

    2006

  30. [38]

    Iranian kinect face database (IKFDB): a color-depth based face database collected by kinect v. 2 sensor

    Mousavi, Seyed Muhammad Hossein, and S. Younes Mirinezhad. "Iranian kinect face database (IKFDB): a color-depth based face database collected by kinect v. 2 sensor." SN Applied Sciences 3.1 (2021): 19

    2021

  31. [39]

    DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices

    Katsigiannis, Stamos, and Naeem Ramzan. "DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices." IEEE journal of biomedical and health informatics 22.1 (2017): 98-107

    2017

  32. [40]

    A multimodal database for affect recognition and implicit tagging

    Soleymani, Mohammad, et al. "A multimodal database for affect recognition and implicit tagging." IEEE transactions on affective computing 3.1 (2011): 42-55

    2011

  33. [41]

    The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system

    Walter, Steffen, et al. "The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system." 2013 IEEE international conference on cybernetics (CYBCO). IEEE, 2013

    2013

  34. [42]

    Introducing wesad, a multimodal dataset for wearable stress and affect detection

    Schmidt, Philip, et al. "Introducing wesad, a multimodal dataset for wearable stress and affect detection." Proceedings of the 20th ACM international conference on multimodal interaction. 2018

    2018

  35. [43]

    Avdos-vr: Affective video database with physiological signals and continuous ratings collected remotely in vr

    Gnacek, Michal, et al. "Avdos-vr: Affective video database with physiological signals and continuous ratings collected remotely in vr." Scientific Data 11.1 (2024): 132

    2024

  36. [44]

    eSEE-d: Emotional state estimation based on eye-tracking dataset

    Skaramagkas, Vasileios, et al. "eSEE-d: Emotional state estimation based on eye-tracking dataset." Brain Sciences 13.4 (2023): 589

    2023

  37. [45]

    EyeT4Empathy: Dataset of foraging for visual information, gaze typing and empathy assessment

    Lencastre, Pedro, et al. "EyeT4Empathy: Dataset of foraging for visual information, gaze typing and empathy assessment." Scientific Data 9.1 (2022): 752

    2022

  38. [46]

    A database of German emotional speech

    Burkhardt, Felix, et al. "A database of German emotional speech." Interspeech. V ol. 5. 2005

    2005

  39. [47]

    Surrey audio-visual expressed emotion (savee) database

    Jackson, Philip, and SJUoSG Haq. "Surrey audio-visual expressed emotion (savee) database." University of Surrey: Guildford, UK (2014)

    2014

  40. [48]

    Crema-d: Crowd-sourced emotional multimodal actors dataset

    Cao, Houwei, et al. "Crema-d: Crowd-sourced emotional multimodal actors dataset." IEEE transactions on affective computing 5.4 (2014): 377-390

    2014

  41. [49]

    The Ryerson Audio-Visual Database of Emotional Speech and Song (RA VDESS): A dynamic, multimodal set of facial and vocal expressions in North American English

    Livingstone, Steven R., and Frank A. Russo. "The Ryerson Audio-Visual Database of Emotional Speech and Song (RA VDESS): A dynamic, multimodal set of facial and vocal expressions in North American English." PloS one 13.5 (2018): e0196391

    2018

  42. [50]

    IEMOCAP: Interactive emotional dyadic motion capture database

    Busso, Carlos, et al. "IEMOCAP: Interactive emotional dyadic motion capture database." Language resources and evaluation 42 (2008): 335-359

    2008

  43. [51]

    GoEmotions: A dataset of fine-grained emotions

    Demszky, Dorottya, et al. "GoEmotions: A dataset of fine-grained emotions." arXiv preprint arXiv:2005.00547 (2020)

    2005 arXiv

  44. [52]

    Semeval-2007 task 14: Affective text

    Strapparava, Carlo, and Rada Mihalcea. "Semeval-2007 task 14: Affective text." Proceedings of the fourth international workshop on semantic evaluations (SemEval-2007). 2007

    2007

  45. [53]

    Emobank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis

    Buechel, Sven, and Udo Hahn. "Emobank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis." arXiv preprint arXiv:2205.01996 (2022)

    2022 arXiv

  46. [54]

    Dailydialog: A manually labelled multi-turn dialogue dataset

    Li, Yanran, et al. "Dailydialog: A manually labelled multi-turn dialogue dataset." arXiv preprint arXiv:1710.03957 (2017)

    2017 arXiv

  47. [55]

    Emotionlines: An emotion corpus of multi-party conversations. arXiv 2018

    Chen, S. Y ., et al. "Emotionlines: An emotion corpus of multi-party conversations. arXiv 2018." arXiv preprint arXiv:1802.08379

    2018 arXiv

  48. [56]

    The MPI emotional body expressions database for narrative scenarios

    V olkova, Ekaterina, et al. "The MPI emotional body expressions database for narrative scenarios." PloS one 9.12 (2014): e113647

    2014

  49. [57]

    Guide to the carnegie mellon university multimodal activity (cmu-mmac) database

    De la Torre, Fernando, et al. "Guide to the carnegie mellon university multimodal activity (cmu-mmac) database." (2009)

    2009

  50. [58]

    Real-time style modelling of human locomotion via feature-wise transformations and local motion phases

    Mason, Ian, Sebastian Starke, and Taku Komura. "Real-time style modelling of human locomotion via feature-wise transformations and local motion phases." Proceedings of the ACM on Computer Graphics and Interactive Techniques 5.1 (2022): 1-18

    2022

  51. [59]

    Kinematic dataset of actors expressing emotions

    Zhang, Mingming, et al. "Kinematic dataset of actors expressing emotions." Scientific data 7.1 (2020): 292

    2020

  52. [60]

    A motion capture library for the study of identity, gender, and emotion perception from biological motion

    Ma, Yingliang, Helena M. Paterson, and Frank E. Pollick. "A motion capture library for the study of identity, gender, and emotion perception from biological motion." Behavior research methods 38.1 (2006): 134-141

    2006

  53. [61]

    AMASS: Archive of motion capture as surface shapes

    Mahmood, Naureen, et al. "AMASS: Archive of motion capture as surface shapes." Proceedings of the IEEE/CVF international conference on computer vision. 2019

    2019

  54. [62]

    Motion capture dataset for practical use of AI-based motion editing and stylization

    Kobayashi, Makito, et al. "Motion capture dataset for practical use of AI-based motion editing and stylization." arXiv preprint arXiv:2306.08861 (2023)

    2023 arXiv

  55. [63]

    Emotion Detection on TV Show Transcripts with Sequence-Based Convolutional Neural Networks

    Zahiri, Sayyed M., and Jinho D. Choi. "Emotion Detection on TV Show Transcripts with Sequence-Based Convolutional Neural Networks." AAAI Workshops. V ol. 18. 2018. 33

    2018

  56. [64]

    A circumplex model of affect

    Russell, James A. "A circumplex model of affect." Journal of personality and social psychology 39.6 (1980): 1161

    1980

  57. [65]

    Neurophysiological and subjective responses to a virtual downhill cycling exercise

    Behava, Mélina, et al. "Neurophysiological and subjective responses to a virtual downhill cycling exercise." Virtual Reality 29.2 (2025): 1-14

    2025

  58. [66]

    Recognising human emotions from body movement and gesture dynamics

    Castellano, Ginevra, Santiago D. Villalba, and Antonio Camurri. "Recognising human emotions from body movement and gesture dynamics." International conference on affective computing and intelligent interaction. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007

    2007

  59. [67]

    Emotion detection from natural walking

    Cui, Liqing, Shun Li, and Tingshao Zhu. "Emotion detection from natural walking." International Conference on Human Centered Computing. Cham: Springer International Publishing, 2016

    2016

  60. [68]

    Optimal processing of surface facial EMG to identify emotional expressions: A data-driven approach

    Rutkowska, Joanna M., et al. "Optimal processing of surface facial EMG to identify emotional expressions: A data-driven approach." Behavior Research Methods 56.7 (2024): 7331-7344

    2024

  61. [69]

    Retracted: Human emotion recognition based on galvanic skin response signal feature selection and svm

    Liu, Mingyang, et al. "Retracted: Human emotion recognition based on galvanic skin response signal feature selection and svm." 2016 international conference on smart city and systems engineering (ICSCSE). IEEE, 2016

    2016

  62. [70]

    Decoding emotional experiences through physiological signal processing

    Perez-Rosero, Maria S., et al. "Decoding emotional experiences through physiological signal processing." 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2017

    2017

  63. [71]

    Pupil dilation reflects the time course of emotion recognition in human vocalizations

    Oliva, Manuel, and Andrey Anikin. "Pupil dilation reflects the time course of emotion recognition in human vocalizations." Scientific reports 8.1 (2018): 4871

    2018

  64. [72]

    Eye‐tracking analysis for emotion recognition

    Tarnowski, Paweł, et al. "Eye‐tracking analysis for emotion recognition." Computational intelligence and neuroscience 2020.1 (2020): 2909267

    2020

  65. [73]

    Emotion recognition using eye-tracking: taxonomy, review and current challenges

    Lim, Jia Zheng, James Mountstephens, and Jason Teo. "Emotion recognition using eye-tracking: taxonomy, review and current challenges." Sensors 20.8 (2020): 2384

    2020

  66. [74]

    Multimodal fusion with deep neural networks for audio-video emotion recognition

    Ortega, Juan DS, et al. "Multimodal fusion with deep neural networks for audio-video emotion recognition." arXiv preprint arXiv:1907.03196 (2019)

    1907 arXiv

  67. [75]

    M3er: Multiplicative multimodal emotion recognition using facial, textual, and speech cues

    Mittal, Trisha, et al. "M3er: Multiplicative multimodal emotion recognition using facial, textual, and speech cues." Proceedings of the AAAI conference on artificial intelligence. V ol. 34. No. 02. 2020

    2020

  68. [76]

    Multimodal alignment and fusion: A survey

    Li, Songtao, and Hao Tang. "Multimodal alignment and fusion: A survey." arXiv preprint arXiv:2411.17040 (2024)

    2024

  69. [77]

    Advances in feature selection with mutual information

    Verleysen, Michel, Fabrice Rossi, and Damien François. "Advances in feature selection with mutual information." Similarity- based clustering: recent developments and biomedical applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009. 52- 69

    2009

  70. [78]

    A novel feature selection based on one-way anova f-test for e-mail spam classification

    Elssied, Nadir Omer Fadl, Othman Ibrahim, and Ahmed Hamza Osman. "A novel feature selection based on one-way anova f-test for e-mail spam classification." Research Journal of Applied Sciences, Engineering and Technology 7.3 (2014): 625-638

    2014

  71. [79]

    Benchmark of filter methods for feature selection in high-dimensional gene expression survival data

    Bommert, Andrea, et al. "Benchmark of filter methods for feature selection in high-dimensional gene expression survival data." Briefings in Bioinformatics 23.1 (2022)

    2022

  72. [80]

    Autoassociative neural networks

    Kramer, Mark A. "Autoassociative neural networks." Computers & chemical engineering 16.4 (1992): 313-328

    1992

  73. [81]

    A multi-stage dynamical fusion network for multimodal emotion recognition

    Chen, Sihan, et al. "A multi-stage dynamical fusion network for multimodal emotion recognition." Cognitive Neurodynamics 17.3 (2023): 671-680

    2023

  74. [82]

    A Comprehensive Survey on Deep Learning Multi-Modal Fusion: Methods, Technologies and Applications

    Jiao, Tianzhe, et al. "A Comprehensive Survey on Deep Learning Multi-Modal Fusion: Methods, Technologies and Applications." Computers, Materials & Continua 80.1 (2024)

    2024

  75. [83]

    Emotion Recognition Using a Siamese Model and a Late Fusion-Based Multimodal Method in the WESAD Dataset with Hardware Accelerators

    Choi, Hyun-Sik. "Emotion Recognition Using a Siamese Model and a Late Fusion-Based Multimodal Method in the WESAD Dataset with Hardware Accelerators." Electronics 14.4 (2025): 723

    2025

  76. [84]

    The elements of statistical learning: data mining, inference, and prediction

    Hastie, Trevor. "The elements of statistical learning: data mining, inference, and prediction." (2009)

    2009

  77. [85]

    Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation

    Powers, David MW. "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation." arXiv preprint arXiv:2010.16061 (2020)

    2010 arXiv

  78. [86]

    A systematic analysis of performance measures for classification tasks

    Sokolova, Marina, and Guy Lapalme. "A systematic analysis of performance measures for classification tasks." Information processing & management 45.4 (2009): 427-437

    2009

  79. [87]

    Assessing feature importance in eye-tracking data within virtual reality using explainable artificial intelligence techniques

    Bekler, Meryem, Murat Yilmaz, and Hüseyin Emre Ilgın. "Assessing feature importance in eye-tracking data within virtual reality using explainable artificial intelligence techniques." Applied Sciences 14.14 (2024): 6042

    2024

  80. [88]

    Similarity judgment within and across categories: A comprehensive model comparison

    Richie, Russell, and Sudeep Bhatia. "Similarity judgment within and across categories: A comprehensive model comparison." Cognitive science 45.8 (2021): e13030

    2021

  81. [89]

    A Review of t-SNE

    Jung, Sangwon, et al. "A Review of t-SNE." Image Processing On Line 14 (2024): 250-270

    2024

  82. [90]

    Umap: Uniform manifold approximation and projection for dimension reduction

    McInnes, Leland, John Healy, and James Melville. "Umap: Uniform manifold approximation and projection for dimension reduction." arXiv preprint arXiv:1802.03426 (2018)

    2018 arXiv

  83. [91]

    Examining convolutional feature extraction using Maximum Entropy (ME) and Signal-to-Noise Ratio (SNR) for image classification

    Gowdra, Nidhi, Roopak Sinha, and Stephen MacDonell. "Examining convolutional feature extraction using Maximum Entropy (ME) and Signal-to-Noise Ratio (SNR) for image classification." IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society. IEEE, 2020

    2020

  84. [92]

    Violin plots: a box plot-density trace synergism

    Hintze, Jerry L., and Ray D. Nelson. "Violin plots: a box plot-density trace synergism." The American Statistician 52.2 (1998): 181-184

    1998

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.