REVIEW 3 major objections 6 minor 2 cited by
FAIRWELL: Fair Multimodal Self-Supervised Learning for Wellbeing Prediction
T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read FAIRWELL claims that making a self-supervised loss subject-aware and modality-aware reduces fairness gaps in multimodal healthcare prediction while keeping accuracy nearly unchanged.
desk verdict The method is a genuine incremental step, but the AGGF aggregate is non-monotonic and miscomputed for DeCUR, so the Pareto-frontier claim is unproven. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a subject-level reworking of the VICReg self-supervised loss. Inputs are split into fixed-length segments; one modality's segment vectors are average-pooled into a subject vector, and the invariance term aligns that pooled vector with each segment of the other modality. The variance and covariance terms are applied per modality to prevent feature collapse and decorrelate dimensions. Four training regimes determine which subject pairs are regularized against each other: M1 aligns a subject with itself, M2 aligns every subject with every other subject in the batch, M3 aligns only subjects with the same label, and M4 alternates between M2 and M3 by epoch. Alongside the loss, th
What would settle it
Run every FAIRWELL variant and every baseline on the same three datasets with at least 20 random seeds and report per-group statistical parity, equal opportunity, equalized odds, and equal accuracy with bootstrap confidence intervals. If the aggregated-fairness advantage of FAIRWELL over VICReg or SimCLR baselines does not exceed the interval width, especially on MODMA's 20-female group, the claimed Pareto-frontier dominance fails.
Extended reading notes
Core claim
On its own terms, the paper's discovery is that fairness in multimodal healthcare prediction can be improved by rewriting the standard VICReg self-supervised loss so that the regularization is applied between segments of the same or different subjects, rather than between arbitrary augmented views. The variance term is reinterpreted as reducing reliance on the protected attribute as a trivial solution, the invariance term as ensuring consistent representations for similar individuals, and the covariance term as minimizing correlational dependence on the protected attribute. In experiments on D-Vlog, MIMIC, and MODMA, FAIRWELL variants, especially M1 (intra-subject) and M2 (inter-subject), co
Load-bearing premise
The fairness gains are real rather than run-to-run noise: the paper reports no confidence intervals or significance tests, and its smallest protected group has only 20 participants.
Editorial extensions
If this is right
- Fairness gains are not tied to one modality family: improvements appear for audio-plus-visual vlogs, EEG-plus-audio, and column-split tabular ICU data.
- The method can be dropped onto existing SSL models such as CoMM, FOCAL, QUEST, and DeCUR, generally improving their fairness, so it functions as a plug-in regularizer rather than a new architecture.
- Single-pooling alignment outperforms fully pooling both modalities, suggesting that preserving segment-level detail on one side of the contrastive loss matters for learning fair representations.
- Blind application of SSL without subject-aware guidance can worsen bias; guided SSL is what shifts the performance-fairness Pareto frontier.
- Using an aggregated fairness measure instead of AUROC/AUPRC alone matters in highly imbalanced clinical datasets, since models can look strong on AUROC while performing poorly on fairness metrics.
Reading between the lines
- If the reported gains replicate across seeds, the invariance term may be acting as a soft clustering by outcome, which would suggest a path to fair SSL even when protected attribute labels are unavailable.
- The paper assumes sensitive-attribute labels are available during training; a natural testable extension is to infer pseudo-groups from learned representations and compare FAIRWELL against demographic-blind baselines.
- The MODMA comparison rests on a female group of only 20 participants, so repeated-seed bootstrap confidence intervals on per-group fairness metrics would tell whether the aggregated-fairness differences are separable from run-to-run noise. That caution is ours, not the paper's claim.
- If the Pareto-frontier improvement holds, combining FAIRWELL with post-hoc thresholding corrections could plausibly push fairness further at the same accuracy level, though the paper does not test this combination.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FAIRWELL, a subject-aware and modality-aware extension of the VICReg self-supervised loss for fair multimodal prediction in healthcare. Four variants (M1–M4) are introduced, modifying the variance, invariance, and covariance terms to reduce reliance on protected attributes. Experiments on D-Vlog, MIMIC, and MODMA compare against existing SSL and multimodal baselines. The authors report that FAIRWELL improves aggregate fairness (AGGF) with minimal loss in classification accuracy and improves the performance–fairness Pareto frontier.
Significance. If the claims hold, the work would be a useful contribution: it is among the first to apply SSL for fairness on non-tabular, variable-length multimodal data, and it evaluates on three heterogeneous healthcare datasets. The segment-based encoding and pooling modifications are sensible engineering contributions. However, the central quantitative evidence is undermined by the definition and computation of the AGGF fairness metric used for the headline claims.
major comments (3)
- [§5.4, Eq. (11)] The AGGF definition is non-monotonic and can rank a highly biased model as perfectly fair. Because of the outer absolute value, an average deviation of 2.0 yields AGGF=1.0, identical to perfect fairness (deviation 0). An average deviation of 0.5 and 1.5 both give 0.5. Since per-measure fairness ratios can exceed 1 substantially (e.g., EOdd=6.64 in Table 5), this is not a corner case. The fairness axis used for the Pareto-frontier claims therefore does not order models monotonically by their average deviation from perfect fairness.
- [Table 5 (D-Vlog, Exp 2)] The reported AGGF for DeCUR is inconsistent with Eq. (11). Using the table's values (SP=1.47, EOpp=3.25, EOdd=6.64, EAcc=0.95), the average absolute deviation is (0.47+2.25+5.64+0.05)/4 = 2.10, so AGGF should be |1−2.10| = 1.10, not the reported 0.06. Other rows (e.g., CoMM, FOCAL, VICReg) match Eq. (11), suggesting the DeCUR entry is a computational or transcription error. Because Figure 3 and the abstract's Pareto-frontier claim rely on Table 5, this inconsistency directly affects the central claim.
- [Reproducibility Checklist 4.11–4.12; §5.1; Tables 3–7] The paper reports only point estimates with no confidence intervals, error bars, or significance tests. This is especially problematic for MODMA (53 subjects, 20 females) and for D-Vlog subgroup analyses (322 males), where ratio-based fairness metrics such as EOdd are extremely noisy. The abstract's phrase 'significantly improves on the performance-fairness Pareto frontier' is not supported by any statistical significance test. If a single random seed changes the per-group ratios by a small amount, the claimed dominance of FAIRWELL variants could collapse.
minor comments (6)
- [Abstract] 'Early efforts on leveraging SSL... has proven promising' should be 'have proven promising' or rephrased.
- [§5.4, Eq. (11)] The double absolute-value notation in the displayed equation is confusing; it should be a single expression, and the direction of the metric should be clarified (the text says 'closer to 1, fairer' but the formula allows values beyond 1 and is V-shaped).
- [Tables 3–7] Fairness values are reported to two decimals but the group sizes (especially MODMA) are tiny; at least one decimal or an indication of uncertainty is needed for meaningful comparison.
- [Figure 3] The Pareto-frontier plot is difficult to read: the markers are small, the legend labels are cryptic, and the axes do not indicate which points correspond to which method beyond color.
- [References] Several in-text citations appear as bare years (e.g., '(2020; 2021; 2023a)' in the Introduction), making it impossible to attribute claims to specific works. The reference list also has incomplete entries (e.g., 'Mari et al. 2025' appears in the text but is cited as '2024' in the reference list).
- [Table 1] The row for FAIRWELL uses 'A V, A-EEG' which is unclear; also 'ND' is defined but not used consistently. Please spell out all abbreviations in the caption.
Circularity Check
No significant circularity: FAIRWELL's fairness loss is independently evaluated on external group-rate metrics; self-citations are contextual and non-load-bearing.
full rationale
FAIRWELL's derivation chain is not circular. The proposed loss (Eqs. 6-10) is an explicit modification of VICReg's variance/invariance/covariance terms; no term is defined in terms of the evaluation metrics SP/EOpp/EOdd/EAcc/AGGF. The fairness scores are computed from held-out group-rate ratios using standard definitions, and AGGF (Eq. 11) is an aggregate of those external measures, not a training objective or fitted parameter. Model selection among M1-M4/pooling variants is post-hoc comparison, not a prediction forced by construction. The paper's self-citations (e.g., Cheong et al. 2023 for gender-specific behavioural cues; Cheong et al. 2023 for fairness-measure choice) are contextual or interpretive and do not carry the central claim; no uniqueness theorem or unverified prior result is imported to forbid alternatives. The paper itself acknowledges missing variation/statistical significance (Reproducibility Checklist 4.11, 4.12 = no) and EEG reproducibility challenges (Discussion, Limitations), and the AGGF formula (Eq. 11) is non-monotonic and inconsistent with Table 5's DeCUR row. These are validity/robustness risks, not circularity: the central fairness claim does not reduce by definition to the loss's own inputs.
Assumptions & free parameters
free parameters (3)
- loss weights lambda, mu, nu =
not reported
- pooling choice (single vs double) =
single pooling on modality 1
- segment length / splitting scheme =
not reported
assumptions (4)
- domain assumption VICReg's variance and covariance terms reduce reliance on the protected attribute
- domain assumption Fairness ratios (SP, EOpp, EOdd, EAcc) are valid, and AGGF aggregation with |1 - mean deviation| is a meaningful summary
- domain assumption Protected attribute labels are available
- domain assumption Modalities contain complementary unique information so that aligning them improves task representations
Cite this review
Pith. "Pith review of FAIRWELL: Fair Multimodal Self-Supervised Learning for Wellbeing Prediction." pith.science (2026). https://pith.science/paper/2CCIWGGE
@misc{pith2026250816748,
author = {Pith},
title = {Pith review of: FAIRWELL: Fair Multimodal Self-Supervised Learning for Wellbeing Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/2CCIWGGE}},
note = {Machine review of arXiv:2508.16748}
}
read the original abstract
Early efforts on leveraging self-supervised learning (SSL) to improve machine learning (ML) fairness has proven promising. However, such an approach has yet to be explored within a multimodal context. Prior work has shown that, within a multimodal setting, different modalities contain modality-unique information that can complement information of other modalities. Leveraging on this, we propose a novel subject-level loss function to learn fairer representations via the following three mechanisms, adapting the variance-invariance-covariance regularization (VICReg) method: (i) the variance term, which reduces reliance on the protected attribute as a trivial solution; (ii) the invariance term, which ensures consistent predictions for similar individuals; and (iii) the covariance term, which minimizes correlational dependence on the protected attribute. Consequently, our loss function, coined as FAIRWELL, aims to obtain subject-independent representations, enforcing fairness in multimodal prediction tasks. We evaluate our method on three challenging real-world heterogeneous healthcare datasets (i.e. D-Vlog, MIMIC and MODMA) which contain different modalities of varying length and different prediction tasks. Our findings indicate that our framework improves overall fairness performance with minimal reduction in classification performance and significantly improves on the performance-fairness Pareto frontier.
Figures
Forward citations
Cited by 2 Pith papers
-
FAIR_XAI: Improving Multimodal Foundation Model Fairness via Explainability for Wellbeing Assessment
Zero-shot vision-language models are unreliable and vary widely for depression screening, and explainability-based fairness interventions often trade away accuracy without reliable fairness gains.
-
EQPO: Equitable Group Relative Policy Optimization for Clinical Reasoning
A GRPO variant that scales advantages by group size and mean reward is claimed to reduce demographic F1 gaps in clinical VLLMs, but the abstract and body report different experiments and the body's own tables contradi...
Reference graph
Works this paper leans on
-
[1]
General Paper Structure 1.1. Includes a conceptual outline and/or pseudocode de- scription of AI methods introduced (yes/partial/no/NA) yes 1.2. Clearly delineates statements that are opinions, hypoth- esis, and speculation from objective facts and results (yes/no) yes 1.3. Provides well-marked pedagogical references for less- familiar readers to gain bac...
-
[2]
Theoretical Contributions 2.1. Does this paper make theoretical contributions? (yes/no) no If yes, please address the following points: 2.2. All assumptions and restrictions are stated clearly and formally (yes/partial/no) Type your response here 2.3. All novel claims are stated formally (e.g., in theorem statements) (yes/partial/no) Type your response he...
-
[3]
Dataset Usage 3.1. Does this paper rely on one or more datasets? (yes/no) yes If yes, please address the following points: 3.2. A motivation is given for why the experiments are conducted on the selected datasets (yes/par- tial/no/NA) yes 3.3. All novel datasets introduced in this paper are in- cluded in a data appendix (yes/partial/no/NA) NA 3.4. All nov...
-
[4]
Computational Experiments 4.1. Does this paper include computational experiments? (yes/no) yes If yes, please address the following points: 4.2. This paper states the number and range of values tried per (hyper-) parameter during development of the paper, along with the criterion used for selecting the final parameter setting (yes/partial/no/NA) yes 4.3. ...
-
[6]
In 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI), 1–5
Machine learning fairness for depression detection us- ing eeg data. In 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI), 1–5. IEEE. Lee, M. H.; Shomanov, A.; Begim, B.; et al. 2024. EA V: EEG-Audio-Video Dataset for Emotion Recognition in Con- versational Contexts. Scientific Data, 11: 1026. Liang, P. P.; Deng, Z.; Ma, M. Q.; Zou, J.; ...
arXiv 2025
-
[8]
Bias and Fairness on Multimodal Emotion Detection Algorithms
Concept-drifts adaptation for machine learning EEG epilepsy seizure prediction. Scientific Reports, 14(1): 8204. Qayyum, A.; Razzak, I.; Tanveer, M.; Mazher, M.; and Al- haqbani, B. 2023. High-Density Electroencephalography and Speech Signal Based Deep Framework for Clinical De- pression Diagnosis. IEEE/ACM Transactions on Computa- tional Biology and Bioi...
work page Pith review arXiv 2023
-
[9]
Machine learning and bias in medical imaging: oppor- tunities and challenges. Circulation: Cardiovascular Imag- ing, 17(2): e015495. Wang, Y .; Albrecht, C. M.; Braham, N. A. A. A.; Liu, C.; Xiong, Z.; and Zhu, X. X. 2024. DeCUR: decou- pling common & unique representations for multimodal self-supervision. Xing, T.; Dou, Y .; Chen, X.; et al. 2024. An ada...
work page 2024
-
[283]
Springer. Mari, T.; Ali, S. H.; Pacinotti, L.; Powsey, S.; and Fallon, N. 2025. Machine learning classification of active viewing of pain and non-pain images using EEG does not exceed chance in external validation samples. Cognitive, Affective, & Behavioral Neuroscience, 1–18. McDermott, M.; Zhang, H.; Hansen, L.; Angelotti, G.; and Gallifant, J. 2024. A ...
work page 2025
Show all 13 references
-
[2020]
In (BigMM), 166–173
A Fairness-Aware Fusion Framework for Multimodal Cyberbullying Detection. In (BigMM), 166–173. Bansal, M. A.; Sharma, D. R.; and Kathuria, D. M. 2022. A systematic review on data scarcity problem in deep learning: solution and applications. ACM Computing Surveys (Csur), 54(10s...
2022
-
[2023]
arXiv preprint
FMMRec: Fairness-aware Multimodal Recommenda- tion. arXiv preprint. Cheong, J.; Bangar, A.; Kalkan, S.; and Gunes, H. 2025. U- Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression Detection. In Proceedings of the 4th Ma- chine Learning for Health Symposi...
2025
-
[2024]
In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, 1464–1474
Achieving Reproducibility in EEG-Based Machine Learning. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, 1464–1474. Kore, A.; Abbasi Bavil, E.; Subasri, V .; Abdalla, M.; Fine, B.; Dolatabadi, E.; and Abdalla, M. 2024. Empirical data dr...
2024
-
[2025]
Gimeno-G´omez, D.; Bucur, A.-M.; Cosma, A.; Mart ´ınez- Hinarejos, C.-D.; and Rosso, P
What to align in multimodal contrastive learning? In The Thirteenth International Conference on Learning Rep- resentations. Gimeno-G´omez, D.; Bucur, A.-M.; Cosma, A.; Mart ´ınez- Hinarejos, C.-D.; and Rosso, P. 2024. Reading Between the Frames: Multi-modal Depression Detectio...
2024 arXiv
-
[5940]
It’s not Fair!
International Joint Conferences on Artificial Intelli- gence Organization. Cheong, J.; Spitale, M.; and Gunes, H. 2023. “It’s not Fair!” – Fairness for a Small Dataset of Multi-modal Dyadic Men- tal Well-being Coaching. In ACII 2023. Cheong, J.; Spitale, M.; and Gunes, H. 2025...
2023 arXiv
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.