Probing Multimodal Fusion in the Brain: The Dominance of Audiovisual Streams in Naturalistic Encoding

Amir Abolfazl Suratgar; Amir Hossein Bagheri Baboukani; Amir Hossein Mansouri Majoumerd; Hamid Abdollahi; Mohammad Bagher Menhaj

REVIEW 3 major objections 6 minor 18 references

Probing Multimodal Fusion in the Brain: The Dominance of Audiovisual Streams in Naturalistic Encoding

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Predicting brain activity from movies: a simple linear fusion of visual and auditory features generalizes better to out-of-distribution films than an attention-based network, while language features add no measurable predictive gain.

desk verdict A useful OOD comparison of fusion architectures for brain encoding, but the headline claims rest on small correlation differences and an unreported BERT analysis. read the letter →

arxiv 2507.19052 v1 pith:YA6EQPFG submitted 2025-07-25 cs.CV

Hamid Abdollahi , Amir Hossein Mansouri Majoumerd , Amir Hossein Bagheri Baboukani , Amir Abolfazl Suratgar , Mohammad Bagher Menhaj This is my paper · ORCID

classification cs.CV

keywords brainencodingmultimodalfMRIout-of-distributiongeneralizationvisual-auditoryfusionnaturalisticstimulilinearversusattentionmodelsWhisperspeechfeaturesX-CLIPvideo

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether brain-encoding models trained on naturalistic movies can predict fMRI responses to films they have never encountered, and what stimulus features carry that prediction. Using pretrained visual (X-CLIP) and auditory (Whisper) features, it compares a simple linear fusion model against a high-capacity attention-based fusion network on both in-distribution and out-of-distribution movies. The central finding is a reversal: the attention model wins on the familiar sitcom test set (0.218 average Pearson), while the linear model generalizes better to six diverse unseen films (0.105, an 18% gain over the baseline), suggesting a complexity-generalization trade-off. The paper also reports that BERT language features added no predictive improvement, so the final models use only the audiovisual streams, pointing to a dominance of continuous sight and sound over text in familiar-language perception. The reason to care: if true, robust neuro-AI encoding should be benchmarked out of distribution, and language encoders may be redundant in audiovisual-rich settings.

What carries the argument

The load-bearing construction is the parcel-wise encoding pipeline: each fMRI time point is predicted from the previous 10 time points' features to account for the hemodynamic delay, with X-CLIP video features (1024-dimensional after spatial and temporal pooling) and Whisper audio features (per-segment averaged encoder states) reduced by PCA and concatenated. Two fusion heads are compared: a plain linear regression per parcel and a three-stage attention network (modality-specific multi-head self-attention, a Squeeze-and-Excitation-style feature-wise gate, and a three-layer MLP prediction head). The decisive experimental mechanism is the split between the in-distribution sitcom test set and the out-of-distribution set of six stylistically different films, which produces the claimed performance reversal.

What would settle it

Retrain both models with and without BERT features on repeated random train/OOD splits; if the BERT-augmented model ever matches or beats the audiovisual-only model on a held-out set, or if the linear model's OOD edge over the attention model does not replicate across subjects and seeds, the paper's central claims are refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that a simple linear regression over delayed, PCA-reduced visual and auditory features is the more generalizable brain encoder: on the out-of-distribution set it reaches 0.105 mean Pearson correlation, beating the benchmark baseline by 18%, while the attention-based fusion model, though best in-distribution (0.218 vs 0.212 for linear and 0.203 for baseline), falls to 0.098 OOD and is outperformed by the linear model in all four subjects. The paper further claims that including BERT-subtitle features did not improve predictive accuracy in preliminary tests and increased overfitting risk, so linguistic content was excluded; the observed gains were spatially concentrated in auditory cortex, attributed to Whisper's speech representations. The paper frames these results as evidence for a complexity-generalization trade-off, for the redundancy of linguistic streams in familiar-language audiovisual perception, and for the necessity of OOD testing in encoding-model evaluation.

Load-bearing premise

The load-bearing premise is that the preliminary, unreported comparison of BERT features actually showed no predictive gain, and that the small OOD correlation differences (0.105 vs 0.098 vs about 0.089 baseline) are real rather than noise, since no significance tests are given.

Editorial extensions

If this is right

Neuro-AI benchmarks should report out-of-distribution performance alongside in-distribution accuracy, since in-distribution winners can be the weakest OOD models.
Audiovisual encoding models for familiar-language content can drop language encoders entirely, lowering complexity and overfitting risk without sacrificing predictive power.
Pretrained speech encoders such as Whisper are worth the cost: they produce the largest gains in auditory cortex relative to the baseline.
Architectural capacity is not the main driver of robust encoding; a fixed linear map over good features can beat a nonlinear fusion network on novel stimuli.
Stimulus congruence matters: the linear model tracks auditory cortex best when narration and visuals align, and worst for silent black-and-white film.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the audiovisual-dominance result is real, language features should matter more in unfamiliar-language movies or in dialogue-free clips, a split the paper does not test; that contrast would sharpen the boundary of the claim.
The linear model's OOD advantage could come from PCA regularization rather than linearity per se; ablating PCA dimensionality across both models would separate those causes.
The same comparison could be run on musical or narration-light documentaries to test whether Whisper's speech focus, rather than acoustic fidelity, drives the auditory-cortex gains.
With only four subjects, the reversal may be subject-specific; a per-subject random-effects analysis would tell whether the trade-off is consistent at the population level.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing to know: this paper asks a good question—whether a simpler fusion model generalizes better than a complex attention model on out-of-distribution movies—and it builds a clean pipeline with X-CLIP and Whisper features on Algonauts 2025. That part is worth engaging with. The problem is that the headline claims, especially the 18% OOD improvement and the redundancy of linguistic features, are not backed by error bars or significance tests, and one of them rests on an analysis that is described but never shown.

The genuine strengths: the OOD test is real held-out data, not a synthetic shift. The features come from externally pretrained models, so no circularity. The spatial finding—Whisper features help most in auditory cortex—is plausible and consistent with prior work. The paper is also honest in Section 4.3 that BERT features were tried and excluded after a preliminary analysis; that is more transparent than most, but it still hides the actual numbers.

Now the soft spots, in proportion. The central trade-off claim depends on 0.105 vs 0.098 vs 0.089 average Pearson. Those are small absolute differences, and with 1,000 parcels, a paired permutation test or bootstrap could easily show the linear model's edge is within noise. The paper says the linear model was better for all four subjects, which is suggestive, but without variance, it is not convincing. The linguistic-redundancy claim is even weaker: we only get a sentence saying BERT features 'did not yield a significant improvement.' No correlations, no p-values, no subject-level numbers. That is load-bearing because the title and abstract claim audiovisual dominance over text. You cannot verify that from the manuscript. Also, several hyperparameters (X-CLIP block index, PCA components, Nd) were chosen from unreported 'preliminary analyses,' which makes the whole pipeline harder to reproduce.

I would take the trade-off as a hypothesis, not a demonstrated result. But the setup is sound and the question matters for neuro-AI model selection. The paper deserves a serious referee, though it needs heavy revision: add per-parcel error bars and paired tests for the model comparisons, and report the BERT inclusion numbers with the same care as the main results. If the findings survive that, they will be a solid contribution. As it stands, I would not cite the current version for the strong claims, but I would cite it for the OOD benchmark methodology if it is fixed. Bring it to reading group as an example of why null results need the same statistical rigor as positive ones.

Referee Report

3 major / 6 minor

Summary. The manuscript presents multimodal brain encoding models for fMRI responses to naturalistic movies, using X-CLIP visual features, Whisper auditory features, and BERT linguistic features, combined through either a linear regression or an attention-based fusion network. The models are evaluated on the Algonauts 2025 challenge data, with in-distribution (ID) testing on Friends season 7 and out-of-distribution (OOD) testing on six diverse movies and documentaries. The central claims are that (i) a higher-capacity attention model performs best on ID data while a simpler linear model generalizes better on OOD data, outperforming a baseline by 18%; (ii) linguistic features do not improve predictive accuracy; and (iii) Whisper-based auditory features provide marked gains in auditory cortex. The paper argues for a model-complexity versus generalization trade-off in brain encoding and for the primacy of audiovisual streams over textual information in familiar-language naturalistic perception.

Significance. If the reported findings are statistically robust, the paper makes a useful contribution by demonstrating the value of out-of-distribution evaluation in brain encoding and by comparing linear versus attention-based fusion mechanisms in a large-scale naturalistic setting. The use of domain-specific pretrained feature extractors (X-CLIP, Whisper) and the spatial analysis of auditory cortex improvements are strengths, as is the explicit testing on a designated OOD set. The central trade-off claim and the linguistic-redundancy claim are interesting and would likely influence future encoding-model design. However, the support for these claims is currently weakened by the absence of error bars and significance tests on the central comparisons, and by the reliance on unreported preliminary analyses for key design decisions, including the exclusion of linguistic features. The potential significance is therefore real but conditional on improved statistical reporting.

major comments (3)

[Section 5.3, Figure 5, Table 1] The central OOD claim rests on average Pearson correlations of 0.105 (linear model), 0.098 (attention-based model), and approximately 0.089 (baseline) across 1000 parcels. These absolute differences are small, yet no error bars, confidence intervals, or significance tests are reported. Because the average is over many parcels, a paired permutation test or bootstrap interval across parcels and/or subjects could reveal whether the linear model's edge is within noise. Without such an analysis, the asserted 18% improvement and the associated complexity-generalization trade-off are not statistically supported.
[Section 4.3] The exclusion of linguistic features is justified by 'preliminary encoding models' in which BERT features 'did not yield a significant improvement,' but no quantitative results, error bars, or test statistics are provided for this comparison. This claim is load-bearing for the abstract and title, which assert the dominance of audiovisual streams over textual information. The authors should either report the full linguistic-feature comparison (with numbers and significance tests) or explicitly moderate the claim to reflect that the evidence is preliminary and not independently verifiable from the manuscript.
[Sections 4.1 and 4.4] Several design decisions that plausibly affect the main results are justified only by 'preliminary analyses' with no details: the selection of the 23rd X-CLIP transformer block (Section 4.1), the number of previous time points Nd=10 (Section 4.4), and the number of PCA components per modality (Section 4.4). Without reporting the hyperparameter search ranges, the validation metric used, and the selected values, the reproducibility of the pipeline is incomplete, and readers cannot assess how sensitive the OOD trade-off and the linguistic-feature conclusion are to these choices.

minor comments (6)

[Section 3, heading] The heading 'Dateset and Challenge' contains a typo; it should read 'Dataset and Challenge.'
[Figure 3 caption] The caption contains an incomplete word: 'the baselin' appears instead of 'the baseline.'
[Section 3, first paragraph] The expression 'seasons 1˘− 6' appears to be a formatting artifact; it should read 'seasons 1–6.'
[Equation (3)] The cost function omits the bias term b that appears in Equation (2); either include it in the squared-error expression or clarify that the bias is absorbed into the weight vector.
[Figures 4 and 6] The glass-brain visualizations use different color-bar ranges between the top and bottom panels, which makes direct visual comparison of model performance misleading; the color bars should be normalized to a common range or the differences should be explicitly stated.
[Table 1] Table 1 reports OOD performance only for the proposed linear model, not for the attention-based model or the baseline, limiting the reader's ability to assess stimulus-specific differences across models; consider adding columns for all models.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: held-out predictions from externally pretrained features; absent significance tests are a correctness concern, not a circular one.

full rationale

The derivation chain is not circular. Visual features come from externally pretrained X-CLIP, auditory features from Whisper, and the baseline is the Algonauts 2025 challenge baseline; no feature, target, or evaluation quantity is defined in terms of the paper's own fitted parameters. The linear and attention models are trained on the challenge training split and evaluated on held-out ID (Friends season 7) and OOD movies, so the reported Pearson correlations are genuine out-of-sample predictions. Hyperparameter choices (PCA dimensionality, transformer block, linguistic-feature exclusion) are described as based on preliminary validation analyses, which is standard model selection rather than fitting the test set; the final claims concern test-set performance. The exclusion of BERT features is an empirical finding whose support is thin, since no numbers or significance tests are reported and the paper itself notes the result 'could also stem from model-specific limitations,' but under-reporting and statistical weakness are correctness risks, not circularity, because the conclusion is not forced by definition or by a fitted parameter renamed as a prediction. The paper's self-citations ([4] and [8]) appear only in Related Work as prior methodological context and do not carry any load-bearing argument, uniqueness theorem, or ansatz. Therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claims rest on several hyperparameters selected via unreported validation analyses, and on the domain assumptions that pretrained feature extractors and the Pearson metric are valid for brain encoding. No new theoretical entities are introduced.

free parameters (4)

Number of previous time points (Nd) = 10
Chosen based on preliminary analyses on the validation set (Section 4.4).
X-CLIP transformer block index = 23
Selected based on 'preliminary analyses' (Section 4.1) as a balance of semantic and temporal information.
PCA components per modality = Not specified
Treated as a hyperparameter, selected to maximize validation accuracy (Section 4.4), exact values not reported.
Linguistic feature inclusion = excluded
Preliminary encoding models with BERT features did not improve accuracy and increased overfitting risk (Section 4.3); the decision is a modeling choice that directly supports the audiovisual dominance claim.

assumptions (3)

domain assumption Pretrained X-CLIP, Whisper, and BERT features are effective encoders of the corresponding sensory streams for neural encoding.
The paper relies on these features without fine-tuning or alignment to brain data (Sections 4.1-4.3).
domain assumption The Pearson correlation coefficient is the appropriate metric for parcel-wise encoding accuracy.
Used as the sole evaluation metric (Section 5.1); no alternative metrics are considered.
domain assumption The Algonauts 2025 train/test split is accepted as out-of-distribution.
The paper assumes the six OOD movies are sufficiently novel to test generalization (Section 5.3).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Probing Multimodal Fusion in the Brain: The Dominance of Audiovisual Streams in Naturalistic Encoding." pith.science (2026). https://pith.science/paper/YA6EQPFG

@misc{pith2026250719052,
  author       = {Pith},
  title        = {Pith review of: Probing Multimodal Fusion in the Brain: The Dominance of Audiovisual Streams in Naturalistic Encoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YA6EQPFG}},
  note         = {Machine review of arXiv:2507.19052}
}

read the original abstract

Predicting brain activity in response to naturalistic, multimodal stimuli is a key challenge in computational neuroscience. While encoding models are becoming more powerful, their ability to generalize to truly novel contexts remains a critical, often untested, question. In this work, we developed brain encoding models using state-of-the-art visual (X-CLIP) and auditory (Whisper) feature extractors and rigorously evaluated them on both in-distribution (ID) and diverse out-of-distribution (OOD) data. Our results reveal a fundamental trade-off between model complexity and generalization: a higher-capacity attention-based model excelled on ID data, but a simpler linear model was more robust, outperforming a competitive baseline by 18\% on the OOD set. Intriguingly, we found that linguistic features did not improve predictive accuracy, suggesting that for familiar languages, neural encoding may be dominated by the continuous visual and auditory streams over redundant textual information. Spatially, our approach showed marked performance gains in the auditory cortex, underscoring the benefit of high-fidelity speech representations. Collectively, our findings demonstrate that rigorous OOD testing is essential for building robust neuro-AI models and provides nuanced insights into how model architecture, stimulus characteristics, and sensory hierarchies shape the neural encoding of our rich, multimodal world.

Figures

Figures reproduced from arXiv: 2507.19052 by the authors.

**Figure 1.** A schematic of the multimodal encoding pipeline. We process naturalistic movie stimuli using state-of-the-art models to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** The Proposed Architecture for Multimodal Brain Response Prediction. The figure details the complete architecture of the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Comparative analysis of multimodal encoding model accuracy on the in-distribution test dataset. The bar graph displays the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Glass brain visualization of the average Pearson correlation coefficient across subjects for the in-distribution test dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Comparative analysis of multimodal encoding model accuracy on the out-of-distribution test dataset. The bar graph [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Glass brain visualization of the average Pearson correlation coefficient across subjects for the out-of-distribution test [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Spatial distribution of model performance across the six out-of-distribution (OOD) stimuli for the second subject. Each [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 9 canonical work pages

[1]

Neural processing of naturalistic audiovisual events in space and time

Yu Hu and Yalda Mohsenzadeh. Neural processing of naturalistic audiovisual events in space and time. Communi- cations Biology, 8(1):110, 2025

work page 2025
[2]

Frontal cortex organization supporting audiovisual processing during naturalistic viewing

Faxin Zhou, Amirhossein Khalilian-Gourtani, Patricia Dugan, Andrew Michalak, Orrin Devinsky, Peter Rozman, 9 Werner Doyle, Daniel Friedman, and Adeen Flinker. Frontal cortex organization supporting audiovisual processing during naturalistic viewing. bioRxiv, 2025

work page 2025
[3]

Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko

Martin Schrimpf, Idan Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko. Artificial neural networks accurately predict language processing in the brain. bioRxiv, 2020

work page 2020
[4]

Compression-enabled interpretability of voxelwise encoding models

Fatemeh Kamali, Amir Abolfazl Suratgar, Mohammadbagher Menhaj, and Reza Abbasi-Asl. Compression-enabled interpretability of voxelwise encoding models. PLOS Computational Biology, 21(2):1–20, 02 2025

work page 2025
[5]

Joint processing of linguistic properties in brains and language models

SubbaReddy Oota, Manish Gupta, and Mariya Toneva. Joint processing of linguistic properties in brains and language models. Advances in Neural Information Processing Systems , 36:18001–18014, 2023

work page 2023
[6]

Multi-modal brain encoding models for multi-modal stimuli

Subba Reddy Oota, Khushbu Pahwa, Mounika Marreddy, Maneesh Singh, Manish Gupta, and Bapi S Raju. Multi-modal brain encoding models for multi-modal stimuli. arXiv preprint arXiv:2505.20027, 2025

work page Pith review arXiv 2025
[7]

Alignment of auditory artificial networks with massive individual fmri brain data leads to generalisable improvements in brain encoding and downstream tasks

Maelle Freteault, Maximilien Le Clei, Loic Tetrel, Lune Bellec, and Nicolas Farrugia. Alignment of auditory artificial networks with massive individual fmri brain data leads to generalisable improvements in brain encoding and downstream tasks. Imaging Neuroscience, 3:imag_a_00525, 2025

2025
[8]

Structurally- constrained encoding framework using a multi-voxel reduced-rank latent model for human natural vision

Amin Ranjbar, Amir Abolfazl Suratgar, Mohammad Bagher Menhaj, and Reza Abbasi-Asl. Structurally- constrained encoding framework using a multi-voxel reduced-rank latent model for human natural vision. Journal of Neural Engineering, 21(4):046027, jul 2024

work page 2024

Show all 18 references

[9]

Scaling up ridge regression for brain encoding in a massive individual fmri dataset

Sana Ahmadi, Pierre Bellec, and Tristan Glatard. Scaling up ridge regression for brain encoding in a massive individual fmri dataset. arXiv preprint arXiv:2403.19421, 2024

2024 arXiv
[10]

The algonauts project 2025 challenge: How the human brain makes sense of multimodal movies

Alessandro T Gifford, Domenic Bersch, Marie St-Laurent, Basile Pinsard, Julie Boyle, Lune Bellec, Aude Oliva, Gemma Roig, and Radoslaw M Cichy. The algonauts project 2025 challenge: How the human brain makes sense of multimodal movies. arXiv preprint arXiv:2501.00504, 2024

2025 arXiv
[11]

The courtois neuromod project: quality assessment of the initial data release (2020)

Julie Boyle, Basile Pinsard, Valentina Borghesani, Francois Paugam, Elizabeth DuPre, and Pierre Bellec. The courtois neuromod project: quality assessment of the initial data release (2020). In 2023 Conference on Cognitive Computational Neuroscience, pages 2023–1602, 2023

2020
[12]

Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity mri

Alexander Schaefer, Ru Kong, Evan M Gordon, Timothy O Laumann, Xi-Nian Zuo, Avram J Holmes, Simon B Eickhoff, and BT Thomas Yeo. Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity mri. Cerebral cortex, 28(9):3095–3114, 2018

2018
[13]

Expanding language-image pretrained models for general video recognition

Bolin Ni, Houwen Peng, Minghao Chen, Songyang Zhang, Gaofeng Meng, Jianlong Fu, Shiming Xiang, and Haibin Ling. Expanding language-image pretrained models for general video recognition. In European conference on computer vision, pages 1–18. Springer, 2022

2022
[14]

The kinetics human action video dataset

Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017

2017 arXiv
[15]

Robust speech recognition via large-scale weak supervision

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning , pages 28492–28518. PMLR, 2023

2023
[16]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human langu...

2019
[17]

Squeeze-and-excitation networks

Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7132–7141, 2018

2018
[18]

Slowfast networks for video recognition

Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision , pages 6202–6211, 2019. 10

2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Neural processing of naturalistic audiovisual events in space and time

Yu Hu and Yalda Mohsenzadeh. Neural processing of naturalistic audiovisual events in space and time. Communi- cations Biology, 8(1):110, 2025

work page 2025

[2] [2]

Frontal cortex organization supporting audiovisual processing during naturalistic viewing

Faxin Zhou, Amirhossein Khalilian-Gourtani, Patricia Dugan, Andrew Michalak, Orrin Devinsky, Peter Rozman, 9 Werner Doyle, Daniel Friedman, and Adeen Flinker. Frontal cortex organization supporting audiovisual processing during naturalistic viewing. bioRxiv, 2025

work page 2025

[3] [3]

Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko

Martin Schrimpf, Idan Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko. Artificial neural networks accurately predict language processing in the brain. bioRxiv, 2020

work page 2020

[4] [4]

Compression-enabled interpretability of voxelwise encoding models

Fatemeh Kamali, Amir Abolfazl Suratgar, Mohammadbagher Menhaj, and Reza Abbasi-Asl. Compression-enabled interpretability of voxelwise encoding models. PLOS Computational Biology, 21(2):1–20, 02 2025

work page 2025

[5] [5]

Joint processing of linguistic properties in brains and language models

SubbaReddy Oota, Manish Gupta, and Mariya Toneva. Joint processing of linguistic properties in brains and language models. Advances in Neural Information Processing Systems , 36:18001–18014, 2023

work page 2023

[6] [6]

Multi-modal brain encoding models for multi-modal stimuli

Subba Reddy Oota, Khushbu Pahwa, Mounika Marreddy, Maneesh Singh, Manish Gupta, and Bapi S Raju. Multi-modal brain encoding models for multi-modal stimuli. arXiv preprint arXiv:2505.20027, 2025

work page Pith review arXiv 2025

[7] [7]

Alignment of auditory artificial networks with massive individual fmri brain data leads to generalisable improvements in brain encoding and downstream tasks

Maelle Freteault, Maximilien Le Clei, Loic Tetrel, Lune Bellec, and Nicolas Farrugia. Alignment of auditory artificial networks with massive individual fmri brain data leads to generalisable improvements in brain encoding and downstream tasks. Imaging Neuroscience, 3:imag_a_00525, 2025

2025

[8] [8]

Structurally- constrained encoding framework using a multi-voxel reduced-rank latent model for human natural vision

Amin Ranjbar, Amir Abolfazl Suratgar, Mohammad Bagher Menhaj, and Reza Abbasi-Asl. Structurally- constrained encoding framework using a multi-voxel reduced-rank latent model for human natural vision. Journal of Neural Engineering, 21(4):046027, jul 2024

work page 2024