Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

NSD-Imagery: A benchmark dataset for extending fMRI vision decoding methods to mental imagery

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new fMRI benchmark shows that mental-image decoding does not follow seen-image decoding: simpler models generalize better.

desk verdict A genuinely useful benchmark dataset, but the architecture-attribution claim is not supported by the uncontrolled method comparison; read it for the dataset, not for the lesson about linear decoders. read the letter →

arxiv 2506.06898 v1 pith:MEXTO7UK submitted 2025-06-07 cs.CV cs.LGeess.IVq-bio.NC

Reese Kneeland , Paul S. Scotti , Ghislain St-Yves , Jesse Breedlove , Kendrick Kay , Thomas Naselaris This is my paper · ORCID
classification cs.CVcs.LGeess.IVq-bio.NC
keywords mentalimagerydecodingfMRI-to-imagereconstructionNSD-Imagerybenchmarkcross-decodinggeneralizationvisionmethodshumanevaluationofreconstructionsbrain-computerinterfaceapplications
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper releases NSD-Imagery, an fMRI benchmark in which the same eight participants who saw thousands of natural images in the Natural Scenes Dataset also imagined a small set of simple shapes, natural scenes, and single-word concepts while brain activity was recorded. Using this held-out data, the authors test five modern fMRI-to-image decoders that were trained only on seen images, asking whether they can reconstruct what a person is imagining. The central finding is that a decoder's quality on seen images does not predict its quality on mental images: the current best seen-image decoder performs near chance on imagined scenes, while older, simpler decoders with regression backbones and multimodal features generalize best. Human raters confirm the quantitative pattern, and the paper argues that mental-imagery benchmarks are therefore essential for building decoders for brain-computer interfaces and clinical use, where the target is always internally generated.

What carries the argument

The load-bearing object is the cross-decoding benchmark protocol: decoders trained exclusively on NSD seen-image trials are applied without retraining to a held-out set of imagery trials from the same subjects, with cue-based mental imagery, 8\textendash16 trial repetitions, separate run-wise z-scoring, and evaluation by both image-feature metrics and large-scale human ratings. The dataset itself is structured as 12 run types spanning three tasks (vision, imagery, attention) and three stimulus classes (simple geometric shapes, complex natural scenes, and single-word concepts), with only 18 imagined stimuli, which is what makes the cross-decoding test possible. On the architecture side, the paper isolates the decoding backbone as the operative difference: ridge-regression feature decoding plus multimodal features is contrasted with deep contrastive embedding decoders.

What would settle it

Collect a second mental-imagery dataset from the same subjects with matched trial counts and preprocessing, then re-run the five decoders after swapping only the decoding head (for example, replacing MindEye2's deep contrastive decoder with a ridge-regression head while keeping its embeddings, or vice versa); if the imagery ranking follows the embedding space or the training recipe rather than the decoder's architectural complexity, the paper's architecture-based explanation of the decoupling would be overturned.

Watch

Extended reading notes

Core claim

On NSD-Imagery, the paper claims, the performance of vision decoding methods on mental images is largely decoupled from their performance on seen-image reconstruction. Across the five tested methods, per-stimulus reconstruction quality is positively but weakly correlated between vision and imagery (human similarity ratings correlate with $r \approx 0.13$\textendash$0.22$), and the ordering of methods changes: MindEye2, the strongest model on the NSD benchmark, drops to 56.96% human identification accuracy on imagined stimuli, close to chance, while MindEye1 (73.00%) and Brain Diffuser (73.95%) remain far above chance. The authors attribute the divergence to architecture: methods with simple linear decoding backbones and multimodal image-plus-text features generalize better, whereas complex deep-network decoders overfit the visual training data. They also find that imagined complex natural scenes are reconstructed about as well as seen simple shapes, suggesting that distributional alignment with training data, not stimulus complexity, currently limits mental-image decoding.

Load-bearing premise

The load-bearing premise is that the five compared decoders differ mainly in architecture, so their different mental-imagery scores can be credited to decoder complexity rather than to their differing embeddings, diffusion priors, training procedures, fMRI preprocessing, or brain regions used.

Editorial extensions

If this is right

  • NSD-Imagery gives any NSD-trained decoder a held-out test set for generalization to internally generated visual content, so future methods can be compared on mental imagery rather than only on seen images.
  • For practical mental-imagery decoding, the best seen-image model should not be selected by default; simple linear decoders with multimodal features currently generalize better.
  • Performance on imagined complex scenes is on par with seen simple shapes, so training-distribution alignment matters more than the inherent complexity of the imagined stimulus.
  • Mental-imagery reconstruction should be evaluated with human raters, because feature-metric rankings alone do not track the decoupling seen here.
  • The near-chance performance of the strongest seen-image decoder means that improving vision decoding does not automatically improve imagery decoding; both benchmarks are needed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the architecture explanation holds beyond this 18-stimulus set, imagery generalization could become a cheap model-selection signal: choose the decoder that best transfers to mental imagery, even if it trails on seen-image benchmarks.
  • The confounds between method and architecture (different embeddings, diffusion priors, training recipes, and preprocessing) leave room for the ranking to shift; a controlled ablation that varies only the decoder head would be the natural next test.
  • The same cross-decoding design could be extended to other internally generated content such as memories, dreams, or planned actions, where the same signal-to-noise and distributional issues arise.
  • The more consistent scaling of ridge-regression decoders with training data suggests that simpler inductive biases may become even more preferable as mental-imagery training data grows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces NSD-Imagery, an extension of the Natural Scenes Dataset (NSD) with 7T fMRI responses collected during a mental imagery task from the same eight NSD participants. The dataset includes vision, imagery, and attention runs for 18 stimuli spanning simple shapes, complex natural scenes, and conceptual cues. The authors benchmark five NSD-trained reconstruction methods (MindEye1, MindEye2, Brain Diffuser, iCNN, and Takagi et al.) on the imagery and vision trials, evaluate reconstructions with both automated feature metrics and large-scale human ratings, and report that method-level performance on mental imagery is largely decoupled from performance on seen-image reconstruction. They further claim that architectural choices—specifically simple linear decoding backbones and multimodal feature decoding—drive better cross-decoding generalization. The dataset release, benchmark evaluations, and human-rating protocols are the paper's main contributions.

Significance. If the dataset is made available as described, it fills a clear gap: existing NSD-trained decoders are evaluated only on seen images, while many downstream applications require decoding internally generated mental images. The dataset is carefully designed, with cue memorization, vividness ratings, and multiple stimulus-complexity levels, and the human-rating experiments (including attention checks and large rater pools) are a strength. The decoupling result—that the best seen-image decoder is not the best imagery decoder—is a useful empirical observation. However, the further claim that simple linear decoding architectures and multimodal features are causally responsible is not supported by the comparison actually performed, because architecture is confounded with embedding space, training objective, generative prior, and implementation changes. The dataset and the method-level benchmark are valuable, but the causal architecture claim needs either reframing or additional controlled experiments.

major comments (4)
  1. [Abstract and Section 5] The central claim that 'architectural choices significantly impact cross-decoding performance' and that 'simple linear decoding architectures and multimodal feature decoding generalize better' is not identifiable from this comparison. The four retained methods differ simultaneously in decoder head (deep MLP vs ridge regression), embedding space (ViT-bigG CLIP in MindEye2 vs CLIP-ViT in MindEye1 vs VGG19 in iCNN), generative prior (SDXL unCLIP, Stable Diffusion, GAN, VAE), and training objective (contrastive alignment, feature regression, multimodal prediction). With these factors perfectly confounded, the observation that MindEye2 underperforms Brain Diffuser and MindEye1 on imagery cannot be attributed to decoder complexity or architecture. I recommend reframing the conclusion as a method-level finding (e.g., 'among the tested methods') or adding a controlled experiment that varies decoder head while holding other components fixed.
  2. [Tables 1, 3, 4 and Section 4.5] The quantitative results report no confidence intervals, subject-level variability, or significance tests for the method rankings. The human identification accuracy for MindEye2 on mental imagery is 56.96% versus 73.95% for Brain Diffuser (Table 2), but with only four subjects (1, 2, 5, 7) and twelve or eighteen stimuli, it is unclear whether these differences are reliable. Similarly, the claim in Section 4.7 that MindEye2's vision-imagery correlation is 'significantly weaker' than the other methods is not supported by any test comparing correlations. Please report subject-level error bars, bootstrap intervals, or a mixed-effects model for the ranking comparisons.
  3. [Section 4.5 and Appendix A.6] The exclusion of Takagi et al. from the main decoupling analysis is post hoc, justified by its chance-level performance on both vision and imagery trials. This removes the only clearly chance-level method and narrows the comparison; the paper should report whether the decoupling ranking is sensitive to inclusion or exclusion of this method. In addition, Appendix A.6 documents that the iCNN implementation was modified (VGG normalization disabled, ridge regression replacing fastl2lir), and Appendix A.1 states that Takagi et al. uses different ROIs from the other methods. These implementation deviations mean that even the method-level comparison is not exactly a comparison of the originally published methods, and they should be acknowledged as potential confounds in the main text.
  4. [Section 4.7 and Figure 6] The correlation between seen- and mental-image reconstruction similarity scores is reported as r = 0.13 to 0.22 with p < 0.001. While statistically significant, these values explain only a few percent of the variance, so the conclusion in Section 5 that 'training a vision decoder can serve as an effective proxy for directly optimizing a mental image decoder' is stronger than the evidence supports. The correlation may be driven by a small number of stimuli, and the claim that the 'degree to which these improvements translate' depends on architecture again relies on the confounded method comparison. I recommend tempering this conclusion and reporting per-stimulus or per-subject scatter with confidence bounds.
minor comments (5)
  1. [Appendix A.6] The appendix title reads 'iCCN implementation'; this should be 'iCNN implementation'.
  2. [Table 1] The table uses the abbreviation '2WC' in the footnote but does not define it until Appendix A.2; please define it in the table caption or footnote.
  3. [Section 4.2] The sentence 'The decoding models enforce this prior on the outputs' is ambiguous; it could be clarified as 'the pretrained diffusion prior' rather than implying the decoding models themselves are enforcing a prior.
  4. [Appendix A.7 and A.8] Figures 13 and 14 use a 'normalized average of all metrics in Table 1' without specifying the normalization procedure; please describe it in the captions or accompanying text.
  5. [Section 3] The paper states that all 8 NSD participants underwent an additional scanning session, but the benchmark results are reported only for subjects 1, 2, 5, and 7; please clarify in the main text that the evaluations are limited to the four subjects who completed the full NSD experiment.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: NSD-Imagery is a held-out benchmark; the decoupling result is empirical.

full rationale

I find no circularity in the paper's claims. NSD-Imagery is a newly collected, held-out fMRI dataset; the five decoding models were trained externally on NSD before any imagery data were used, so the reported vision-vs-imagery comparisons are genuine evaluations rather than predictions derived from fitted parameters. The central result that mental-image decoding performance is decoupled from seen-image decoding performance is a descriptive measurement on this held-out data, not a quantity constructed from the paper's own inputs. No equation in the paper defines a target metric in terms of the fitted model, and no parameter is fitted to the imagery trials and then renamed as a prediction. The paper does contain self-citations to prior work by overlapping authors (NSD, MindEye2, GNet, and a preliminary NSD-Imagery retrieval study), but none is load-bearing: NSD is an externally released dataset, MindEye2 is evaluated and reported as failing, and the GNet encoding model is used only for auxiliary brain-correlation metrics. The architecture-attribution claim in Section 5 is vulnerable to confounding—the compared methods differ in decoder head, embedding space, generative prior, training objective, and ROI selection—but confounding is an identifiability/validity limitation, not circularity. The paper also transparently documents implementation modifications to iCNN and the exclusion of Takagi et al., which further weakens the architecture comparison but does not make any claim reduce to its own inputs. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper's contribution is empirical. It introduces no free parameters in its analyses and no new theoretical entities. The central benchmark rests on assumptions about the validity of mental imagery trials and about cross-decoding, plus an uncontrolled comparison across methods for the architecture claim.

assumptions (4)
  • domain assumption Brain activity patterns during mental imagery overlap enough with vision that decoders trained on seen images can be tested on imagery without retraining.
    Section 1 justifies cross-decoding by citing shared representations (Albers 2013; Cichy 2012; Kosslyn 2006; Reddy 2010; Stokes 2009). The entire benchmark design depends on this transfer being meaningful.
  • domain assumption Participants' self-reported vividness button presses confirm that imagery trials contain genuine mental images of the cued stimulus.
    Section 3: participants were instructed to imagine the cued stimulus and rate vividness as 'vivid' or 'not vivid', but there is no objective verification of the content of the mental image.
  • domain assumption The GLMsingle beta estimates and per-run z-scoring are valid for the shorter 12-run NSD-Imagery sessions and control for cross-session non-stationarity.
    Appendix A.1 describes use of GLMsingle and within-run z-scoring; the paper also notes that cross-session non-stationarity may explain drops in the vision trials relative to shared1000.
  • domain assumption Differences among the four compared methods can be attributed to architecture rather than to training details or implementations.
    Section 5 attributes MindEye2's failure to high-dimensional ViT-bigG embeddings and Brain Diffuser's success to ridge regression, but the methods differ in many uncontrolled ways, including training procedure, embedding space, and implementation changes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NSD-Imagery: A benchmark dataset for extending fMRI vision decoding methods to mental imagery." pith.science (2026). https://pith.science/paper/MEXTO7UK

@misc{pith2026250606898,
  author       = {Pith},
  title        = {Pith review of: NSD-Imagery: A benchmark dataset for extending fMRI vision decoding methods to mental imagery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MEXTO7UK}},
  note         = {Machine review of arXiv:2506.06898}
}
read the original abstract

We release NSD-Imagery, a benchmark dataset of human fMRI activity paired with mental images, to complement the existing Natural Scenes Dataset (NSD), a large-scale dataset of fMRI activity paired with seen images that enabled unprecedented improvements in fMRI-to-image reconstruction efforts. Recent models trained on NSD have been evaluated only on seen image reconstruction. Using NSD-Imagery, it is possible to assess how well these models perform on mental image reconstruction. This is a challenging generalization requirement because mental images are encoded in human brain activity with relatively lower signal-to-noise and spatial resolution; however, generalization from seen to mental imagery is critical for real-world applications in medical domains and brain-computer interfaces, where the desired information is always internally generated. We provide benchmarks for a suite of recent NSD-trained open-source visual decoding models (MindEye1, MindEye2, Brain Diffuser, iCNN, Takagi et al.) on NSD-Imagery, and show that the performance of decoding methods on mental images is largely decoupled from performance on vision reconstruction. We further demonstrate that architectural choices significantly impact cross-decoding performance: models employing simple linear decoding architectures and multimodal feature decoding generalize better to mental imagery, while complex architectures tend to overfit visual training data. Our findings indicate that mental imagery datasets are critical for the development of practical applications, and establish NSD-Imagery as a useful resource for better aligning visual decoding methods with this goal.

Figures

Figures reproduced from arXiv: 2506.06898 by the authors.

Figure 1
Figure 1. Overview of our dataset (NSD-Imagery) against previous imagery reconstruction datasets. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the tasks utilized for the NSD-Imagery [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison of reconstruction methods on [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of reconstruction methods on all [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Human similarity score results for the simple and complex [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Cumulative distribution plot of human two-way iden [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of the median-case reconstruc [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison of the median-case reconstruc [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison of the worst-case reconstructions [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison of the worst-case reconstructions [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 12
Figure 12. Figure 12: An example of similarity score task used in experiment [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 11
Figure 11. Figure 11: An example of the 2 alternative forced choice task used [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 14
Figure 14. Figure 14: Performance of various methods on NSD-Imagery for [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 13
Figure 13. Figure 13: Performance of various methods when averaging across [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Real-time Reconstruction of Human Visual Perception from fMRI

    cs.CV 2026-07 conditional novelty 6.0 of 10

    First demonstration that single-trial visual images can be decoded from fMRI in near-real-time (about 10-15 seconds) with roughly one hour of fine-tuning data.

Reference graph

Works this paper leans on

76 extracted references · 66 canonical work pages · cited by 1 Pith paper

  1. [1]

    Shared representations for working memory and mental imagery in early visual cortex.Current Biology, 23(15):1427–1431, 2013

    Anke Marit Albers, Peter Kok, Ivan Toni, H Chris Dijkerman, and Floris P De Lange. Shared representations for working memory and mental imagery in early visual cortex.Current Biology, 23(15):1427–1431, 2013. 3

    work page 2013

  2. [2]

    Allen, Ghislain St-Yves, Yihan Wu, Jesse L

    Emily J. Allen, Ghislain St-Yves, Yihan Wu, Jesse L. Breedlove, Jacob S. Prince, Logan T. Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, J. Benjamin Hutchin- son, Thomas Naselaris, and Kendrick Kay. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence.Nature Neuroscience, 25(1):116–126, 2022. 1, 2, 4, 12, 17

    work page 2022

  3. [3]

    Breedlove, Ghislain St-Yves, Cheryl A

    Jesse L. Breedlove, Ghislain St-Yves, Cheryl A. Olman, and Thomas Naselaris. Generative feedback explains distinct brain activity codes for seen and mental images.Current Biology, 30(12):2211–2224.e6, 2020. 2, 5

    work page 2020

  4. [4]

    Unsupervised learning of visual features by contrasting cluster assignments.CoRR, abs/2006.09882, 2020

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments.CoRR, abs/2006.09882, 2020. 6

    arXiv 2006

  5. [5]

    Seeing beyond the brain: Conditional diffu- sion model with sparse masked modeling for vision decoding

    Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou. Seeing beyond the brain: Conditional diffu- sion model with sparse masked modeling for vision decoding. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22710–22720, 2022. 2

    work page 2023

  6. [6]

    Cinematic mindscapes: High-quality video reconstruction from brain activity

    Zijiao Chen, Jiaxin Qing, and Juan Helen Zhou. Cinematic mindscapes: High-quality video reconstruction from brain activity. InThirty-seventh Conference on Neural Information Processing Systems, 2023. 2

    work page 2023

  7. [7]

    Very deep {vae}s generalize autoregressive models and can outperform them on images

    Rewon Child. Very deep {vae}s generalize autoregressive models and can outperform them on images. InInternational Conference on Learning Representations, 2021. 2

    work page 2021

  8. [8]

    Imagery and perception share cortical representations of con- tent and location.Cerebral cortex, 22(2):372–380, 2012

    Radoslaw M Cichy, Jakob Heinzle, and John-Dylan Haynes. Imagery and perception share cortical representations of con- tent and location.Cerebral cortex, 22(2):372–380, 2012. 3

    work page 2012

Show all 76 references
  1. [9]

    Uncovering the role of the early visual cortex in visual mental imagery.Vision, 8(2):29, 2024

    Nadine Dijkstra. Uncovering the role of the early visual cortex in visual mental imagery.Vision, 8(2):29, 2024. 3

    2024

  2. [10]

    Differential temporal dynamics during visual imagery and perception.Elife, 7: e33904, 2018

    Nadine Dijkstra, Pim Mostert, Floris P de Lange, Sander Bosch, and Marcel AJ van Gerven. Differential temporal dynamics during visual imagery and perception.Elife, 7: e33904, 2018. 3

    2018

  3. [11]

    Spencer, Cather- ine J

    Brian L Edlow, Camille Chatelle, Camille A. Spencer, Cather- ine J. Chu, Yelena G. Bodien, Kathryn L. O’Connor, Ronald E. Hirschberg, Leigh R. Hochberg, Joseph T. Giacino, Eric S. Rosenthal, and et al. Early detection of consciousness in patients with acute severe traumatic br...

    2017

  4. [12]

    Spatial perception and memory have distinct activation profiles in human visual cortex.BioRxiv, page 811331, 2019

    Serra E Favila, Brice A Kuhl, and Jonathan Winawer. Spatial perception and memory have distinct activation profiles in human visual cortex.BioRxiv, page 811331, 2019. 2

    2019

  5. [13]

    Brain Captioning: De- coding human brain activity into images and text, 2023

    Matteo Ferrante, Furkan Ozcelik, Tommaso Boccato, Rufin VanRullen, and Nicola Toschi. Brain Captioning: De- coding human brain activity into images and text, 2023. arXiv:2305.11560 [cs]. 2

    2023 arXiv

  6. [14]

    Through their eyes: multi- subject brain decoding with simple alignment techniques

    Matteo Ferrante, Tommaso Boccato, Furkan Ozcelik, Rufin VanRullen, and Nicola Toschi. Through their eyes: multi- subject brain decoding with simple alignment techniques. Imaging Neuroscience, 2, 2024. 2

    2024

  7. [15]

    Distinct top-down and bottom-up brain connectiv- ity during visual perception and imagery.Scientific reports, 7 (1):5677, 2017

    K Friston. Distinct top-down and bottom-up brain connectiv- ity during visual perception and imagery.Scientific reports, 7 (1):5677, 2017. 3

    2017

  8. [16]

    Self- supervised Natural Image Reconstruction and Large-scale Semantic Classification from Brain Activity.NeuroImage, 254:119121, 2022

    Guy Gaziv, Roman Beliy, Niv Granot, Assaf Hoogi, Francesca Strappini, Tal Golan, and Michal Irani. Self- supervised Natural Image Reconstruction and Large-scale Semantic Classification from Brain Activity.NeuroImage, 254:119121, 2022. 2

    2022

  9. [17]

    Giacino and Kathleen Kalmar

    Joseph T. Giacino and Kathleen Kalmar. The vegetative and minimally conscious states: A comparison of clinical features and functional outcome.Journal of Head Trauma Rehabilitation, 12(4):36–51, 1997. 8

    1997

  10. [18]

    Gifford, Radoslaw M

    Alessandro T. Gifford, Radoslaw M. Cichy, Thomas Naselaris, and Kendrick Kay. A 7t fmri dataset of synthetic images for out-of-distribution modeling of vision, 2025. 4

    2025

  11. [19]

    Solomon G/Michael, Bewuketu Terefe, Marye Getnet Asfaw, and Bikis Liyew. Outcomes and associated factors of trau- matic brain injury among adult patients treated in amhara regional state comprehensive specialized hospitals.BMC Emergency Medicine, 23(1), 2023. 8

    2023

  12. [20]

    Reading Imagined Letter Shapes from the Mind’s Eye Using Real-time 7 Tesla fMRI

    Rainer Goebel, Rick van Hoof, Salil Bhat, Michael L ¨uhrs, and Mario Senden. Reading Imagined Letter Shapes from the Mind’s Eye Using Real-time 7 Tesla fMRI. In2022 10th International Winter Conference on Brain-Computer Interface (BCI), pages 1–3, 2022. ISSN: 2572-7672. 1, 2, 3

    2022

  13. [21]

    Zijin Gu, Keith Jamison, Amy Kuceyeski, and Mert R. Sabuncu. Decoding natural image stimuli from fMRI data with a surface-based convolutional network. InMedical Imag- ing with Deep Learning, 2023. 2

    2023

  14. [22]

    Mental imagery in emotion and emotional disorders.Clinical psychology review, 30(3):349–362, 2010

    Emily A Holmes and Andrew Mathews. Mental imagery in emotion and emotional disorders.Clinical psychology review, 30(3):349–362, 2010. 2, 8

    2010

  15. [23]

    Decoding the visual and subjective contents of the human brain.Nature Neuroscience, 8(5):679–685, 2005

    Yukiyasu Kamitani and Frank Tong. Decoding the visual and subjective contents of the human brain.Nature Neuroscience, 8(5):679–685, 2005. 2

    2005

  16. [24]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. InThirty- seventh Conference on Neural Information Processing Sys- tems, 2023. 6

    2023

  17. [25]

    Reconstructing seen images from human brain activity via guided stochastic search

    Reese Kneeland, Jordyn Ojeda, Ghislain St-Yves, and Thomas Naselaris. Reconstructing seen images from human brain activity via guided stochastic search. InConference on Cognitive Computational Neuroscience, 2023. 2

    2023

  18. [26]

    Brain-optimized inference improves re- constructions of fMRI brain activity, 2023

    Reese Kneeland, Jordyn Ojeda, Ghislain St-Yves, and Thomas Naselaris. Brain-optimized inference improves re- constructions of fMRI brain activity, 2023. arXiv:2312.07705 [cs, q-bio]. 12

    2023 arXiv

  19. [27]

    Second Sight: Using brain-optimized encoding models to align image distributions with human brain activity, 2023

    Reese Kneeland, Jordyn Ojeda, Ghislain St-Yves, and Thomas Naselaris. Second Sight: Using brain-optimized encoding models to align image distributions with human brain activity, 2023. arXiv:2306.00927 [cs, q-bio]. 2

    2023 arXiv

  20. [28]

    Mental image reconstruction from human brain activity: Neu- ral decoding of mental imagery via deep neural network-based bayesian estimation.Neural Networks, 170:349–363, 2024

    Naoko Koide-Majima, Shinji Nishimoto, and Kei Majima. Mental image reconstruction from human brain activity: Neu- ral decoding of mental imagery via deep neural network-based bayesian estimation.Neural Networks, 170:349–363, 2024. 1, 2, 3, 5

    2024

  21. [29]

    The case for mental imagery

    Stephen M Kosslyn, William L Thompson, and Giorgio Ganis. The case for mental imagery. Oxford University Press, 2006. 2

    2006

  22. [30]

    Im- agenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Im- agenet classification with deep convolutional neural networks. InAdvances in Neural Information Processing Systems. Cur- ran Associates, Inc., 2012. 6

    2012

  23. [31]

    Hongmi Lee and Brice A. Kuhl. Reconstructing perceived and retrieved faces from activity patterns in lateral parietal cortex.Journal of Neuroscience, 36(22):6069–6082, 2016. Publisher: Soc Neuroscience. 1, 2, 3

    2016

  24. [32]

    Disen- tangling visual imagery and perception of real-world objects

    Sue-Hyun Lee, Dwight J Kravitz, and Chris I Baker. Disen- tangling visual imagery and perception of real-world objects. Neuroimage, 59(4):4064–4073, 2012. 2, 3

    2012

  25. [33]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision – ECCV 2014, pages 740–755, Cham, 2014. Springer International Publishing. 4

    2014

  26. [34]

    Dcnn-gan: Recon- structing realistic image from fmri.2019 16th International Conference on Machine Vision Applications (MVA), pages 1–6, 2019

    Yunfeng Lin, Jiang Li, and Han Wang. Dcnn-gan: Recon- structing realistic image from fmri.2019 16th International Conference on Machine Vision Applications (MVA), pages 1–6, 2019. 2

    2019

  27. [35]

    UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity, 2023

    Weijian Mai and Zhijun Zhang. UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity, 2023. arXiv:2308.07428 [cs]. 2

    2023 arXiv

  28. [36]

    Prenger, Kendrick N

    Thomas Naselaris, Ryan J. Prenger, Kendrick N. Kay, Michael Oliver, and Jack L. Gallant. Bayesian reconstruction of natural images from human brain activity.Neuron, 63(6): 902–915, 2009. Publisher: Elsevier. 2

    2009

  29. [37]

    Olman, Dustin E

    Thomas Naselaris, Cheryl A. Olman, Dustin E. Stansbury, Kamil Ugurbil, and Jack L. Gallant. A voxel-wise encod- ing model for early visual areas decodes mental images of remembered scenes.NeuroImage, 105:215–228, 2015. 2, 3

    2015

  30. [38]

    Vu, Thomas Naselaris, Yuval Ben- jamini, Bin Yu, and Jack L

    Shinji Nishimoto, An T. Vu, Thomas Naselaris, Yuval Ben- jamini, Bin Yu, and Jack L. Gallant. Reconstructing visual experiences from brain activity evoked by natural movies. Current Biology, 21(19):1641–1646, 2011. 2

    2011

  31. [39]

    Natural scene recon- struction from fmri signals using generative latent diffusion

    Furkan Ozcelik and Rufin VanRullen. Natural scene recon- struction from fmri signals using generative latent diffusion. Scientific Reports, 13, 2023. 1, 2, 4, 6, 12, 15

    2023

  32. [40]

    Furkan Ozcelik, Bhavin Choksi, Milad Mozafari, Leila Reddy, and Rufin VanRullen. Reconstruction of perceived images from fmri patterns and semantic brain exploration using instance-conditioned gans.2022 International Joint Con- ference on Neural Networks (IJCNN), pages 1–8, 2022. 2

    2022

  33. [41]

    The human imagination: the cognitive neu- roscience of visual mental imagery.Nature reviews neuro- science, 20(10):624–634, 2019

    Joel Pearson. The human imagination: the cognitive neu- roscience of visual mental imagery.Nature reviews neuro- science, 20(10):624–634, 2019. 2

    2019

  34. [42]

    Mental imagery: functional mecha- nisms and clinical applications.Trends in cognitive sciences, 19(10):590–602, 2015

    Joel Pearson, Thomas Naselaris, Emily A Holmes, and Stephen M Kosslyn. Mental imagery: functional mecha- nisms and clinical applications.Trends in cognitive sciences, 19(10):590–602, 2015. 2

    2015

  35. [43]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. InThe Twelfth International Conference on Learning Representations, 2024. 2

    2024

  36. [44]

    Improving the accuracy of single-trial fMRI response estimates using GLMsingle

    Jacob S Prince, Ian Charest, Jan W Kurzawski, John A Pyles, Michael J Tarr, and Kendrick N Kay. Improving the accuracy of single-trial fMRI response estimates using GLMsingle. eLife, 11:e77599, 2022. Publisher: eLife Sciences Publica- tions, Ltd. 12

    2022

  37. [45]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InProceedings of the...

    2021

  38. [46]

    Reading the mind’s eye: Decoding category information during mental imagery.NeuroImage, 50(2):818–825, 2010

    Leila Reddy, Naotsugu Tsuchiya, and Thomas Serre. Reading the mind’s eye: Decoding category information during mental imagery.NeuroImage, 50(2):818–825, 2010. 2

    2010

  39. [47]

    High-resolution image syn- thesis with latent diffusion models.CoRR, abs/2112.10752,

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models.CoRR, abs/2112.10752,

    arXiv

  40. [48]

    Comparison of signal to noise in vision and imagery for qualitatively different kinds of stimuli.Journal of Vision, 23(9):5961, 2023

    Tiasha Saha Roy, Jesse Breedlove, Ghislain St-Yves, Kendrick Kay, and Thomas Naselaris. Comparison of signal to noise in vision and imagery for qualitatively different kinds of stimuli.Journal of Vision, 23(9):5961, 2023. 2, 5

    2023

  41. [49]

    Mental imagery: Weak vision or compressed vision? InConference on Cognitive Computational Neuroscience, 2023

    Tiasha Saha Roy, Jesse Breedlove, Ghislain St-Yves, Kendrick Kay, and Thomas Naselaris. Mental imagery: Weak vision or compressed vision? InConference on Cognitive Computational Neuroscience, 2023. 2, 5

    2023

  42. [50]

    Recon- structing the mind’s eye: fMRI-to-image with contrastive learning and diffusion priors

    Paul Steven Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Cohen Ethan, Aidan James Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, and Tanishq Mathew Abraham. Recon- structing the mind’s eye: fMRI-to-image with contrastive le...

    2023

  43. [51]

    Norman, and Tanishq Mathew Abraham

    Paul Steven Scotti, Mihir Tripathy, Cesar Torrico, Reese Knee- land, Tong Chen, Ashutosh Narang, Charan Santhirasegaran, Jonathan Xu, Thomas Naselaris, Kenneth A. Norman, and Tanishq Mathew Abraham. Mindeye2: Shared-subject mod- els enable fMRI-to-image with 1 hour of data. In...

    2024

  44. [52]

    Seeliger, U

    K. Seeliger, U. G ¨uc ¸l¨u, L. Ambrogioni, Y . G ¨uc ¸l¨ut¨urk, and M.A.J. van Gerven. Generative adversarial networks for re- constructing natural images from brain activity.NeuroImage, 181:775–785, 2018. 2

    2018

  45. [53]

    Emmerling, Rick van Hoof, Mar- tin A

    Mario Senden, Thomas C. Emmerling, Rick van Hoof, Mar- tin A. Frost, and Rainer Goebel. Reconstructing imagined letters from early visual cortex reveals tight topographic cor- respondence between visual mental imagery and perception. Brain Structure and Function, 224(3):1167–1...

    2019

  46. [54]

    End-to-End Deep Im- age Reconstruction From Human Brain Activity.Frontiers in Computational Neuroscience, 13, 2019

    Guohua Shen, Kshitij Dwivedi, Kei Majima, Tomoyasu Horikawa, and Yukiyasu Kamitani. End-to-End Deep Im- age Reconstruction From Human Brain Activity.Frontiers in Computational Neuroscience, 13, 2019. 2

    2019

  47. [55]

    Deep image reconstruction from human brain activity.PLOS Computational Biology, 15(1):e1006633,

    Guohua Shen, Tomoyasu Horikawa, Kei Majima, and Yukiyasu Kamitani. Deep image reconstruction from human brain activity.PLOS Computational Biology, 15(1):e1006633,

  48. [56]

    Aoki, Kei Majima, Yusuke Muraki, and Yukiyasu Kamitani

    Ken Shirakawa, Yoshihiro Nagano, Misato Tanaka, Shuntaro C. Aoki, Kei Majima, Yusuke Muraki, and Yukiyasu Kamitani. Spurious reconstruction from brain activity: The thin line between reconstruction, classification, and hallucina- tion.Journal of Vision, 2024. 4, 17

    2024

  49. [57]

    A perceptually based comparison of image similarity metrics.Perception, 40(11): 1269–1281, 2011

    Pawan Sinha and Richard Russell. A perceptually based comparison of image similarity metrics.Perception, 40(11): 1269–1281, 2011. PMID: 22416586. 6

    2011

  50. [58]

    Generative Adver- sarial Networks Conditioned on Brain Activity Reconstruct Seen Images

    Ghislain St-Yves and Thomas Naselaris. Generative Adver- sarial Networks Conditioned on Brain Activity Reconstruct Seen Images. In2018 IEEE International Conference on Sys- tems, Man, and Cybernetics (SMC), pages 1054–1061, 2018. ISSN: 2577-1655. 2

    2018

  51. [59]

    Allen, Yihan Wu, Kendrick Kay, and Thomas Naselaris

    Ghislain St-Yves, Emily J. Allen, Yihan Wu, Kendrick Kay, and Thomas Naselaris. Brain-optimized deep neural network models of human visual areas learn non-hierarchical repre- sentations.Nature Communications, 14(1):3329, 2023. 12

    2023

  52. [60]

    Do better models of fmri visual response better predict mental imagery responses? InConference on Cognitive Computational Neuroscience, 2023

    Ghislain St-Yves, Jesse Breedlove, Kendrick Kay, and Thomas Naselaris. Do better models of fmri visual response better predict mental imagery responses? InConference on Cognitive Computational Neuroscience, 2023. 3

    2023

  53. [61]

    Top-Down Activation of Shape-Specific Population Codes in Visual Cortex during Mental Imagery.Journal of Neuroscience, 29(5):1565–1572, 2009

    Mark Stokes, Russell Thompson, Rhodri Cusack, and John Duncan. Top-Down Activation of Shape-Specific Population Codes in Visual Cortex during Mental Imagery.Journal of Neuroscience, 29(5):1565–1572, 2009. Publisher: Society for Neuroscience Section: Articles. 2

    2009

  54. [62]

    Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities, 2023

    Jingyuan Sun, Mingxiao Li, Zijiao Chen, Yunhao Zhang, Shaonan Wang, and Marie-Francine Moens. Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities, 2023. arXiv:2305.17214 [cs]. 2

    2023 arXiv

  55. [63]

    Rethinking the incep- tion architecture for computer vision.CoRR, abs/1512.00567,

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the incep- tion architecture for computer vision.CoRR, abs/1512.00567,

    arXiv

  56. [64]

    High-resolution image re- construction with latent diffusion models from human brain activity

    Yu Takagi and Shinji Nishimoto. High-resolution image re- construction with latent diffusion models from human brain activity. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 14453–14463,

  57. [65]

    Improving visual image re- construction from human brain activity using latent diffusion models via multiple decoded inputs, 2023

    Yu Takagi and Shinji Nishimoto. Improving visual image re- construction from human brain activity using latent diffusion models via multiple decoded inputs, 2023. 2, 4, 6, 12, 15

    2023

  58. [66]

    Mingxing Tan and Quoc V . Le. Efficientnet: Rethinking model scaling for convolutional neural networks. InPro- ceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, Califor- nia, USA, pages 6105–6114. PMLR, 2019. 6

    2019

  59. [67]

    Inverse retinotopy: Inferring the visual content of images from brain activation patterns.NeuroImage, 33(4): 1104–1116, 2006

    Bertrand Thirion, Edouard Duchesnay, Edward Hubbard, Jes- sica Dubois, Jean-Baptiste Poline, Denis Lebihan, and Stanis- las Dehaene. Inverse retinotopy: Inferring the visual content of images from brain activation patterns.NeuroImage, 33(4): 1104–1116, 2006. 1, 2, 3

    2006

  60. [68]

    Align- ing brain functions boosts the decoding of visual semantics in novel subjects, 2023

    Alexis Thual, Yohann Benchetrit, Felix Geilert, J´er´emy Rapin, Iurii Makarov, Hubert Banville, and Jean-R´emi King. Align- ing brain functions boosts the decoding of visual semantics in novel subjects, 2023. arXiv:2312.06467 [cs, eess, q-bio]. 2

    2023 arXiv

  61. [69]

    Turgeon, Franc ¸ois Lauzier, Jean-Franc ¸ois Simard, Damon C

    Alexis F. Turgeon, Franc ¸ois Lauzier, Jean-Franc ¸ois Simard, Damon C. Scales, Karen E.A. Burns, Lynne Moore, David A. Zygun, Francis Bernard, Maureen O. Meade, Tran Cong Dung, and et al. Mortality associated with withdrawal of life-sustaining therapy for patients with severe...

    2011

  62. [70]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4): 600–612, 2004. Conference Name: IEEE Transactions on Image Processing. 6

    2004

  63. [71]

    Dream: Visual decoding from reversing human visual system

    Weihao Xia, Raoul de Charette, Cengiz¨Oztireli, and Jing-Hao Xue. Dream: Visual decoding from reversing human visual system. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024. 2

    2024

  64. [72]

    nsdgeneral

    Xingqian Xu, Zhangyang Wang, Eric Zhang, Kai Wang, and Humphrey Shi. Versatile diffusion: Text, images and vari- ations all in one diffusion model.2023 IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 7720– 7731, 2022. 2 A. Appendix A.1. Additional dataset ...

    2023

  65. [74]

    During our initial trials, normalization led to unexpected color distortions in the reconstructed images

    Normalization of images:We disabled normalization of images when computing VGG19 features. During our initial trials, normalization led to unexpected color distortions in the reconstructed images. Removing nor- malization allowed the reconstructions to maintain their original ...

  66. [75]

    This change enhanced compatibility with the rest of our workflow and provided better support for managing memory-intensive computations

    Feature decoding with Ridge Regression:Instead of the fastl2lir library, we employed the Ridge Regression implementation from the sklearn library. This change enhanced compatibility with the rest of our workflow and provided better support for managing memory-intensive computa...

  67. [76]

    and NSD-Imagery is the number of times each stimu- lus was presented in the experiment, also called the number of trial repetitions. NSD contained 3 trial repetitions of each stimulus in both the training and test sets, while NSD- Imagery contains 8 trial repetitions for the v...

  68. [2019]

    1, 2, 3, 4, 6, 15, 17

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.