Content area
A major challenge in brain decoding is to reconstruct internally generated visual representations—mental images—from human brain activity. While the Natural Scenes Dataset (NSD) has enabled unprecedented improvements in reconstructing seen images from fMRI signals, methods training on it have primarily been limited to decoding only externally presented stimuli. In an analysis of the new NSD-Imagery dataset, I demonstrate that while some modern NSD-trained vision decoders can generalize quite well in reconstructing mental images, some fail, and that state-of-the-art (SOTA) performance on seen image reconstruction is no guarantee of good performance on mental image reconstruction. Motivated by these findings, I developed MIRAGE, a novel method explicitly designed to train on vision datasets and cross-decode mental images from brain activity. MIRAGE employs a simple and robust ridge regression backbone, maps to multi-modal text and image features, and adopts the Stable Cascade diffusion model which accepts multi-modal conditioning and small image embeddings as input. Evaluations on the NSD-Imagery benchmark—supported by human ratings and feature-based metrics—establish MIRAGE as the SOTA method for producing mental image reconstructions. This result indicates that--given the right architecture--existing large-scale datasets using external stimuli are viable training data for decoding mental images, yielding new computationally-driven insights into how mental images are represented in the brain, and offering concrete paths forward for more accurate and flexible mental imagery decoding.