AIMA 2026 projects ✦ Project I ✦ Neuro-AI
Brain decoding: from fMRI & EEG to images and video
In three years, brain decoding has gone from blurry shapes to photo-like images and smooth video. It has also learned that a strong diffusion prior can paint a convincing picture that the brain never saw. Our group will build on the best recent methods and ask how much of each reconstruction truly came from the brain.
In partnership with
Seeing, recording, decoding
1 What they see
a real photograph
2 The viewer
3 Recorded signal
4 Decoder
brain encoder → shared latent
5 Reconstruction
Photos: CC0, Wikimedia Commons. Steps 3–5 are illustrations of how decoders work, not output from a real recording; the reconstructions are softened versions of the photo made for this page.
Three years of the field, on one map
Thirty-one papers from 2023 to 2026, most at CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, AAAI and Nature-family journals. Filter by signal and output; hover or tap a paper for its one-line summary and link.
Eight open research gaps
Each gap is backed by the papers above and sized for one student group and one to four GPUs. The group picks one or two with its mentors in month one.
01Cross-subject decoding with little or no data
+
Shared-subject models cut new-subject data from 40 hours to about one3–6, and zero-shot decoding is emerging7. But zero-shot still trails fine-tuning, and almost all results come from the same 7–8 NSD subjects24.
DirectionA leave-one-subject-out benchmark of alignment methods (linear, transfer matrix, LoRA, adversarial) at 0, 15 and 60 minutes of new-subject data, plus a new hyperalignment-adapter baseline.
02Faithfulness: is it the brain, or the prior?
+
A 30–50-dimensional bottleneck recovers most reconstruction quality9, and text-guided NSD models behave like category classifiers plus diffusion “hallucination”8. Common metrics reward plausible pictures, not faithful ones.
DirectionA faithfulness audit suite (bottleneck, shuffled-signal and noise controls, out-of-distribution test sets, category-leakage checks) used to re-score 4–5 open-source decoders. Mostly inference, so it needs little compute.
03The EEG fidelity gap
+
EEG is cheap and portable, but reconstructions are semantically right and low-level wrong11,12. MEG decodes high-level features well over time13. Older EEG datasets carry a block-design confound18.
DirectionEEG-to-image on THINGS-EEG225 with a pretrained EEG foundation encoder23 and a separate low-level branch, reporting low- and high-level metrics separately. Trains on a single RTX 4090.
04Low-level detail versus meaning
+
Most strong decoders split the problem into a low-level layout path and a semantic path2,16, but the trade-off between them is tuned by hand.
DirectionA controlled study of the low-level/semantic Pareto front per brain region (early visual cortex versus higher areas) on NSD, with region-specific adapters.
05Temporal consistency in video decoding
+
fMRI is slow and the standard video set has only three subjects27. Motion can be invented by the video generator; permutation tests are needed to show it was decoded10,15–17.
DirectionAn explicit motion-decoding head with motion-shuffled baselines and temporal metrics on CC2017. Needs an A100 for video diffusion.
06Real-time, portable decoding
+
07Generalizing beyond one dataset
+
08Brain to language, responsibly
+
Reproduce and audit
Reproduce two open decoders (for example MindEye2 and ATM) and run the faithfulness controls from gap 02 on them.
Close one gap
Propose a method for the chosen gap: few-data cross-subject decoding, or low-level fidelity for EEG.
Evaluate honestly, then publish
Test on a second dataset, release code, and submit to a NeurIPS/ICLR workshop or a MICCAI-track venue.
Data
You will learn
- fMRI and EEG preprocessing
- Contrastive alignment and diffusion priors
- Cross-subject adaptation (LoRA, alignment)
- Evaluating generative models honestly
Good fit if you
- Like neuroscience as much as generative models
- Are comfortable with PyTorch and GPUs
- Enjoy asking “is this result real?”
Data & signals
Preprocessing pipelines for NSD, THINGS and EEG; subject splits.
Baselines
Reproduces open decoders and keeps results reproducible.
Method
Designs the alignment, encoder or adapter for the chosen gap.
Evaluation
Owns metrics, faithfulness controls and figures.
The plan · 6 months
From first paper to submission.
Six months, from first paper to submission, with weekly group meetings and a monthly review with mentors.
Onboard
Read 10–15 key papers, set up data access and compute, write a one-page proposal.
Reproduce
Match a published baseline. Share a reproducibility report.
Propose
First version of the method, defended in a mentor review.
Iterate
Ablations, failure analysis, fix what the review found.
Validate
External data or pilot study; freeze results.
Submit
Paper, code release and a talk for the AIMA community.
Project I · Neuro-AI
Want to work on
this question?
Apply through the form by November 7, 2026, choosing Project I, with an academic CV and a statement of purpose of up to 300 words. Interviews run November 9 to 13, and groups start around November 18. Mentors from Duke University · Northwestern University.
References
- Takagi & Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. CVPR 2023. link
- Scotti et al. Reconstructing the Mind’s Eye: fMRI-to-image with contrastive learning and diffusion priors. NeurIPS 2023. arXiv
- Scotti, Tripathy et al. MindEye2: Shared-subject models enable fMRI-to-image with 1 hour of data. ICML 2024. PMLR
- Wang et al. MindBridge: A cross-subject brain decoding framework. CVPR 2024. arXiv
- Gong et al. MindTuner: Cross-subject visual decoding with visual fingerprint and semantic correction. AAAI 2025. link
- Dai et al. MindAligner: Explicit brain functional alignment for cross-subject visual decoding from limited fMRI data. ICML 2025. PMLR
- Wang et al. ZEBRA: Towards zero-shot cross-subject generalization for universal brain visual decoding. NeurIPS 2025. OpenReview
- Shirakawa et al. Spurious reconstruction from brain activity. Neural Networks 2025. DOI
- Mayo et al. BrainBits: How much of the brain are generative reconstruction methods using? NeurIPS 2024. arXiv
- Lu et al. Animate your thoughts: Decoupled reconstruction of dynamic natural vision from slow brain activity (Mind-Animator). ICLR 2025. arXiv
- Song et al. Decoding natural images from EEG for object recognition (NICE). ICLR 2024. arXiv
- Li et al. Visual decoding and reconstruction via EEG embeddings with guided diffusion (ATM). NeurIPS 2024. arXiv
- Benchetrit, Banville & King. Brain decoding: toward real-time reconstruction of visual perception. ICLR 2024. arXiv
- Kneeland et al. ENIGMA: A unified lightweight EEG-to-image model for multi-subject visual decoding. NeurIPS 2025 Workshop. link
- Chen, Qing & Zhou. Cinematic Mindscapes: High-quality video reconstruction from brain activity (MinD-Video). NeurIPS 2023. arXiv
- Gong et al. NeuroClips: Towards high-fidelity and smooth fMRI-to-video reconstruction. NeurIPS 2024. arXiv
- Wang et al. NEURONS: Emulating the human visual cortex improves fidelity and interpretability in fMRI-to-video reconstruction. ICCV 2025. CVF
- Li et al. The perils and pitfalls of block design for EEG classification experiments. IEEE TPAMI 2021. DOI
- Xia et al. UMBRAE: Unified multimodal brain decoding. ECCV 2024. arXiv
- Qiu et al. MindLLM: A subject-agnostic and versatile model for fMRI-to-text decoding. ICML 2025. PMLR
- Tang et al. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience 2023. DOI
- Horikawa. Mind captioning: Evolving descriptive text of mental content from human brain activity. Science Advances 2025. DOI
- Jiang et al. Large brain model for learning generic representations with tremendous EEG data in BCI (LaBraM). ICLR 2024. arXiv
- Allen et al. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence (NSD). Nature Neuroscience 2022. DOI
- Gifford et al. A large and rich EEG dataset for modeling human visual object recognition (THINGS-EEG2). NeuroImage 2022. DOI
- Hebart et al. THINGS-data: fMRI, MEG and behavioral data on object representations. eLife 2023. DOI
- Wen et al. Neural encoding and decoding with deep learning for dynamic natural vision (CC2017). Cerebral Cortex 2018. DOI
- Alljoined: EEG datasets for natural-image decoding, including consumer-grade Alljoined-1.6M. arXiv · arXiv