AIMA 2026 projects ✦ Project IX ✦ Radiology
Grounded CT reports, across hospitals
A CT has hundreds of slices and one report. Vision-language models can now draft that report, but a fluent sentence can be wrong, and a model trained at one hospital can fail at the next. We will build report generation that ties every finding to the image and is tested across hospitals, so that a radiologist can trust it, check it and edit it.
In partnership with
Every sentence points at the scan
CTs: Site A, Sean Novak, Wikimedia Commons, CC BY-SA 4.0; Sites B and C, Hellerhoff, polycystic liver and rectus sheath haematoma, Wikimedia Commons, CC BY-SA 4.0 (axial panels cropped). The true findings follow each image’s published description; the wrong sentence in each draft was written for this page to show a typical error.
What makes this possible now
- Multi-center CT at scale: AbdomenAtlas has 20,460 CTs from 112 hospitals2, and AbdomenAtlas 3.0 adds 9,262 CT-mask-report triplets written with radiologists; segmentation made generated reports better at finding tumours3.
- Generalist CT models: CT-RATE pairs 25,692 chest CTs with reports4; Merlin was tested on 44,098 CTs from several sites5.
- Clinicians and a report model made different errors on chest X-rays, so working together can help1.
The gap we target
Most report metrics were built for chest X-rays6,7; whether they track clinically significant errors8 on 3D CT is untested. Report models are rarely tested at a hospital they never saw, and few tie each sentence to the image. Reports themselves can also teach segmentation, which could make grounding cheap9.
Reproduce the field
Run open CT report models on AbdomenAtlas 3.0 and score them with several metrics and a small radiologist error audit.
Ground every finding
Segmentation-grounded generation that links each sentence to a region and drops claims with no support in the image.
New hospitals, real readers
Zero-shot tests on CTs from other institutions, and a small reader study: clinically significant errors with and without AI drafts.
Data
You will learn
- 3D vision-language models
- Grounding with segmentation
- Evaluating generated text clinically
- Designing a reader study
Good fit if you
- Like LLMs and want them to be right
- Are comfortable with 3D data and big GPUs
- Want radiologists to actually use what you build
Data & reports
CT-report pairs, splits by hospital.
Generation
3D vision-language baselines and training.
Grounding
Segmentation links and claim checking.
Evaluation
Metrics, error audit and the reader study.
The plan · 6 months
From first paper to submission.
Six months, from first paper to submission, with weekly group meetings and a monthly review with mentors.
Onboard
Read 10–15 key papers, set up data access and compute, write a one-page proposal.
Reproduce
Match a published baseline. Share a reproducibility report.
Propose
First version of the method, defended in a mentor review.
Iterate
Ablations, failure analysis, fix what the review found.
Validate
External data or pilot study; freeze results.
Submit
Paper, code release and a talk for the AIMA community.
Project IX · Radiology
Want to work on
this question?
Apply through the form by November 7, 2026, choosing Project IX, with an academic CV and a statement of purpose of up to 300 words. Interviews run November 9 to 13, and groups start around November 18. Mentors from Northwestern University · Johns Hopkins University.
References
- Tanno et al. Collaboration between clinicians and vision–language models in radiology report generation. Nature Medicine 2025. DOI
- Li et al. AbdomenAtlas: A large-scale, detailed-annotated, & multi-center dataset for efficient transfer learning and open algorithmic benchmarking. Medical Image Analysis 2024. DOI
- Bassi et al. RadGPT: Constructing 3D Image-Text Tumor Datasets. ICCV 2025. DOI
- Hamamci et al. Generalist foundation models from a multimodal dataset for 3D computed tomography. Nature Biomedical Engineering 2026. DOI
- Blankemeier et al. Merlin: a computed tomography vision–language foundation model and dataset. Nature 2026. DOI
- Jain et al. RadGraph: Extracting Clinical Entities and Relations from Radiology Reports. NeurIPS 2021 Datasets & Benchmarks · arXiv. arXiv
- Zhang et al. ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generation. arXiv 2024. arXiv
- Ostmeier et al. GREEN: Generative radiology report evaluation and error notation. Findings of EMNLP 2024. ACL Anthology
- Bassi et al. Learning Segmentation from Radiology Reports. MICCAI 2025 · arXiv. arXiv
- Wasserthal et al. TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images. Radiology: Artificial Intelligence 2023. DOI