AIMA 2026 projects ✦ Project III ✦ AI for medical education

AI-guided eye-gaze training for medical students & junior radiologists

Human-CenteredTranslational

AI now matches or beats less-experienced readers on several imaging tasks. Instead of only giving learners the answer, we want AI that teaches them how to find it: software that learns from where expert radiologists look, then guides a student’s gaze toward the regions and the search pattern an expert would use.

In partnership with

  • Northwestern University
  • University of Houston

Learning where to look

    Chest radiograph of the same pleural tumour: Sean Novak, Wikimedia Commons, CC BY-SA 4.0. Scanpaths are illustrative: numbered circles are fixations (larger means longer), the glow is the gaze heatmap.

    The evidence

    • Deep-learning systems outperformed non-radiologist physicians and radiologists on chest radiographs, and improved every reader as a second opinion2,3.
    • A stand-alone mammography AI was non-inferior to the average of 101 radiologists4.
    • LLM help raised residents’ brain-MRI diagnostic accuracy far more than neuroradiologists’5, and wrong AI suggestions misled inexperienced readers the most1.

    The gap we target

    Gaze already helps train better models8–11, and public chest X-ray gaze datasets exist6,7. Yet a systematic review found little evidence that teaching novices to “search like an expert” improves diagnosis12. Whether AI-guided gaze training works is an open question, and we will test it.

    Aim 1

    Model the expert

    Train on expert-radiologist gaze only to predict, for a new image, the heatmap and the ordered scanpath an expert would follow.

    Aim 2

    Guide the learner

    Software that tracks a learner’s gaze (webcam or low-cost tracker), compares it with the expert model, and suggests the next region to check and what it has missed.

    Aim 3

    Does it teach?

    A pilot with medical students: guided versus unguided practice, measuring detection, coverage and whether the skill stays once guidance is turned off.

    Data

    • REFLACX: 3,032 cases with gaze and dictation7
    • EGD-CXR: 1,083 cases with gaze and reports6
    • Both use MIMIC-CXR images: PhysioNet credentialed access and data use agreements required

    You will learn

    • Eye-tracking data and saliency / scanpath models
    • Vision transformers for medical images
    • Designing an education study

    Good fit if you

    • Are a medical student or have a clinical interest
    • Like building interactive tools
    • Want to study how people learn
    1
    Gaze data

    REFLACX / EGD-CXR processing, fixations and scanpaths.

    2
    Expert model

    Heatmap and scanpath prediction from expert gaze.

    3
    Guidance app

    Real-time gaze capture and the guidance interface.

    4
    Education study

    Study design, metrics and analysis with mentors.

    ✦

    The plan · 6 months

    From first paper to submission.

    Six months, from first paper to submission, with weekly group meetings and a monthly review with mentors.

    Month 1
    Month 2
    Month 3
    Month 4
    Month 5
    Month 6
    III · Gaze guidance
    Access & gaze dataExpert gaze modelGuidance appLearner pilotWrite & submit
    Onboarding Baselines Method Experiments & studies Writing
    M1
    Onboard

    Read 10–15 key papers, set up data access and compute, write a one-page proposal.

    M2
    Reproduce

    Match a published baseline. Share a reproducibility report.

    M3
    Propose

    First version of the method, defended in a mentor review.

    M4
    Iterate

    Ablations, failure analysis, fix what the review found.

    M5
    Validate

    External data or pilot study; freeze results.

    M6
    Submit

    Paper, code release and a talk for the AIMA community.

    Project III · AI for medical education

    Want to work on
    this question?

    Apply through the form by November 7, 2026, choosing Project III, with an academic CV and a statement of purpose of up to 300 words. Interviews run November 9 to 13, and groups start around November 18. Mentors from Northwestern University · University of Houston.

    Apply by November 7 All nine projects

    References

    1. Dratsch et al. Automation bias in mammography: The impact of artificial intelligence BI-RADS suggestions on reader performance. Radiology 2023. DOI
    2. Hwang et al. Development and validation of a deep learning–based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Network Open 2019. link
    3. Nam et al. Development and validation of deep learning–based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology 2019. DOI
    4. Rodriguez-Ruiz et al. Stand-alone artificial intelligence for breast cancer detection in mammography: Comparison with 101 radiologists. JNCI 2019. DOI
    5. Performing best when needed least: Reader experience shapes accuracy gains in LLM-assisted brain MRI differential diagnosis. Radiology 2026. DOI
    6. Karargyris et al. Creation and validation of a chest X-ray dataset with eye-tracking and report dictation for AI development (EGD-CXR). Scientific Data 2021. DOI
    7. Bigolin Lanfredi et al. REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays. Scientific Data 2022. DOI
    8. Bhattacharya et al. GazeRadar: A gaze and radiomics-guided disease localization framework. MICCAI 2022. link
    9. Bhattacharya et al. RadioTransformer: A cascaded global-focal transformer for visual attention-guided disease classification. ECCV 2022. arXiv
    10. Wang et al. Follow my eye: Using gaze to supervise computer-aided diagnosis. IEEE TMI 2022. arXiv
    11. Ma et al. Eye-gaze-guided vision transformer for rectifying shortcut learning. IEEE TMI 2023. DOI
    12. van der Gijp et al. How visual search relates to visual diagnostic performance: A narrative systematic review of eye-tracking research in radiology. Advances in Health Sciences Education 2017. DOI