On June 11, the research group led by Professor Xuejun Qian from the School of Biomedical Engineering at ShanghaiTech University, in collaboration with the Eye & ENT Hospital of Fudan University and other institutions, published a research paper titled “An ultrasound foundation model for the stratification of vision impairment and eye cancer risk” online in npj Digital Medicine. The study proposed SonoEye, an ophthalmic ultrasound vision-language foundation model, and for the first time established Eye-RADS, an ophthalmic risk stratification system. It realized full-process intelligent analysis from abnormality screening and disease diagnosis to risk stratification and report generation, providing important technical support for large-scale eye disease screening, early warning of vision impairment, and early detection of ocular tumors in the context of an aging society.
As global population aging intensifies, the disease burden of vision impairment and ocular tumors continues to rise. Current ophthalmic screening and risk assessment still rely heavily on the experience of specialized physicians and lack unified, standardized, and easily scalable tools. Compared with fundus photography and optical coherence tomography (OCT), ophthalmic ultrasound is low-cost, has strong penetration capability, and is applicable to a wide range of populations, including those with vitreous opacity. It is an important means for evaluating adolescent myopia, age-related cataracts, retinal diseases, and intraocular tumors. However, issues such as low image contrast and strong operator dependence have limited its widespread application.
To address the above challenges, the research team conducted contrastive learning pretraining based on 215,356 paired ultrasound image-report datasets from 70,452 patients, constructing one of the largest vision-language pretraining systems to date in the field of ophthalmic ultrasound. Through three key innovations, the study achieved clinical-grade intelligent analysis of eye diseases: establishing a unified representation space between ultrasound images and clinical reports through vision-language contrastive learning; introducing an attention-based multi-instance learning module to realize patient-level fusion reasoning from multi-view ultrasound images; and combining a clinical knowledge base with a prototype learning mechanism to achieve fine-grained differential diagnosis covering 18 common ophthalmic diseases.
The results showed that SonoEye achieved 98.3% sensitivity and 98.9% AUC in the eye disease screening task, effectively identifying abnormal patients requiring further examination. In the differential diagnosis task of 18 ophthalmic diseases, the patient-level average accuracy reached 96.3%, and stable performance was maintained across multiple external testing centers. In particular, the model demonstrated excellent diagnostic capability in key ultrasound-related diseases such as retinal detachment, retinal tears, and intraocular tumors.
In addition, the research team proposed Eye-RADS (Eye Reporting and Data System), a risk stratification system, for the first time, which uniformly classifies complex ophthalmic diseases into four levels: normal, low risk for vision impairment, high risk for vision impairment, and tumor risk. After incorporating age factors, the model’s risk assessment performance in the elderly population was further improved. In addition, SonoEye can automatically generate structured diagnostic reports by leveraging its vision-language alignment capability and provide interpretable heatmaps to help physicians understand the regions of interest identified by the model. In a reading experiment involving ophthalmologists with different levels of experience, AI assistance significantly improved the diagnostic accuracy of non-specialist and young physicians, demonstrating its important application value in primary healthcare institutions and medically underdeveloped regions.
ShanghaiTech University undergraduate student Zicheng Zhou, Fudan University master’s student Xin Chen, and ShanghaiTech University doctoral student Dongsheng Yu are co-first authors of the paper. Professor Xuejun Qian from the School of Biomedical Engineering at ShanghaiTech University, Director Ting Zhang of the Eye & ENT Hospital of Fudan University, and Director Jie Guo are co-corresponding authors of the paper. ShanghaiTech University is the first completing institution.
Paper link:
https://doi.org/10.1038/s41746-026-02870-5
