Translational Evidence for Retinal Foundation Models: Self-Supervised Representations, Prevalence, and Clinical Workflows

1. Self-supervised pretraining addresses label utilization, not automatically every clinical problem

The original RETFound study used approximately 160 ten-thousand unlabeled images to learn adaptable representations and evaluated them on labeled tasks. This shows that unlabeled data can support multiple downstream tasks, but the number of pretraining samples is not the number of independent patients, nor does it mean that all diseases, devices, and populations have been validated. This article recommends recording four units in parallel: image, eye, patient, visit, and institution, to avoid placing multiple images from the same patient across training and test sets. Comparisons of label efficiency also require matched adaptation budgets, input quality, and task definitions, rather than comparing only the final AUC.[1] A foundation model for generalizable disease detection

2. Representations, ocular lesions, and systemic risk are different targets

Predicting ocular status from retinal images and predicting future systemic events require different labels, follow-up, and confounding control. RETFound explores multiple types of tasks, but its conclusions remain tied to the respective samples and endpoints. This article advocates a separate task card for each adapted model version: observation window, outcome definition, censoring procedures, label sources, and exclusion criteria. If imaging occurs close to the outcome, the model may exploit disease signals that already exist but have not yet been coded; this differs from genuine long-term risk prediction. Transfer across tasks cannot be presumed merely because they share a foundation model name.[1] A foundation model for generalizable disease detection

3. Workflows with an interpretable intermediate layer still require external validation

The retinal work by De Fauw and colleagues separates tissue segmentation and referral decisions into layers, demonstrating the role of interpretable intermediate results in a particular system while retaining ambiguous and failure examples. An intermediate layer facilitates human inspection but cannot guarantee that nothing is missed or incorrect. This article recommends evaluating image quality, tissue segmentation, downstream decisions, and human handling separately; when the intermediate layer is uncertain, the system should retain an indeterminate state. Forcing every anomalous image into a category disguises measurement failure as a definite clinical conclusion.[2] Clinically applicable deep learning for diagnosis and r

4. Derivation in this article: identical sensitivity and specificity yield different meanings of a positive result

Consider a classifier with sensitivity 0.9 and specificity 0.95. At a prevalence of 1%, the expected counts among 10000 people are 90 true positives and 495 false positives, with a positive predictive value of approximately 15.4%; at a prevalence of 20%, the expected counts are 1800 true positives and 400 false positives, with a positive predictive value of approximately 81.8%. These numbers are neither RETFound performance nor actual screening recommendations; they illustrate how the composition of an external population changes workflow burden and the interpretation of positive results. Results from research case–control samples cannot be treated directly as actual yields in a low-prevalence population.作者分析与复核建议 / Author analysis

5. Discrimination, calibration, and thresholds serve different functions

AUC measures ranking-related discrimination; it does not guarantee that a given score corresponds to a particular probability. Probability calibration must be tested for the task and population, while thresholds also depend on workflow resources and the costs of errors. This article recommends retaining ROC or PR curves, reliability diagrams, and confusion matrices at prespecified thresholds, with intervals calculated using patients as the statistical unit. If a threshold is reselected at a new institution, this step should be treated as adaptation and an independent test set retained; optimizing it on all external labels and then calling the result unadjusted generalization is not acceptable.作者分析与复核建议 / Author analysis

6. External generalization must actively test differences across devices and populations

Research on distribution shifts shows that performance within the training environment is insufficient to guarantee reliability in real-world settings. Retinal tasks may be affected by devices, imaging procedures, image quality, disease spectrum, and institutional selection. This article proposes device and institution holdouts, temporal holdouts, and stratification by quality; when a group has insufficient samples, its status should explicitly be marked unknown rather than replaced with the overall mean. For both eyes or repeated visits of the same patient, the assumption of independent samples should be avoided. Group-level results also require consistent endpoint definitions; differences in institutions’ coding practices cannot simply be treated as biological differences.[3] WILDS: A Benchmark of in-the-Wild Distribution Shifts[1] A foundation model for generalizable disease detection

7. Analysis in this article: abstention and referral to human review are also system outputs

If a model abstains on some low-quality or uncertain samples, it may reduce errors among accepted samples, but it also increases human workload. This article uses fixed hypothetical proportions to illustrate the relationship among coverage, the number of abstentions, and automated processing risk, and requires that all presenting individuals remain in the denominator. An abstention strategy must not retain only the easiest subset and then report high performance; it should also examine whether abstained cases are concentrated in a particular device, disease, or population group. A clinical workflow needs to know who received no answer and whether those individuals were ultimately handled in a timely manner.作者分析与复核建议 / Author analysis

8. Error attribution should return to measurement and workflow

Failures in retinal research may arise from image quality, segmentation, label ambiguity, coexisting diseases, or downstream thresholds. This article recommends building a case-by-case error tree and checking it in an independently drawn sample, rather than merely displaying striking heatmaps. Even a heatmap focused on vessels or lesion regions does not establish that the model’s basis is entirely correct; device markings, edge information, or other nontarget cues may contribute to ranking. Important conclusions should be tested against alternative explanations through input perturbations, external devices, and appropriate clinical annotations, retaining cases that cannot be uniquely attributed.作者分析与复核建议 / Author analysis

9. From retrospective metrics to prospective workflows

Retrospective studies can test the relationship between a model and previously collected labels, whereas prospective workflows also involve acquisition failures, waiting times, human review, and cases left unprocessed. This article proposes shadow evaluation after the model is frozen, followed by workflow comparisons using preregistered endpoints; primary endpoints should match the research question, such as agreement in referral decisions, workload, or relevant outcomes, rather than assuming that improved AUC implies patient benefit. Any actual workflow study should be conducted and verified by the appropriate professional team. This article discusses only the evaluation structure and has neither implemented nor approved clinical deployment.作者分析与复核建议 / Author analysis

Technical supplement A · Masked reconstruction objectives and clinical endpoints

Self-supervised masked learning obtains training signals through reconstruction of image portions, encouraging representations to preserve predictable structure, but recovering pixels does not require understanding every clinical factor. RETFound connects pretraining with labeled adaptation; genuinely clinically relevant capabilities still require validation through downstream tasks. This article recommends examining the contributions of the pretraining objective, frozen representations, and full fine-tuning separately, with comparisons under the same label budget. If simple image statistics already explain most of the gains, this should be reported accurately rather than attributing all improvements to general medical knowledge.[1] A foundation model for generalizable disease detection

Technical supplement B · Prevalence shift differs from concept shift

If only the proportion of cases changes while the image distribution conditional on status remains unchanged, some probabilities can be recalibrated through prior adjustment; if devices, disease spectrum, or label definitions change, adjusting priors alone is usually insufficient. This article recommends first testing which type of distribution shift is present and then deciding on adaptation. Evidence for a system that performs well after recalibration at a new institution should be described as adapted evidence, not mixed with unadjusted external generalization. Every adjustment requires independent holdout validation to avoid fitting and evaluating on the same test labels.作者分析与复核建议 / Author analysis

Technical supplement C · Human–model collaboration is not simple averaging

Model prompts may shift human attention, lead to overreliance, or trigger repeated review, so team performance cannot be obtained by directly adding the independent accuracies of models and humans. This article proposes recording final decisions, reasons for revisions, processing time, and indeterminate cases in workflow studies, with prompt visibility conditions included in the study design. Primary endpoints should compare real workflows rather than treating a single offline model score as if it had already changed clinical behavior. Explanatory figures also require separate testing of whether they help detect errors, rather than merely measuring how much users like them.作者分析与复核建议 / Author analysis

Technical supplement D · Follow-up tasks must account for time and censoring

Predicting future events involves who enters the risk set and when, how long follow-up lasts, and how loss to follow-up is handled. Treating individuals with short follow-up as event-free, or acquiring images only after disease has become apparent, changes the task. This article recommends fixing the index time and prediction window, recording censoring and repeated visits, and using evaluation compatible with the endpoint. A single classification AUC cannot automatically replace time-dependent risk evaluation. Foundation models can provide representations, but they cannot eliminate these statistical conditions in cohort design.作者分析与复核建议 / Author analysis

10. Conclusion: foundation model value must be realized in specific tasks

Large-scale self-supervised learning has brought reusable representations to medical imaging, but scientific promises must still be tied to a clearly defined task and population. This article’s prevalence and workload derivations show that ranking ability, probabilities, and workflows are not the same problem. Auditable deliverables should retain patient-level splits, external conditions, thresholds, abstentions, and failure cases, with evidence reported separately for each adaptation. Such a structure can accurately demonstrate the value of foundation models while showing subsequent research which additional measurements are still needed for reliable clinical use.作者分析与复核建议 / Author analysis

Core equations and derivations

Methods and evidence workflow

Analytical figures and numerical examples

Recomputable worked example

Prevalence 1%: TP 90, FP 495, PPV≈0.153846; prevalence 20%: TP 1800, FP 400, PPV≈0.818182. These are expected counts only.

Method comparison

Verification and experimental plan

1. Fix the labels, time window, model, and input-quality rules for each adaptation task.
2. Separate training and test sets by patient rather than image, retaining links between both eyes and repeated visits.
3. Match adaptation and hyperparameter-tuning budgets, and document overlap between pretraining and external data.
4. Create holdouts and strata separately across devices, institutions, time, and populations.
5. Report discrimination, calibration, prespecified thresholds, and coverage under abstention together.
6. Include all acquisition failures and indeterminate cases in the workflow denominator.
7. Specify separate workflow endpoints for shadow or prospective studies, under the responsibility of a professional research team.
8. Publish de-identified statistics and analysis rules; the constructed numbers in this article are not for individual medical judgments.

Scope and limitations

• No patient data or actual medical model testing.
• Sensitivity and specificity are assumed values, not RETFound results.
• Different tasks and follow-up endpoints cannot be combined into general diagnostic performance.
• Distribution shifts and label bias may coexist.
• This article does not constitute diagnostic, treatment, or clinical deployment advice.

Evidence from the original studies and its scope

Research questions worth testing

1. After matching label and adaptation budgets, on which external devices are gains maintained?
2. Does the abstention strategy shift workload disproportionately onto particular groups?
3. Can improved discrimination bring substantial benefits in a prospective workflow with frozen thresholds?

Primary sources and corresponding claims

1. A foundation model for generalizable disease detection from retinal imagesZhou et al. · 2023-09-131.6 million unlabeled retinal images; task-specific adaptation and diagnosis/prognosis evaluation.
2. Clinically applicable deep learning for diagnosis and referral in retinal diseaseDe Fauw et al. · 2018-08-13Device-independent tissue segmentation and referral classification; ambiguities and failure examples.
3. WILDS: A Benchmark of in-the-Wild Distribution Shifts, , , , , , , , , , , , , , , , , , , , , , -------------------------------------------------------------------------------- · 2020-12-14Abstract; 原文方法、理论和实验章节见作者原始论文链接