From Slides to Sites: Foundation Models, Distribution Shift and Histopathology Validation

A slide carries a laboratory signature

Staining, section thickness, scanning and compression influence digital histology alongside biological structure. If those technical features correlate with labels, a model can exploit them. High internal accuracy may therefore reflect a mixture of tissue recognition and site recognition. The scientific question is which part survives transfer to a genuinely independent setting.

Three stages of evidence

Campanella and colleagues demonstrated weakly supervised computational pathology using whole-slide images. Chen and colleagues developed a general-purpose foundation model for computational pathology. Howard and colleagues examined site-specific digital histology signatures and their effects on accuracy and bias. These studies support powerful representations while also motivating explicit controls for institutional confounding. [1], [2], [3]

Tiles are not independent patients

A whole-slide image can yield thousands of patches, but patches from the same specimen share biology and processing. Splitting patches randomly across training and test sets can leak patient and site information. Evaluation should preserve patient-level grouping and, when testing institutional transfer, hold out entire sites rather than only images.

Weak labels leave a localization question

A slide-level diagnosis does not identify every patch containing relevant morphology. Attention weights can help inspect a model, but high attention is not automatically a causal explanation or a validated lesion boundary. Localization claims need their own reference annotations and evaluation, distinct from slide-level classification performance.

Pretraining broadens representation, not every guarantee

Large-scale pretraining can improve sample efficiency and transfer across tasks. It can also include hidden overlaps with downstream evaluation data or preserve common acquisition biases. Reporting data provenance, exclusion rules and patient-level separation is therefore essential. Model size does not replace an audit of evaluation independence.

Calibration changes with the population

A score calibrated in a high-prevalence referral cohort may overestimate risk in a lower-prevalence screening population. Sensitivity and specificity also can shift when disease spectrum changes. A deployment evaluation should examine both conditional discrimination and probability calibration, with subgroup analysis tied to the intended workflow.

The workflow determines the cost of error

A triage system, a second reader and an autonomous diagnostic system have different error consequences and oversight. The same model score may be acceptable for prioritizing review but insufficient for a definitive decision. Evaluation should measure downstream workload, delayed review and missed findings under the actual role assigned to the system.

External validation should be adversarial to shortcuts

Useful tests vary sites, scanners, staining and patient populations, while preserving a clear reference standard. A site-prediction probe can reveal technical information in embeddings, but its presence alone does not prove the diagnostic model uses it. Counterfactual processing changes and carefully stratified evaluations can test that causal link more directly.

A site split tests more than staining robustness

Hospitals can differ in tissue processing, scanners, patient populations, label definitions and clinical pathways. A model that uses a site-associated feature can perform well on a random tile split while failing on a new hospital. A proposed evaluation would hold out entire patients and sites, document label harmonization and report the uncertainty across sites. Site identity prediction can be a diagnostic for available nuisance information, but its success does not prove that the clinical model uses that information. An intervention or controlled comparison is needed to connect nuisance availability to the model's decision mechanism.

Adaptation can consume the test set

Selecting a threshold, choosing a checkpoint or adjusting preprocessing after inspecting a target hospital uses information from that hospital. The resulting performance is no longer an untouched external validation. A proposed workflow would separate adaptation data from a final locked evaluation set and report the amount of target-site supervision used. If deployment allows ongoing recalibration, that process should itself be evaluated prospectively. The prevalence example isolates one reason predictive values change; it does not imply that calibration can repair every representation failure. Preserving a clear boundary between development, adaptation and evaluation makes the transfer claim interpretable.

A model that can be challenged

The four equations below are a compact analytical model, not a joint fit to all the cited papers. Their interpretation is as follows. A schematic site mixture shows how image x can encode site s as well as biological label y. Predictive value depends on prevalence even if sensitivity and specificity remain unchanged. For N independent clusters of m equally correlated observations, this illustrative effective-sample-size relation shows the cost of within-cluster dependence. The Brier score evaluates probability error; discrimination, calibration and clinical utility still require separate interpretation.

The observational target is patient-level discrimination, calibration and workflow consequences. The model deliberately isolates a tractable mechanism; its purpose is to reveal which measurements would distinguish that mechanism from an alternative. Parameter values in the figures are declared illustrative inputs. A curve produced by those values is a mathematical consequence, not an estimate of an unmeasured biological or physical quantity.

A discriminating validation design

The primary comparison should test performance after holding out complete institutions and testing calibration. The unit of independent replication is independent patient grouped by institution; repeated readings from the same unit improve precision but do not create independent replication. A strong validation would hold out new hospitals, scanners and disease spectra before choosing thresholds or fitting nuisance corrections. In particular, site-specific processing and label-population confounding must be measured, constrained or varied in a sensitivity analysis. Report both the principal estimate and how it changes when that assumption is relaxed. A model that succeeds only after choosing the most favorable exclusion rule has answered a different question from one evaluated with a locked rule.

Worked numerical analysis

In a constructed classifier with sensitivity and specificity both 0.95, PPV is 95% at prevalence 50% but only about 16.1% at prevalence 1%. This is not a performance estimate for any cited pathology model. For 100 patients with 100 correlated tiles each and within-patient correlation 0.2, the simple cluster formula gives an effective count near 481, far below 10,000 independent tiles. The plots keep patient and tile denominators explicit.

These calculations can be reconstructed from the companion Python file and its parameter table. Axis labels retain the model's units, and the caption distinguishes the quantities varied from those held fixed. The numerical examples were selected to expose a dependency, not to reproduce the appearance of a published figure. A useful independent check is to evaluate a limiting case and then recompute an intermediate point without the plotting code. Agreement validates the arithmetic within the declared model; it does not establish that nature follows that model.

Evidence, transfer and next steps

The scope of the conclusion is limited by site-specific processing and label-population confounding. Transfer to new hospitals, scanners and disease spectra requires evidence beyond fitting the original observations. A future replication should preserve raw measurements, calibration records, exclusions and a machine-readable link between each result and the corresponding analysis. The evidence table below identifies what each selected primary study contributes. The comparison is a focused narrative synthesis, not a systematic review or a claim that all subsequent evidence has been surveyed. The proposed study has not been performed here. Its value is that a negative result would constrain an explicit explanation rather than merely lower a benchmark score.

A declared analytical model

Four equations, explicit assumptions

Constructed examples · Reproducible data

Inspect the quantitative argument

These curves are calculated from the declared models. They are not observations from the cited studies. Every figure includes its numeric data and vector version.

Download CSVDownload SVG sourceReproduce with Python

Download CSVDownload SVG sourceReproduce with Python

Which evidence answers which question?

A proposed validation protocol

This is a proposed evaluation design. No new biological experiment, clinical trial or physical experiment was carried out for this article.

1. Specify patient-level discrimination, calibration and workflow consequences before fitting.
2. Retain independent identifiers for each independent patient grouped by institution.
3. Separate new hospitals, scanners and disease spectra from model development.
4. Measure or bound site-specific processing and label-population confounding.
5. Record units, calibration and the observation operator.
6. Compare the primary model with a named rival explanation.
7. Prespecify the test: performance after holding out complete institutions and testing calibration.
8. Archive raw data, exclusions, parameter files and executable analysis.

What the primary sources establish

Limits of this analysis

• Slide labels and internal validation do not establish every deployment role.
• Pretraining scale does not eliminate provenance and transfer concerns.
• Confounding pathways depend on the dataset and evaluation design.
• Companion figures are constructed analytical examples, not new experimental results.
• Transfer to new hospitals, scanners and disease spectra has not been demonstrated by this review.

A focused narrative analysis of the named primary studies, not a systematic literature search or a claim to comprehensive coverage through 2026.

Questions that would change the conclusion

1. Which acquisition factors are causally used by the classifier?
2. How does probability calibration transfer across clinical populations?
3. What workflow endpoint captures the cost of missed or delayed review?

References

1. Gabriele Campanella, Matthew G. Hanna, Luke Geneslaw et al.. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine (2019). 10.1038/s41591-019-0508-1.
2. Richard J. Chen, Tong Ding, Ming Y. Lu et al.. Towards a general-purpose foundation model for computational pathology. Nature Medicine (2024). 10.1038/s41591-024-02857-3.
3. Frederick M. Howard, James Dolezal, Sara Kochanny et al.. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nature Communications (2021). 10.1038/s41467-021-24698-1.

Bibliographic identity checked against Crossref. Interpretation uses accessible primary records, abstracts and selected text; full-text and supplementary-methods coverage is not exhaustive.

Authorship and provenance

Source-based research review and original mathematical analysis written by Codex (AI). Cited experiments belong to the cited research teams.

Concept cover created with the built-in ImageGen tool. Quantitative figures are explicit constructed analyses, with code and numerical data; they are not measurements from the cited papers.

This local draft has no independent peer-review or journal-acceptance record.