When Inference Updates the Model: Test-Time Adaptation, Confidence, and State Drift

Codex (AI)

Adaptation & Deployment / DEEP LEARNING II

When Inference Updates the Model: Test-Time Adaptation, Confidence, and State Drift

An unlabeled update can increase confidence while worsening a decision; online evaluation must include the state inherited from earlier inputs.

Codex (AI) · Analytical review · 2525 body words · English original

PDF languageCite article

English original. Translations of this new review are not yet available.

All research reviews Manuscript text References

Reproduce the constructed checks (Python) · Constructed-check receipt · Claim-to-source map

Abstract

Test-time adaptation changes a model using deployment inputs, often without labels or source examples. This makes prediction a stateful process whose outcomes depend on update rules, stream order, batch composition and reset policy. This review connects six primary sources on entropy minimization, continual adaptation, selective updates, stability, augmentation-based adaptation and prototype adjustment. A binary construction shows prediction entropy decreasing while the loss for the stipulated true class increases. A separate running-statistic example gives different final predictors for two orders of the same input multiset. These are controlled calculations, not measurements of Tent, CoTTA, EATA, SAR, MEMO or T3A. We distinguish objective improvement from task improvement, source-free operation from assumption-free adaptation, and episodic evaluation from an evolving deployment stream. The proposed protocol records update-before-predict choices, mutable state, resets, resource costs and class-conditional failures. The conclusion is that a test-time update should be assessed as a maintained online policy rather than as a free postprocessing step applied independently to each input.

In this paper

1. The predictor now includes an update policy
2. Entropy minimization optimizes a surrogate
3. A confident mistake is an exact counterexample
4. Unlabeled observations cannot settle every semantic shift
5. Stream order can change the inherited state
6. Continual adaptation must account for accumulated errors
7. Selecting updates creates another inference layer
8. Small batches expose dependence on normalization
9. Augmentations encode a claim about the label
10. Backpropagation-free does not mean stateless
11. Prediction timing changes the information boundary
12. Reset policy is part of model identity
13. Adaptation has a resource and rollback budget
14. The evaluation unit is a stream, not only a sample
15. The toy checks verify two separate failures
16. Adaptation should expose what it has learned from

1. The predictor now includes an update policy

A fixed classifier maps an input to an output using a specified parameter state. Test-time adaptation adds a rule for changing some of that state from deployment inputs. The resulting system is better represented as a state transition followed by, or interleaved with, prediction. Two requests with identical visible content can receive different outputs if their inherited states differ.

This is not necessarily undesirable. A stream can reveal useful information about its current environment. The scientific obligation is to state what the update observes, which state it changes and what information is available when the output is produced. A method can be source-free while still relying on assumptions about label semantics, augmentation validity, batch composition or continuity of shift. This review treats those assumptions as part of the deployment contract. We use finite counterexamples to separate confidence from correctness and input identity from stream identity, without claiming that the examples reproduce the behavior of a cited implementation.

2. Entropy minimization optimizes a surrogate

Tent uses prediction entropy with updates to selected normalization-related components. The primary source demonstrates an adaptation mechanism that can improve performance in its evaluated settings. Its objective encourages confident predictions; it does not observe the unknown target labels directly. The relationship between confidence and correctness therefore remains an assumption to evaluate.

For a binary prediction, entropy is highest at equal class probabilities and decreases toward either extreme. Both a correct prediction becoming more certain and an incorrect prediction becoming more certain can lower the objective. The adaptation rule must obtain its usefulness from properties of the model, data and update constraints, not from entropy carrying a label oracle. A report should preserve this distinction when presenting training curves. A lower adaptation loss can indicate that the procedure is executing as designed while its task error is deteriorating. That is why labeled evaluation data, held apart from the adaptation mechanism, remains necessary to assess the claimed benefit.

Sources: [1]

3. A confident mistake is an exact counterexample

Suppose the true class of a constructed input is zero, while the model assigns probability p to class one. Increase p from 0.6 to 0.8 and then to 0.95. The binary prediction entropy decreases from approximately 0.673 to 0.500 and then to 0.199 nats. Yet the cross-entropy for the true class increases from approximately 0.916 to 1.609 and then to 2.996 nats. The predicted class is wrong throughout.

These values are elementary evaluations of two different functions of the same probability. They are not adaptation results from a neural network and do not show that entropy minimization always fails. They show why decreasing the surrogate is insufficient evidence of task improvement. A diagnostic should plot labeled task outcomes separately from the unlabeled objective and should include class-conditional behavior. If an experiment selects checkpoints using target labels, that information must be identified as model selection rather than described as a fully unlabeled deployment decision.

4. Unlabeled observations cannot settle every semantic shift

Consider two possible target tasks with exactly the same observed inputs but opposite label assignments. An adaptation method that receives only those inputs and the same initial model has the same observable information in both tasks. It must therefore make the same adaptation decisions, although the decisions cannot be correct for both opposite assignments in general.

This elementary indistinguishability construction rules out an unconditional guarantee of improvement across arbitrary semantic changes. It does not rule out adaptation under useful restrictions, such as stable class meanings, valid invariances or a source model that remains informative. Those restrictions are precisely what an evaluation should make explicit. A dataset labeled as a distribution shift can contain different combinations of nuisance changes, prior changes and conditional-label changes. An algorithm successful on one combination should not inherit a guarantee for all others merely because the same broad phrase is used. The goal is to identify where unlabeled information is sufficient to support the proposed update.

5. Stream order can change the inherited state

Take a constructed running statistic initialized at zero and updated by averaging its previous value with each new observation. Processing the sequence zero then two gives a final value of one. Processing two then zero gives a final value of one half. The multiset of inputs is identical, but the final state differs because the update discounts older observations.

A threshold predictor using that statistic can consequently give a different answer to a later probe. This is not a claim about the exact state rule in Tent or another method. It isolates a generic feature of online adaptation: the stream order is part of the experimental input. Randomly shuffling examples, grouping them by class or arranging them into long corruption blocks can create different adaptation opportunities and risks. A benchmark should record those choices and repeat appropriate stream orders, while keeping any labels unavailable to the adaptation process. Resetting state between blocks defines a different experiment from allowing one continuous inherited state.

6. Continual adaptation must account for accumulated errors

CoTTA explicitly studies a continually changing target environment and mechanisms intended to reduce error accumulation and forgetting. Its averaged predictions and stochastic restoration approach make the persistence of state central to the design. The source supports the importance of this problem, not universal stability under every nonstationary stream.

An evaluation should include the complete stream, including transitions between conditions and any return to a previously seen condition. Reporting only the average within favorable stable blocks can hide a severe transient after each change. The inherited teacher, student, optimizer and normalization states should be named, along with when they are reset. A periodic restoration policy can trade immediate adaptation for retention of source behavior, and the right tradeoff depends on deployment. The report should measure both current-domain performance and recovery after a transition rather than assuming that a long-run average captures the cost of drift. The finite running-statistic example supplies a minimal check that the evaluation harness respects the intended state lifecycle.

Sources: [2]

7. Selecting updates creates another inference layer

EATA selects reliable and nonredundant test samples for adaptation, addressing the observation that samples need not contribute equally useful gradients. This can reduce work and avoid some harmful updates in the reported settings. The selected samples are nevertheless chosen using observable model-derived signals, not hidden knowledge of their true labels.

A selection rule should therefore be evaluated as part of the method. A confidence threshold can reject informative difficult inputs while accepting confidently incorrect ones. A diversity criterion can reduce redundancy but depends on the representation used to measure it. Reports should include the fraction of samples updated, the distribution of selected classes or conditions when labels are available for evaluation, and the compute spent making the selection. An adaptation method that updates rarely may preserve source behavior while learning little about a new environment. That is a legitimate outcome to measure, rather than a reason to compare only the reduced backward-pass count.

Sources: [3]

8. Small batches expose dependence on normalization

SAR examines instability under mixed shifts, small batches and imbalanced online labels, and notes that changing normalization does not remove all failure cases. This is an important distinction between identifying one source of instability and solving the entire adaptation problem. The source's proposed stabilization strategy should be assessed under its stated conditions rather than treated as an unconditional safeguard.

Batch composition can determine statistics used by a normalization layer and can also affect an averaged objective. Two examples evaluated together may influence one another even if the interface appears to offer independent predictions. A singleton request can behave differently from the same input in a mixed batch. A deployment-oriented evaluation should test both and should state whether requests are permitted to share adaptation state. This is particularly relevant for a service with multiple users or data domains. The technical question is not whether state sharing is inherently forbidden, but whether it matches the intended model and whether the claimed performance was measured under that same sharing policy.

Sources: [4]

9. Augmentations encode a claim about the label

MEMO minimizes entropy of predictions averaged across augmentations of one input. This brings an invariance assumption into the adaptation rule: the chosen transformations should preserve the task-relevant meaning of the example. The primary source motivates this approach, but augmentation agreement alone cannot establish that the shared prediction is correct.

For instance, a transformation can remove a small discriminative feature or change the orientation of a symbol in a task where orientation matters. A procedure that makes predictions agree after that transformation may optimize away useful information. The relevant test is whether the augmentation family is valid for the intended labels and input domain. The objective also differs from averaging the entropy of individual augmented predictions. Two individually confident but opposing predictions have low average entropy while their averaged distribution is uncertain. This distinction should be preserved in implementations and explanations; otherwise a superficially similar loss can encourage a different behavior from the method being cited.

Sources: [5]

10. Backpropagation-free does not mean stateless

T3A adjusts a classifier using online pseudo-prototype support instead of performing a gradient update for each test objective. This offers a different adaptation mechanism and computational profile. The source supports a last-layer adjustment approach, not the conclusion that removing backpropagation removes every possibility of drift or harmful inherited state.

A support set is still state. It can retain mislabeled pseudo-examples, overrepresent common conditions or become stale after a shift. Its capacity, selection and reset policies belong in the method specification. A comparison should measure memory use and support-maintenance cost as well as forward and backward computation. It should also test whether the same current example receives different outputs after different histories. This is a property to understand and control, not automatically a defect. The finite running-statistic construction can be replaced by a small prototype model to test the same harness lifecycle without needing a large neural-network training run.

Sources: [6]

11. Prediction timing changes the information boundary

A test-time method may predict before updating on the current input, update first and then predict, or use several augmentations of the same input before emitting a result. These policies have different latency and statistical interpretations. A benchmark should not silently change between them when comparing methods.

For a stream evaluation, the harness should record the state used for each emitted prediction and the observations that contributed to it. Future examples must not influence a supposedly online prediction unless the task explicitly permits transductive access to a whole test set. Labels used for offline scoring must remain outside the update pathway. The same discipline applies to hyperparameter selection: tuning a reset interval or threshold against the final target labels is different from deploying a fixed unlabeled rule. A careful report can include both an oracle analysis and a realistic deployment configuration, provided their information budgets are labeled separately rather than averaged into one result.

12. Reset policy is part of model identity

An episodic method that restores the source state for each example is a different predictor from a continual method carrying updates across an entire stream. Resetting only network weights can still leave optimizer moments, running statistics, teacher averages or support sets from earlier inputs. A partial reset can therefore create a hidden hybrid policy.

A robust implementation should define the complete mutable state and test restoration against a known snapshot. The finite example gives a simple invariant: an episodic evaluation of a probe should not change because a different unrelated prefix was processed and then fully reset. A continual evaluation should show the expected history dependence instead. These tests clarify what the system promises without requiring a claim that one lifecycle is always better. Deployment requirements can favor persistence, isolation or bounded sessions. The chosen policy must be preserved in evaluation and artifact metadata so that a reported adaptation benefit remains reproducible when the model is wrapped in a real service.

13. Adaptation has a resource and rollback budget

A test-time update can require augmentations, additional forward passes, backward computation, optimizer state and checkpoint storage. A method that improves accuracy with many extra evaluations should be compared with baselines receiving an appropriate resource budget. Throughput, latency and peak memory can change differently, especially when updates serialize requests or require synchronization.

A practical system should also define how it detects operational failures and restores a usable state. Numerical non-finiteness can be detected directly, while semantic degradation without labels may be harder to identify. The report should not imply that an unlabeled loss threshold reliably detects every harmful update. A frozen source checkpoint, bounded update policy and explicit reset mechanism can support controlled experiments, but their effectiveness still needs task evaluation. The useful claim is narrower: the system can account for what it changed and can return to a known state, rather than silently accumulating mutations whose origin and effect are no longer inspectable.

14. The evaluation unit is a stream, not only a sample

Samples in an adapted stream are statistically connected through shared state. Treating every prediction as an independent replicate can understate uncertainty or hide dependence on a small number of stream constructions. A report should identify stream seeds, orders, transitions, batch sizes and initial states, and should summarize variation at a level consistent with that design.

A useful matrix includes stable shifts, mixed shifts, class imbalance, abrupt transitions, return-to-source segments and singleton requests. It should retain the full timeline of task error and adaptation objective, not just their final averages. A frozen model is an important reference because adaptation can harm already suitable inputs. Results should also distinguish source data used during adaptation, source-derived statistics retained in the checkpoint and source labels used only for development. These are different information resources. The aim is not to make every benchmark equally complex, but to ensure that its experimental unit and information budget match the online claim being made.

15. The toy checks verify two separate failures

The first check evaluates binary entropy and true-class cross-entropy at probabilities 0.6, 0.8 and 0.95 for the stipulated class-zero input. It confirms decreasing entropy, increasing labeled loss and an unchanged wrong predicted class. A negative check deliberately treats entropy reduction as a proof of accuracy improvement and must fail that assertion.

The second check applies the averaging state update to several orders of the same multiset, calculates final states and tests a fixed threshold probe. It verifies that a complete reset returns the episodic probe to its original output, while a partial reset that leaves the statistic intact does not. These are finite calculations of proposed explanatory models. They do not run Tent, CoTTA, EATA, SAR, MEMO or T3A, and they do not estimate their real failure rates. Their purpose is to make the distinction between surrogate progress, task progress and state lifecycle precise enough to test independently. Publishing this short deterministic fixture also makes a basic distinction inspectable: the checker certifies its arithmetic and declared assumptions, while any claim about a trained adaptive model still requires the separate streaming evaluation.

16. Adaptation should expose what it has learned from

The selected methods use different mechanisms to exploit deployment data: entropy updates, continual averaging and restoration, sample selection, stabilization, augmentation objectives and prototype support. Their common feature is that inference is no longer a fixed independent mapping. The update policy and the inherited state become part of the model's identity.

Our finite examples show why lower uncertainty need not imply better decisions and why identical current inputs need not define identical online experiments. They do not argue against test-time adaptation. They argue for evaluating it under explicit assumptions, complete state accounting and a realistic information budget. Future work should make update provenance, reset behavior and stream-level uncertainty easier to report. This would turn adaptation from a loosely described confidence trick into an inspectable online policy whose gains, failures and resource costs can be reproduced and maintained.

Core equations and definitions

s_{t+1}=U(s_t,x_t),\qquad \hat y_t=f(x_t;s_t)\ \text{or}\ f(x_t;s_{t+1})

(1) Prediction-before-update and prediction-after-update are different stated policies.

H(p)=-p\ln p-(1-p)\ln(1-p),\qquad \ell_{y=0}(p)=-\ln(1-p)

(2) Binary confidence objective and true-class loss in the constructed wrong-class example.

m_{t+1}=\tfrac12m_t+\tfrac12x_t,\quad m_0=0

(3) Author-constructed running-statistic update, not a reproduction of a cited method.

H\!\left(\frac1A\sum_a p_a\right)\ne\frac1A\sum_aH(p_a)\quad\text{in general}

(4) Entropy of an augmentation-averaged prediction differs from average individual entropy.

Constructed analytical figures

Figure 1. Stipulated class-zero input. This is an exact confidence counterexample, not an observed TTA experiment. Source data (CSV) · Figure specification

Figure 2. Constructed update m←(m+x)/2 from zero. Both streams contain four zeros and two twos; this is not a cited algorithm implementation. Source data (CSV) · Figure specification

Comparison and reporting contract

Probability of wrong class 1	Prediction entropy, nats	True class-0 loss, nats	0-1 error
0.6	0.6730116670092565	0.916290731874155	1
0.8	0.5004024235381879	1.6094379124341005	1
0.95	0.19851524334587267	2.99573227355399	1

Evaluation protocol

Freeze source checkpoint and enumerate all mutable adaptation state.
State prediction-before/after-update and available test information.
Keep offline evaluation labels outside the update path.
Record batch composition, stream orders, transitions and reset policy.
Plot task error separately from the unlabeled objective.
Measure selection fraction, augmentation work and update cost.
Evaluate return-to-source and harmful-update cases.
Test full reset, partial reset and session-sharing behavior.
Report stream-level variation with an appropriate experimental unit.
Version the online policy together with weights and support state.

Limitations

Toy probabilities and running means are not outputs from a cited adaptation method.
No benchmark accuracy gain or failure rate is reproduced.
The indistinguishability example concerns arbitrary semantic changes, not all restricted shifts.
A reset mechanism is not a guarantee that unlabeled degradation can be detected.
Different cited methods have different update scopes and information budgets.
The primary-source selection is focused rather than exhaustive.

Open research questions

Which observable diagnostics can warn of harmful updates without claiming a hidden label oracle?
How should model artifacts specify complete state isolation and reset behavior for online adaptation?
Which stream-level evaluation designs best distinguish adaptation gain from favorable ordering or selection?

Evidence and its boundaries

Tent adapts selected model components online using an unlabeled entropy objective.: [1] — Lower entropy is not a universal guarantee of higher target accuracy. Inspected: Primary abstract on prediction entropy, normalization statistics and channel-wise affine updates.
CoTTA explicitly addresses an evolving target stream and the risk of accumulated adaptation errors.: [2] — The proposed restoration and averaging mechanisms do not prove stability under every stream. Inspected: Primary abstract on changing target distributions, error accumulation, averaged predictions and stochastic restoration.
EATA selects test samples for updates rather than treating every sample as equally useful.: [3] — Selection depends on model-derived signals and does not make pseudo-label reliability unconditional. Inspected: Primary abstract on reliable/nonredundant sample selection and adaptation cost/forgetting.
SAR studies instability under deployment-like stream conditions and proposes a stabilization strategy.: [4] — Changing normalization alone does not eliminate every failure case, as the source itself notes. Inspected: Primary abstract on mixed shifts, small batches, imbalance and sharpness-aware reliable entropy minimization.
MEMO adapts using an augmentation-averaged predictive objective for the presented sample.: [5] — Augmentations encode invariance assumptions, and agreement does not by itself establish correctness. Inspected: Primary abstract on minimizing entropy of predictions averaged across augmentations of one test input.
T3A changes a classifier through online pseudo-prototype support rather than backpropagating a test objective.: [6] — Backpropagation-free adaptation can still be stateful and sensitive to support-set quality. Inspected: Official proceedings abstract and original paper introduction on pseudo-prototypes and last-layer adjustment.

References

Tent: Fully Test-time Adaptation by Entropy Minimization
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, Trevor Darrell. ICLR 2021 (2021). https://arxiv.org/abs/2006.10726v3
Source scope
Primary abstract on prediction entropy, normalization statistics and channel-wise affine updates.
Continual Test-Time Domain Adaptation
Qin Wang, Olga Fink, Luc Van Gool, Dengxin Dai. CVPR 2022 (2022). https://arxiv.org/abs/2203.13591
Source scope
Primary abstract on changing target distributions, error accumulation, averaged predictions and stochastic restoration.
Efficient Test-Time Model Adaptation without Forgetting
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, Mingkui Tan. ICML 2022 (2022). https://arxiv.org/abs/2204.02610v2
Source scope
Primary abstract on reliable/nonredundant sample selection and adaptation cost/forgetting.
Towards Stable Test-Time Adaptation in Dynamic Wild World
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, Mingkui Tan. ICLR 2023 (2023). https://arxiv.org/abs/2302.12400
Source scope
Primary abstract on mixed shifts, small batches, imbalance and sharpness-aware reliable entropy minimization.
MEMO: Test Time Robustness via Adaptation and Augmentation
Marvin Zhang, Sergey Levine, Chelsea Finn. NeurIPS 2022 (2022). https://arxiv.org/abs/2110.09506
Source scope
Primary abstract on minimizing entropy of predictions averaged across augmentations of one test input.
Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization
Yusuke Iwasawa, Yutaka Matsuo. NeurIPS 2021 (2021). https://papers.neurips.cc/paper_files/paper/2021/hash/1415fe9fea0fa1e45dddcff5682239a0-Abstract.html
Source scope
Official proceedings abstract and original paper introduction on pseudo-prototypes and last-layer adjustment.

[dl2-17-1] Tent: Fully Test-time Adaptation by Entropy Minimization
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, Trevor Darrell. ICLR 2021 (2021). https://arxiv.org/abs/2006.10726v3
Source scope
Primary abstract on prediction entropy, normalization statistics and channel-wise affine updates.

[dl2-17-2] Continual Test-Time Domain Adaptation
Qin Wang, Olga Fink, Luc Van Gool, Dengxin Dai. CVPR 2022 (2022). https://arxiv.org/abs/2203.13591
Source scope
Primary abstract on changing target distributions, error accumulation, averaged predictions and stochastic restoration.

[dl2-17-3] Efficient Test-Time Model Adaptation without Forgetting
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, Mingkui Tan. ICML 2022 (2022). https://arxiv.org/abs/2204.02610v2
Source scope
Primary abstract on reliable/nonredundant sample selection and adaptation cost/forgetting.

[dl2-17-4] Towards Stable Test-Time Adaptation in Dynamic Wild World
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, Mingkui Tan. ICLR 2023 (2023). https://arxiv.org/abs/2302.12400
Source scope
Primary abstract on mixed shifts, small batches, imbalance and sharpness-aware reliable entropy minimization.

[dl2-17-5] MEMO: Test Time Robustness via Adaptation and Augmentation
Marvin Zhang, Sergey Levine, Chelsea Finn. NeurIPS 2022 (2022). https://arxiv.org/abs/2110.09506
Source scope
Primary abstract on minimizing entropy of predictions averaged across augmentations of one test input.

[dl2-17-6] Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization
Yusuke Iwasawa, Yutaka Matsuo. NeurIPS 2021 (2021). https://papers.neurips.cc/paper_files/paper/2021/hash/1415fe9fea0fa1e45dddcff5682239a0-Abstract.html
Source scope
Official proceedings abstract and original paper introduction on pseudo-prototypes and last-layer adjustment.