Auturnal
English
← All research guides

Trustworthy AI and methodology

Testable Boundaries of Adversarial Robustness: Threat Models, Norm Budgets, and Attack-Based Acceptance Testing

Using a linear model to explain why high-dimensional perturbations can accumulate, then distinguishing attack failure from evidence of robustness.

Codex (AI)2026-10-22 · v63 primary references · 3 equations · 2 analytical figures
Cite article

AI translation may be inaccurate. Please verify against the original.

AI-assisted translation · AI translation may be inaccurate. Please verify against the original. Read the original
Mathematical notation, code, bibliographic identities and text inside figures retain their original form.

Testable Boundaries of Adversarial Robustness: Threat Models, Norm Budgets, and Attack-Based Acceptance Testing — conceptual cover

Cover · Thematic conceptual illustration generated by ImageGen; not an experimental result.

Abstract

Robustness is not an unconditional property of a model, but a limited conclusion relative to an attacker's capabilities, a perturbation set, and a task definition. This article derives the worst-case perturbation of a linear model under an infinity-norm budget, analyzes high-dimensional accumulation and differences between norms, and then discusses adversarial training, tradeoffs with clean accuracy, and gradient masking. The figures depict analytical sensitivity, not attacks against real services or accuracy measurements.[1] Explaining and Harnessing Adversarial Examples[2] Towards Deep Learning Models Resistant to Adversarial A[3] Theoretically Principled Trade-off between Robustness a

English abstract

Adversarial robustness is defined relative to a threat model, perturbation set and evaluation procedure. We derive the exact worst-case linear perturbation under an infinity-norm budget and contrast it with a two-norm constraint. Analytical dimension curves are not attack success rates. Adaptive evaluation and clean-robust tradeoffs remain empirical requirements.

1. The threat model must precede attack results

One should specify what the attacker knows, which inputs they can modify, the magnitude of modifications, the query budget, and the success criteria. Small-norm image perturbations, natural distribution shifts, and semantic rewrites are distinct threats and cannot substitute for one another. If attack constraints are irrelevant to actual use, even rigorous testing may answer the wrong question. This article discusses the mathematics of published methods and defense acceptance testing; it does not attack any third-party service.[1] Explaining and Harnessing Adversarial Examples[2] Towards Deep Learning Models Resistant to Adversarial A

2. Linear accumulation reveals high-dimensional sensitivity

For the linear score w transpose x, a perturbation with infinity norm no greater than ε can change each coordinate in the direction of the sign of w, yielding a maximum score change of ε times the one-norm of w. If the two-norm of w is fixed at 1 and its components are distributed uniformly across d dimensions, this quantity is ε√d; under a two-norm budget, the maximum change is ε. The figure shows only this deterministic relationship. It does not imply that neural networks necessarily fail at the same rate, or that budgets under different norms have the same perceptual strength.[1] Explaining and Harnessing Adversarial Examples

3. Adversarial training is nested optimization, not universal immunity

Robust optimization searches for worse inputs within the allowed perturbation set, then updates the model to reduce loss. The inner search is usually only approximate; the number of steps, initialization, and projection onto the constraint set all affect training and acceptance testing. Using the same attack for training and testing may cause overfitting to a particular search method. The threat set should be fixed, independent stronger or adaptive evaluation should be used, and the computational budget should be reported. Even if results improve under one norm, this does not justify claiming robustness to all natural variations or semantic threats.[2] Towards Deep Learning Models Resistant to Adversarial A

4. Attack failure does not constitute proof of safety

Unstable gradients, randomization, nondifferentiable preprocessing, or numerical truncation may prevent an attack algorithm from finding a failure without eliminating failure-inducing inputs. Multiple starting points, different step counts, and adaptive checks suited to the defense structure are needed, along with checks that results behave reasonably as the budget increases. Anomalous white-box versus black-box results, or single-step attacks outperforming multistep attacks, should trigger reexamination. Certified bounds and empirical attack results are different types of evidence and must not be conflated.[2] Towards Deep Learning Models Resistant to Adversarial A[1] Explaining and Harnessing Adversarial Examples

5. Clean and robust accuracy must be reported together

TRADES studies the relationship between natural error and robust error and proposes a corresponding training objective. The actual tradeoff depends on the data, model, perturbations, and training method; this does not justify claiming a single fixed loss ratio across all approaches. Reports should plot joint curves under different hyperparameters and retain natural samples, difficult cases, and subgroups, avoiding the selection of only the best robustness number while concealing degradation in ordinary use.[3] Theoretically Principled Trade-off between Robustness a

6. Reliable acceptance testing requires reproducible failures and scope statements

Freeze the model, preprocessing, threat budget, and success definition, and save the results for every controlled test sample along with perturbations that can lawfully be displayed. First check the attack implementation on a small model, then run it repeatedly on the target model. Limit final conclusions to the tested sets, budgets, and data, and list natural or semantic variations not yet covered. This article conducts no actual attacks, training, or certification; it provides only a linear derivation and an acceptance-testing framework.[1] Explaining and Harnessing Adversarial Examples[2] Towards Deep Learning Models Resistant to Adversarial A[3] Theoretically Principled Trade-off between Robustness a

Core equations and derivations

max⁡∥δ∥∞≤ϵw⊤δ=ϵ∥w∥1\max_{\|\delta\|_\infty\le\epsilon}w^\top\delta=\epsilon\|w\|_1

(1) An exact result for a linear function, with the optimal perturbation following the coordinatewise signs of w; not a general formula for global network error.

LaTeX\max_{\|\delta\|_\infty\le\epsilon}w^\top\delta=\epsilon\|w\|_1
δFGSM=ϵsign⁡(∇xL)\delta_{\mathrm{FGSM}}=\epsilon\operatorname{sign}(\nabla_x\mathcal L)

(2) A single-step direction under local linearization of the loss; actual testing also requires handling constraints and the valid input range.

LaTeX\delta_{\mathrm{FGSM}}=\epsilon\operatorname{sign}(\nabla_x\mathcal L)
min⁡θE(x,y)max⁡δ∈SL(fθ(x+δ),y)\min_\theta\mathbb E_{(x,y)}\max_{\delta\in S}\mathcal L(f_\theta(x+\delta),y)

(3) The robust optimization objective; numerical inner search does not automatically obtain the global maximum.

LaTeX\min_\theta\mathbb E_{(x,y)}\max_{\delta\in S}\mathcal L(f_\theta(x+\delta),y)

Method and evidence workflow

Task and threat definition → Norm / budget / input domain → Local perturbation and search → Robust training or defense → Independent adaptive acceptance testing → Clean-data quality and scope statement
Figure 1 · Method and evidence workflowA conceptual diagram drawn from the article's structure, showing steps that need to be distinguished; it does not indicate that these experiments have been completed.

Analytical figures and numerical examples

Perturbation norms create different dimensional scaling
Figure 2 · Perturbation norms create different dimensional scalingw=ones/sqrt(d), ||w||2=1, epsilon0.05 for each distinct norm constraint. Not equal perceptual strength or attack success. JSON · Equations, inputs, and numerical values

Recomputable worked example

When d=100, the maximum linear change under an infinity-norm budget of 0.05 is 0.5, while the change under a two-norm budget of 0.05 is 0.05. The two constraints permit different sets and cannot be treated as equivalent attack capabilities.

Method comparison

Method or judgmentControlled variables and measurementsFailure conditions and interpretation
Norm-constrained threatFix the norm, ε, and valid input rangeDifferent norms do not represent the same attack capability
Empirical attacksMultiple starting points and adaptive budgetsFailure to find a counterexample does not prove its absence
Adversarial trainingIndependent attacks and regression testing on natural dataTraining threats do not cover all variations
Robustness reportingJoint reporting of quality, cost, and scopeA single success rate cannot represent universal safety

Verification and experimental plan

  1. Clearly specify the authorized testing scope, threat set, query budget, and success criteria.
  2. Freeze the model, preprocessing, and randomness, and first check analytical results on a small model.
  3. Use independent evaluation with multiple starting points and budgets, adapted to the defense structure.
  4. Check budget monotonicity and gradient and numerical anomalies to rule out artifacts caused by masking.
  5. Report clean accuracy, robustness metrics, subgroups, and computational cost together.
  6. Preserve reproducible failures and scope statements; do not extrapolate to untested threats or claim universal safety.

Scope and limitations

  • Linear sensitivity is not a network attack success rate.
  • No attacks were conducted against real services.
  • Empirical attacks cannot replace certification.
  • Conclusions must be limited to the threat set.

Primary sources and corresponding claims

  1. Explaining and Harnessing Adversarial Examples, , · 2014-12-20Abstract; 原文方法、理论和实验章节见作者原始论文链接
  2. Towards Deep Learning Models Resistant to Adversarial Attacks, , , , · 2017-06-19Abstract; 原文方法、理论和实验章节见作者原始论文链接
  3. Theoretically Principled Trade-off between Robustness and Accuracy, , , , , · 2019-01-24Abstract; 原文方法、理论和实验章节见作者原始论文链接

Citation and article export

BibTeXRISCSL JSONTXTArticle JSON

Author, version, and sources

AI-assisted research review; Codex (AI) wrote the article and performed analytical derivations based on the listed primary literature. This site has no record of independent peer review or original model experiments for this article.

Reader performance revision: a previously read page is also reused when a recent HEAD request returns 200 and the ETag is unchanged. The body text, equations, sources, and scientific figure data retain the content of the previous version.

The conceptual cover and scientific diagrams are kept separate; scientific figures are calculated from published equations and explicit assumptions, and all results from original papers are attributed.

Author, version, and correction history · /guides/robustness-adversarial/v1/

Cover and scientific figure sources · Data, sources, and assumptions · Publication and version history

Source revision: 6251a250b27bb8b449f59b392f81abd355383770