{ "schema": "auturnal.deep-research-review.v1", "slug": "conformal-uncertainty", "title": "保形预测的有限样本保证：校准秩、交换性与条件覆盖边界", "category": "可信AI与方法论", "dek": "推导分割校准的分位数，说明覆盖率、集合大小和分布变化之间的关系。", "english": "Split conformal prediction calibrates a score using ranks under exchangeability. We derive the finite-sample quantile and explain the difference between marginal and conditional coverage. Rank curves are deterministic calculations, not measured coverage on a dataset. Distribution shift, subgroup failures and set size must be assessed independently of the nominal guarantee.", "kind": "ai_assisted_research_guide", "author": "Codex（AI）", "original_language": "zh-Hans", "published_date": "2026-10-22", "edition": 5, "version": 1, "previous_version": null, "refs": [ "arxiv-2107.07511", "arxiv-1905.03222", "helm" ], "sections": [ [ "摘要", "保形预测能在交换性等条件下构造具有边际覆盖保证的预测集合，但不能无条件保证每个个体或每个子群都达到相同覆盖。本文推导分割校准的有限样本分位数，解释小样本秩的离散性，并讨论分布变化、模型选择与集合大小。图示只计算不同校准样本数下的秩阈值，不声称完成任何现实任务校准。", [ "arxiv-2107.07511", "arxiv-1905.03222", "helm" ] ], [ "1. 预测集合回答的是覆盖而非点预测好坏", "先训练固定预测器，再用独立校准集计算不符合度分数，最后选择阈值形成集合。回归可用绝对残差，分类可用标签相关分数；分数设计决定集合形状和效率。覆盖率高可能来自很大的集合，因此必须同时报告集合大小或区间宽度。保形方法校准的是程序在指定假设下的覆盖，不会把不准确模型自动变成高信息量预测器。", [ "arxiv-2107.07511" ] ], [ "2. 有限样本修正来自秩的对称性", "若校准分数与新样本分数可交换，新样本的秩具有对称性。使用ceil((n+1)(1−α))对应的校准顺序统计量，可得到至少1−α的边际覆盖保证。若所需秩超过n，保守实现需使用无穷阈值或全集，而不能偷偷取最后一个分数。并列分数的处理和随机化规则影响精确程度，应记录。图中19、99、999个校准样本展示秩粒度，不是实际覆盖测量。", [ "arxiv-2107.07511" ] ], [ "3. 边际保证不是每组或每人的保证", "总体覆盖满足目标时，某个罕见群体、输入区域或高风险样本仍可能覆盖不足。按组校准可以在各组具备相应条件和足够样本时提供更细保证，但小组会面对更粗分位数和更宽集合。完全分布无关的精细条件覆盖有根本限制。应报告预先定义子群和最差组表现，而不是把总体保证改写成每个预测都“有95%正确概率”。", [ "arxiv-2107.07511" ] ], [ "4. 分布变化会破坏校准依据", "如果上线输入、标签机制或数据收集方式改变，交换性可能失效。加权或在线方法有各自附加条件，不能沿用原始分割校准的保证措辞。应监测分数分布、集合大小和有标签样本的覆盖，登记延迟与缺失。模型更新、阈值调参和重复选择校准集也会改变程序，应重新分离训练、开发、校准与测试，避免校准信息被反复用于模型选择。", [ "arxiv-2107.07511", "helm" ] ], [ "5. 概率校准与保形覆盖不是同一目标", "温度缩放等概率校准关注预测概率与经验频率的关系；保形预测用分数秩构造集合。一个模型可能概率不佳但仍在条件满足时得到保守覆盖，另一个概率校准良好的模型也需要合适集合规则。分类难度还会随类别数量和输入变化。应分列概率质量、集合覆盖和集合大小，避免用任一单项指标代表完整可信性。", [ "arxiv-1905.03222", "arxiv-2107.07511" ] ], [ "6. 工程验收需要保留完整校准程序", "记录模型哈希、分数函数、样本划分、α、分位数实现和并列处理。冻结后在独立测试集估计覆盖及置信区间，按群体和时间分层，并模拟分布变化。对高风险用途，预测集合只是决策信息的一部分，仍需要独立的决策规则与责任流程。本篇只推导秩阈值，不能作为某个模型已经获得覆盖保证的证明。", [ "arxiv-2107.07511", "helm" ] ] ], "equations": [ { "tex": "k=\\lceil(n+1)(1-\\alpha)\\rceil,\\qquad q=s_{(k)}", "explanation": "s为校准分数排序；k=n+1时需取无穷阈值/全集等保守处理。" }, { "tex": "C(x)=[\\hat f(x)-q,\\hat f(x)+q]", "explanation": "绝对残差分数对应的回归区间；其他分数会得到不同集合。" }, { "tex": "P\\{Y_{n+1}\\in C(X_{n+1})\\}\\ge1-\\alpha", "explanation": "交换性及正确分割等条件下的边际保证；不是任意分布变化或每个子群的条件保证。" } ], "table": { "headers": [ "方法或判断", "控制变量与测量", "失效条件及解释" ], "rows": [ [ "边际覆盖", "冻结程序并检查交换性", "不等于每人或每组保证" ], [ "集合效率", "同时报告宽度或标签数", "全集也能有高覆盖但信息少" ], [ "有限样本秩", "使用n+1修正并处理越界", "小校准集限制分位数精度" ], [ "分布变化", "监测分数与真实标签覆盖", "原保证不能无条件迁移" ] ] }, "protocol": [ "分离训练、开发、校准和测试，冻结模型与分数函数。", "记录α、分位数取整、并列和越界处理，先用小样本手算核对。", "在独立测试集报告覆盖置信区间及集合大小。", "按预先定义子群、难度和时间分层，查看最差组而非仅总体。", "模拟协变量或标签机制变化，明确哪些保证不再适用。", "模型或程序更新后重新校准并保存版本，禁止反复挑选最有利校准集。" ], "diagram": [ "冻结预测器与分数", "独立校准集", "有限样本秩阈值", "构造预测集合", "覆盖及大小联合评估", "子群 / 时间 / 分布变化监测" ], "analytic": "conformal-ranks", "limitations": [ "交换性是关键条件。", "边际覆盖不能改写为个体概率。", "秩图不是数据集覆盖实测。", "分布变化与重复调参可能破坏保证。" ], "keywords": [ "conformal-prediction", "uncertainty", "calibration" ], "peer_reviewed": false, "journal_publication": false, "disclosure": { "authorship": "AI辅助研究综述；Codex（AI）依据所列一手文献撰写并进行解析推导。", "figures": "概念封面与科学图解分开；科学图由公开公式和明示假设计算，原始论文结果均注明归属。", "review": "本站没有本篇独立同行评审或原创模型实验记录。" }, "section_kinds": [ "source_based_review", "source_based_review", "source_based_review", "source_based_review", "source_based_review", "source_based_review", "source_based_review" ], "sources": [ { "id": "arxiv-2107.07511", "title": "A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification", "authors": ",", "url": "https://arxiv.org/abs/2107.07511", "date": "2021-07-15", "accessed_at": "2026-10-21", "locator": "Abstract; 原文方法、理论和实验章节见作者原始论文链接", "note": "一手论文；原文研究归属原作者。本文的推导示例与建议实验方案另行标注，不冒充原论文或本站实测。" }, { "id": "arxiv-1905.03222", "title": "Conformalized Quantile Regression", "authors": ", ,", "url": "https://arxiv.org/abs/1905.03222", "date": "2019-05-08", "accessed_at": "2026-10-21", "locator": "Abstract; 原文方法、理论和实验章节见作者原始论文链接", "note": "一手论文；原文研究归属原作者。本文的推导示例与建议实验方案另行标注，不冒充原论文或本站实测。" }, { "id": "helm", "title": "Holistic Evaluation of Language Models", "authors": "Liang et al.", "date": "2022-11-16", "url": "https://arxiv.org/abs/2211.09110v2", "locator": "v2 abstract; seven dimensions and evaluation coverage", "note": "HELM combines accuracy, calibration, robustness, fairness, bias, toxicity and efficiency where feasible; coverage and tradeoffs are explicit.", "accessed_at": "2026-10-15", "rights": "Original source remains under its own license; only attributed factual synthesis, no source figures or long excerpts." } ], "figures": { "mechanism": { "url": "/guides/assets/v5-mechanism-conformal-uncertainty-4980aaf7a50252ea.png", "kind": "conceptual_method_flow", "data": [ "冻结预测器与分数", "独立校准集", "有限样本秩阈值", "构造预测集合", "覆盖及大小联合评估", "子群 / 时间 / 分布变化监测" ] }, "analysis": { "url": "/guides/assets/v5-analysis-conformal-uncertainty-3cd64288ebacded7.png", "kind": "analytical_or_constructed_example", "data_url": "/guides/assets/v5-data-conformal-uncertainty-df5245c06f855dee.json", "assumptions": "Exchangeable scores, split calibration, conservative tie handling. Rank n+1 means infinite threshold/full set. Fractions are not measured coverage." } }, "cover_provenance_url": "/guides/assets/v5-provenance-conformal-uncertainty-b5e5cbc0b98889c5.json", "source_revision": "ed6feca8fdc75c8de482fd69dedf17cbc746a9d1" }