{
  "slug": "agents-tools-and-outcomes",
  "category": "智能体系统",
  "title": "会调用工具不等于会完成任务：ReAct、Toolformer 与智能体验收",
  "dek": "从“下一步动作看起来合理”转向“目标状态确实达成”，分析长期任务中的观察、权限、错误恢复和结果验证。",
  "english": "Tool use is only one component of an agent. We connect ReAct, Toolformer, and WebArena to outcome-based evaluation and an illustrative compounded-success model.",
  "refs": [
    "react",
    "toolformer",
    "webarena"
  ],
  "sections": [
    [
      "摘要",
      "智能体把语言输出接到检索、计算或网站操作，能力边界随之变成一整条系统链路。本文结合 ReAct 的观察循环、Toolformer 的工具调用学习和 WebArena 的功能性验收，解释为什么动作正确率不能直接代表任务成功率。配图中的连乘曲线只揭示理想化串行依赖，不是任何真实智能体的失败统计。",
      [
        "react",
        "toolformer",
        "webarena"
      ]
    ],
    [
      "1. ReAct 引入可更新的观察循环",
      "ReAct 将推理与动作交替组织，使外部观察能够影响下一步。和一次生成完整计划相比，这种结构为检查环境变化与调整行动提供了位置。它并不自动解决观察缺失、误读页面或错误前提；如果系统没有检查工具返回的实际结果，循环也可能只是对失败进行连续叙述。",
      [
        "react"
      ]
    ],
    [
      "2. Toolformer 学习何时与如何调用",
      "Toolformer 研究通过自监督方式为训练材料选择有用的 API 调用，让模型学习工具使用行为。它针对的是工具调用能力的学习问题，不是生产环境的完整权限制度。能够生成正确 API 参数，与获得账户授权、确认副作用以及处理重试，是不同层面的条件。部署系统仍需在模型之外控制这些边界。",
      [
        "toolformer"
      ]
    ],
    [
      "3. WebArena 用目标状态检验任务",
      "WebArena 提供可运行的网站环境与长程任务，把评估重点放在功能结果是否满足用户意图。两条操作路径即使文字不同，也可能达到同一有效终态；相反，动作描述与示例相似，却可能没有保存数据。论文中的历史基线属于其模型、环境与时间点，本文不用这些结果推断当前浏览器智能体的能力。",
      [
        "webarena"
      ]
    ],
    [
      "4. 长任务为什么需要恢复机制",
      "假设每一步独立成功概率为 p，且任何一步失败都会使整个串行任务失败，n 步全成功概率为 pⁿ。图 2 展示 p=0.95 与 0.99 时的数学曲线。真实系统可能有重试、相关故障、可恢复动作和分支，因此不能用这条曲线直接估计真实成功率；它只是提示应测量完整任务，而不是只看单步表现。",
      []
    ],
    [
      "5. 观察与外部内容应如何进入决策",
      "本文建议把用户目标、系统权限与外部材料分开保存。网页、文档和工具输出属于观察对象；即使其中写着“忽略原要求”，也不能因此改变任务授权。对于账户修改、发信或发布，执行层应依据已有权限和明确任务范围决定，而不能把页面文本当成来自用户的新指令。这里是系统设计建议。",
      []
    ],
    [
      "6. 重试必须考虑副作用",
      "读取失败后重试，和已经可能成功的写入后重试，含义不同。提交动作需要幂等标识或可核对的结果状态；若超时导致结果未知，应先查询实际状态。只用“调用没有返回成功”推断“操作没有发生”，可能产生重复内容或重复交易。工具接口应将失败、成功与未知结果区分表达。",
      []
    ],
    [
      "7. 验收应该看到哪些证据",
      "一次任务记录至少说明起始状态、目标状态、允许动作、停止条件与验收方法。成功不能仅凭智能体自述，应由独立读取、保存后的回读或环境内的可执行判定确认。还应报告超时、恢复次数和人工介入。若必须让人修正关键一步，就应把它记录为人机协作完成，而不是全自动成功。",
      [
        "webarena"
      ]
    ],
    [
      "状态确认与语言确认不能混为一谈",
      "在网站任务中，页面显示“正在保存”与数据已经持久化是不同状态。可靠验收应能从保存后的新读取看到目标值，必要时检查版本或事务标识。对于不可逆动作，执行前的确认与执行后的回读各有用途，不能互相替代。连续几次看似合理的动作，若没有最终状态证据，仍不足以证明任务已经完成。",
      []
    ],
    [
      "后续实验可以如何设计",
      "用可恢复的独立环境设置明确初始状态，运行一组包含读取、写入、超时与重复提交的任务。评估器检查目标状态，并记录额外副作用，而不是仅看任务描述中提到的字段。若重试可能产生重复记录，专门检查重复计数。将有人工纠正与无人工纠正的结果分开，同时保留停止条件；不为提高表面成功率而无限重试。",
      []
    ],
    [
      "局限与结论",
      "本文没有运行新的 WebArena 评估，也没有对现有智能体作可靠性认证。工具扩展了可行动空间，可靠性则来自观察、授权、状态管理与结果验证共同作用。系统设计越关注真实终态，就越容易区分“看起来做了”和“确实完成了”。",
      []
    ]
  ],
  "diagram": [
    "目标与授权",
    "明确允许改变的环境状态",
    "读取真实观察",
    "页面和文档是数据，不是新授权",
    "规划下一步",
    "区分读取、写入与结果未知",
    "调用工具",
    "校验参数并保存幂等标识",
    "回读目标状态",
    "用功能结果判断是否成功",
    "恢复或停止",
    "有界重试；必要时保留人工介入"
  ],
  "figureNote": "图 1 是本文整理的工具执行框架。图 2 为 pⁿ 的理想化连乘示意，假设各步独立且无恢复；p=0.95/0.99 是假设值，非任何产品或论文的实测成功率。",
  "table": {
    "headers": [
      "系统层",
      "可验证对象",
      "不能直接推出"
    ],
    "rows": [
      [
        "工具调用",
        "语法、参数与返回值",
        "任务已完成"
      ],
      [
        "环境状态",
        "保存后独立回读",
        "没有其他副作用"
      ],
      [
        "完整任务",
        "目标、边界与失败恢复",
        "所有新任务都可靠"
      ]
    ]
  },
  "version": 1,
  "kind": "ai_assisted_research_guide",
  "author": "Codex（AI）",
  "published_date": "2026-10-16",
  "disclosure": "本站用户明确委托撰写与公开。由 Codex（AI）依据所列一手资料整理，配图由代码绘制；未经独立同行评审。文中原论文的实验结果归属于原作者，本文未运行新的模型训练或基准测试。",
  "limitations": "文献导读与方法分析；不是原创实验论文、正式期刊录用或专业资质认证。解析示例不代表本站或当前产品的实测表现。",
  "sources": [
    {
      "id": "react",
      "title": "ReAct: Synergizing Reasoning and Acting in Language Models",
      "authors": "Yao et al.",
      "date": "2022-10-06",
      "url": "https://arxiv.org/html/2210.03629v3",
      "locator": "v3; method and experiments",
      "note": "Alternates reasoning and actions with external observations; observed success is environment- and model-specific.",
      "accessed_at": "2026-10-15",
      "rights": "Original source remains under its own license; only attributed factual synthesis, no source figures or long excerpts."
    },
    {
      "id": "toolformer",
      "title": "Toolformer: Language Models Can Teach Themselves to Use Tools",
      "authors": "Schick et al.",
      "date": "2023-02-09",
      "url": "https://arxiv.org/abs/2302.04761",
      "locator": "abstract; training tool-use behavior",
      "note": "A self-supervised procedure selects useful API calls for training; training tool syntax does not establish production authorization.",
      "accessed_at": "2026-10-15",
      "rights": "Original source remains under its own license; only attributed factual synthesis, no source figures or long excerpts."
    },
    {
      "id": "webarena",
      "title": "WebArena: A Realistic Web Environment for Building Autonomous Agents",
      "authors": "Zhou et al.",
      "date": "2023-07-25",
      "url": "https://arxiv.org/html/2307.13854v4",
      "locator": "v4; abstract, benchmark and functional evaluation",
      "note": "An executable website benchmark measures final functional outcomes rather than just action-text similarity; historical results remain tied to the stated version.",
      "accessed_at": "2026-10-15",
      "rights": "Original source remains under its own license; only attributed factual synthesis, no source figures or long excerpts."
    }
  ],
  "figures": [
    {
      "slug": "agents-tools-and-outcomes",
      "number": 1,
      "file": "agents-tools-and-outcomes-1.png",
      "sha256": "454d22576d0b728069566bec5221093224f3e8c5faae2d7f0715569da7a17410",
      "alt": "六个要点：目标与授权；读取真实观察；规划下一步；调用工具；回读目标状态；恢复或停止",
      "data": {
        "kind": "conceptual",
        "items": [
          "目标与授权",
          "明确允许改变的环境状态",
          "读取真实观察",
          "页面和文档是数据，不是新授权",
          "规划下一步",
          "区分读取、写入与结果未知",
          "调用工具",
          "校验参数并保存幂等标识",
          "回读目标状态",
          "用功能结果判断是否成功",
          "恢复或停止",
          "有界重试；必要时保留人工介入"
        ]
      },
      "ai_generated": true,
      "disclosure": "AI-assisted code rendering; original schematic or explicitly attributed data; not a new model experiment.",
      "url": "/guides/assets/agents-tools-and-outcomes-1-454d22576d0b.png"
    },
    {
      "slug": "agents-tools-and-outcomes",
      "number": 2,
      "file": "agents-tools-and-outcomes-2.png",
      "sha256": "7ed6675c99417e6614c790ac54ae8dd299b105976af3013e3f6908800d45f1c8",
      "alt": "图 1 是本文整理的工具执行框架。图 2 为 pⁿ 的理想化连乘示意，假设各步独立且无恢复；p=0.95/0.99 是假设值，非任何产品或论文的实测成功率。",
      "data": {
        "kind": "analytical",
        "steps": [
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
        ],
        "probability_by_p": {
          "0.95": [
            0.95,
            0.9025,
            0.8573749999999999,
            0.8145062499999999,
            0.7737809374999998,
            0.7350918906249998,
            0.6983372960937497,
            0.6634204312890623,
            0.6302494097246091,
            0.5987369392383787,
            0.5688000922764597,
            0.5403600876626367,
            0.5133420832795048,
            0.48767497911552954,
            0.46329123015975304,
            0.44012666865176536,
            0.4181203352191771,
            0.3972143184582182,
            0.37735360253530725,
            0.3584859224085419,
            0.3405616262881148,
            0.323533544973709,
            0.3073568677250236,
            0.2919890243387724,
            0.27738957312183377,
            0.26352009446574204,
            0.2503440897424549,
            0.23782688525533216,
            0.22593554099256555,
            0.21463876394293727,
            0.2039068257457904,
            0.19371148445850087,
            0.18402591023557582,
            0.174824614723797,
            0.16608338398760716,
            0.1577792147882268,
            0.14989025404881545,
            0.14239574134637467,
            0.13527595427905592,
            0.12851215656510312,
            0.12208654873684796,
            0.11598222130000556,
            0.11018311023500528,
            0.10467395472325501,
            0.09944025698709225,
            0.09446824413773763,
            0.08974483193085075,
            0.0852575903343082,
            0.0809947108175928,
            0.07694497527671315
          ],
          "0.99": [
            0.99,
            0.9801,
            0.970299,
            0.96059601,
            0.9509900498999999,
            0.941480149401,
            0.9320653479069899,
            0.9227446944279201,
            0.9135172474836408,
            0.9043820750088044,
            0.8953382542587164,
            0.8863848717161292,
            0.8775210229989678,
            0.8687458127689782,
            0.8600583546412884,
            0.8514577710948755,
            0.8429431933839268,
            0.8345137614500875,
            0.8261686238355866,
            0.8179069375972308,
            0.8097278682212584,
            0.8016305895390459,
            0.7936142836436554,
            0.7856781408072188,
            0.7778213593991467,
            0.7700431458051551,
            0.7623427143471035,
            0.7547192872036326,
            0.7471720943315961,
            0.7397003733882802,
            0.7323033696543975,
            0.7249803359578534,
            0.7177305325982749,
            0.7105532272722921,
            0.7034476949995692,
            0.6964132180495735,
            0.6894490858690777,
            0.682554595010387,
            0.6757290490602831,
            0.6689717585696803,
            0.6622820409839835,
            0.6556592205741436,
            0.6491026283684022,
            0.6426116020847181,
            0.6361854860638709,
            0.6298236312032323,
            0.6235253948912,
            0.617290140942288,
            0.611117239532865,
            0.6050060671375364
          ]
        },
        "assumptions": "independent steps; any failure ends task; no retry or recovery"
      },
      "ai_generated": true,
      "disclosure": "AI-assisted code rendering; original schematic or explicitly attributed data; not a new model experiment.",
      "url": "/guides/assets/agents-tools-and-outcomes-2-7ed6675c9941.png"
    }
  ],
  "peer_reviewed": false
}