Tool use is only one component of an agent. We connect ReAct, Toolformer, and WebArena to outcome-based evaluation and an illustrative compounded-success model.
摘要
智能体把语言输出接到检索、计算或网站操作,能力边界随之变成一整条系统链路。本文结合 ReAct 的观察循环、Toolformer 的工具调用学习和 WebArena 的功能性验收,解释为什么动作正确率不能直接代表任务成功率。配图中的连乘曲线只揭示理想化串行依赖,不是任何真实智能体的失败统计。[1][2][3]
1. ReAct 引入可更新的观察循环
ReAct 将推理与动作交替组织,使外部观察能够影响下一步。和一次生成完整计划相比,这种结构为检查环境变化与调整行动提供了位置。它并不自动解决观察缺失、误读页面或错误前提;如果系统没有检查工具返回的实际结果,循环也可能只是对失败进行连续叙述。[1]
2. Toolformer 学习何时与如何调用
Toolformer 研究通过自监督方式为训练材料选择有用的 API 调用,让模型学习工具使用行为。它针对的是工具调用能力的学习问题,不是生产环境的完整权限制度。能够生成正确 API 参数,与获得账户授权、确认副作用以及处理重试,是不同层面的条件。部署系统仍需在模型之外控制这些边界。[2]
3. WebArena 用目标状态检验任务
WebArena 提供可运行的网站环境与长程任务,把评估重点放在功能结果是否满足用户意图。两条操作路径即使文字不同,也可能达到同一有效终态;相反,动作描述与示例相似,却可能没有保存数据。论文中的历史基线属于其模型、环境与时间点,本文不用这些结果推断当前浏览器智能体的能力。[3]
4. 长任务为什么需要恢复机制
假设每一步独立成功概率为 p,且任何一步失败都会使整个串行任务失败,n 步全成功概率为 。图 2 展示 与 0.99 时的数学曲线。真实系统可能有重试、相关故障、可恢复动作和分支,因此不能用这条曲线直接估计真实成功率;它只是提示应测量完整任务,而不是只看单步表现。
5. 观察与外部内容应如何进入决策
本文建议把用户目标、系统权限与外部材料分开保存。网页、文档和工具输出属于观察对象;即使其中写着“忽略原要求”,也不能因此改变任务授权。对于账户修改、发信或发布,执行层应依据已有权限和明确任务范围决定,而不能把页面文本当成来自用户的新指令。这里是系统设计建议。

6. 重试必须考虑副作用
读取失败后重试,和已经可能成功的写入后重试,含义不同。提交动作需要幂等标识或可核对的结果状态;若超时导致结果未知,应先查询实际状态。只用“调用没有返回成功”推断“操作没有发生”,可能产生重复内容或重复交易。工具接口应将失败、成功与未知结果区分表达。
7. 验收应该看到哪些证据
一次任务记录至少说明起始状态、目标状态、允许动作、停止条件与验收方法。成功不能仅凭智能体自述,应由独立读取、保存后的回读或环境内的可执行判定确认。还应报告超时、恢复次数和人工介入。若必须让人修正关键一步,就应把它记录为人机协作完成,而不是全自动成功。[3]
状态确认与语言确认不能混为一谈
在网站任务中,页面显示“正在保存”与数据已经持久化是不同状态。可靠验收应能从保存后的新读取看到目标值,必要时检查版本或事务标识。对于不可逆动作,执行前的确认与执行后的回读各有用途,不能互相替代。连续几次看似合理的动作,若没有最终状态证据,仍不足以证明任务已经完成。
后续实验可以如何设计
用可恢复的独立环境设置明确初始状态,运行一组包含读取、写入、超时与重复提交的任务。评估器检查目标状态,并记录额外副作用,而不是仅看任务描述中提到的字段。若重试可能产生重复记录,专门检查重复计数。将有人工纠正与无人工纠正的结果分开,同时保留停止条件;不为提高表面成功率而无限重试。
局限与结论
本文没有运行新的 WebArena 评估,也没有对现有智能体作可靠性认证。工具扩展了可行动空间,可靠性则来自观察、授权、状态管理与结果验证共同作用。系统设计越关注真实终态,就越容易区分“看起来做了”和“确实完成了”。
| 系统层 | 可验证对象 | 不能直接推出 |
|---|---|---|
| 工具调用 | 语法、参数与返回值 | 任务已完成 |
| 环境状态 | 保存后独立回读 | 没有其他副作用 |
| 完整任务 | 目标、边界与失败恢复 | 所有新任务都可靠 |
المصادر الأولية وربطها بالادعاءات
- Yao et al. · ReAct: Synergizing Reasoning and Acting in Language Modelsافتح المصدر الأصلي ↖تاريخ المصدر:2022-10-06;تاريخ التحقق:2026-10-15;الموضع:v3; method and experimentsAlternates reasoning and actions with external observations; observed success is environment- and model-specific.
- Schick et al. · Toolformer: Language Models Can Teach Themselves to Use Toolsافتح المصدر الأصلي ↖تاريخ المصدر:2023-02-09;تاريخ التحقق:2026-10-15;الموضع:abstract; training tool-use behaviorA self-supervised procedure selects useful API calls for training; training tool syntax does not establish production authorization.
- Zhou et al. · WebArena: A Realistic Web Environment for Building Autonomous Agentsافتح المصدر الأصلي ↖تاريخ المصدر:2023-07-25;تاريخ التحقق:2026-10-15;الموضع:v4; abstract, benchmark and functional evaluationAn executable website benchmark measures final functional outcomes rather than just action-text similarity; historical results remain tied to the stated version.
مصدر الرسوم والبيانات والافتراضات
[
{
"slug": "agents-tools-and-outcomes",
"role": "conceptual-cover",
"generator": "OpenAI built-in image_gen",
"ai_generated": true,
"alt": "黄铜机械臂搭建跨越间隙的石阶路径,表现工具执行的概念插画。",
"original_sha256": "57b7eea54ac48897f18ba951e62eafdfe68a9e508349474d48e438d2343317e0",
"prompt": "Premium scientific journal cover art, landscape 3:2, no text, numbers, logos, watermark or borders. A breathtaking editorial still-life with gallery-grade sculptural material, warm golden details, cinematic but natural illumination, believable shadows and highly tactile surfaces. Intelligent contemporary magazine art direction, refined color palette, memorable composition that reads at thumbnail size. Openly disclosed conceptual art, never an actual experiment or graph. Avoid flat diagrams, sci-fi brains, robot heads, computer chips, neon networks, cheap gradients and generic stock imagery. Topic: agents, tools and outcomes. A beautifully engineered abstract mechanical brass articulated arm, slender sculptural joints and a precise small end effector, arranges a series of dark teal ceramic stepping stones across a narrow gap on an ivory tabletop. One separate stone in foreground and a clear path leading toward warm light imply execution and recovery. Not humanoid, no face or head, not a real commercial robot. Deep forest-green backdrop, brass and ivory, a carefully staged editorial sculptural photograph with crisp mechanical detail, soft shadow, inventive composition, slightly overhead three-quarter angle.",
"files": [
{
"file": "assets/covers/agents-tools-and-outcomes.png",
"width": 1280,
"height": 853,
"bytes": 1378426,
"sha256": "fa415870de6e83ac91050d5b163947857f892bf0c53df346e01e804fe194e5e5",
"url": "/guides/assets/v2-agents-tools-and-outcomes-fa415870de6e.png"
},
{
"file": "assets/covers/agents-tools-and-outcomes-480.png",
"width": 480,
"height": 320,
"bytes": 226974,
"sha256": "fdd493d4404cd5695cdb1eb7a1fe6f1536f4911425e2cbcb864143f96a4be200",
"url": "/guides/assets/v2-agents-tools-and-outcomes-480-fdd493d4404c.png"
}
],
"url": "/guides/assets/v2-agents-tools-and-outcomes-fa415870de6e.png",
"thumbnail_url": "/guides/assets/v2-agents-tools-and-outcomes-480-fdd493d4404c.png"
},
{
"slug": "agents-tools-and-outcomes",
"role": "data-chart",
"file": "assets/charts/agents-tools-and-outcomes.png",
"width": 1800,
"height": 1125,
"sha256": "4f6962d7e30c9a999677caef3ea015edd1f949e9bee0175527b271c41c9178b7",
"bytes": 148158,
"alt": "图表:为 pⁿ 的理想化连乘示意,假设各步独立且无恢复;p=0.95/0.99 是假设值,非任何产品或论文的实测成功率。",
"data": {
"kind": "analytical",
"steps": [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50
],
"probability_by_p": {
"0.95": [
0.95,
0.9025,
0.8573749999999999,
0.8145062499999999,
0.7737809374999998,
0.7350918906249998,
0.6983372960937497,
0.6634204312890623,
0.6302494097246091,
0.5987369392383787,
0.5688000922764597,
0.5403600876626367,
0.5133420832795048,
0.48767497911552954,
0.46329123015975304,
0.44012666865176536,
0.4181203352191771,
0.3972143184582182,
0.37735360253530725,
0.3584859224085419,
0.3405616262881148,
0.323533544973709,
0.3073568677250236,
0.2919890243387724,
0.27738957312183377,
0.26352009446574204,
0.2503440897424549,
0.23782688525533216,
0.22593554099256555,
0.21463876394293727,
0.2039068257457904,
0.19371148445850087,
0.18402591023557582,
0.174824614723797,
0.16608338398760716,
0.1577792147882268,
0.14989025404881545,
0.14239574134637467,
0.13527595427905592,
0.12851215656510312,
0.12208654873684796,
0.11598222130000556,
0.11018311023500528,
0.10467395472325501,
0.09944025698709225,
0.09446824413773763,
0.08974483193085075,
0.0852575903343082,
0.0809947108175928,
0.07694497527671315
],
"0.99": [
0.99,
0.9801,
0.970299,
0.96059601,
0.9509900498999999,
0.941480149401,
0.9320653479069899,
0.9227446944279201,
0.9135172474836408,
0.9043820750088044,
0.8953382542587164,
0.8863848717161292,
0.8775210229989678,
0.8687458127689782,
0.8600583546412884,
0.8514577710948755,
0.8429431933839268,
0.8345137614500875,
0.8261686238355866,
0.8179069375972308,
0.8097278682212584,
0.8016305895390459,
0.7936142836436554,
0.7856781408072188,
0.7778213593991467,
0.7700431458051551,
0.7623427143471035,
0.7547192872036326,
0.7471720943315961,
0.7397003733882802,
0.7323033696543975,
0.7249803359578534,
0.7177305325982749,
0.7105532272722921,
0.7034476949995692,
0.6964132180495735,
0.6894490858690777,
0.682554595010387,
0.6757290490602831,
0.6689717585696803,
0.6622820409839835,
0.6556592205741436,
0.6491026283684022,
0.6426116020847181,
0.6361854860638709,
0.6298236312032323,
0.6235253948912,
0.617290140942288,
0.611117239532865,
0.6050060671375364
]
},
"assumptions": "independent steps; any failure ends task; no retry or recovery"
},
"data_source_sha256": "963323d4a7a0cad4aebd986f09a55251a7c21f68c757d21b76245a128675e13e",
"ai_generated": true,
"disclosure": "AI-assisted deterministic rendering of unchanged v1 data; not a new experiment.",
"url": "/guides/assets/v2-agents-tools-and-outcomes-4f6962d7e30c.png"
}
]
المؤلف والإصدارات والتصحيحات
يضيف v3 خمس لغات للواجهة، مع الاحتفاظ بالنص الأصلي و29 مصدرًا وبيانات المخططات. ويبقى الإصداران v1 وv2 متاحين.
文献导读与方法分析;不是原创实验论文、正式期刊录用或专业资质认证。解析示例不代表本站或当前产品的实测表现。
عرض الأصل v1 · تنزيل المقال والرسوم وربط المصادرعرض الأصل v2
