【HK】RL 与自蒸馏梯度在打架：港中文 Irwin King 等 CORD，ALFWorld 65.0→80.9%
【HK】RL 与自蒸馏梯度在打架：港中文 Irwin King 等 CORD，ALFWorld 65.0→80.9%
View PDF HTML (experimental) Abstract: Joint reinforcement learning and self-distillation combines task rewards with dense teacher guidance for agent training, but their gradients can partly cancel. A mixing coefficient controls relative scale without selectively removing...
CORD 框架助力 ALFWorld 成功率提升 (arXiv:2610.29965)
“View PDF HTML (experimental) Abstract: Joint reinforcement learning and self-distillation combines task rewards with dense teacher guidance”
Abstract:Joint reinforcement learning and self-distillation combines task rewards with dense teacher guidance for agent training, but their gradients can partly cancel. A mixing coefficient controls relative scale without selectively removing opposing components. We present CORD, a general framework for conflict-aware optimization of two objectives. CORD separates the choice of protected objective from the condition that triggers projection, and combines the gradients once per optimizer mini-batch. A nearest-point formulation yields a closed-form update and characterizes its first-order effect on the protected objective before optimizer transformations. Gradient diagnostics connect conflict in agent training to lost local optimization progress and motivate selective correction. As a gradient-combination mechanism, CORD can be added to different joint training recipes while retaining their objectives. Across three such recipes, it improves ALFWorld success by 1.5-15.9 percentage points. On WebShop, it improves two of the three pairs, with SDAR rising from 60.7% to 64.6% success. Our analyses explain why protection direction and projection gate must be chosen together, and why the distillation objective and its mixing weight remain consequential after projection.
| Comments: | 38 pages, 12 figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.29965 [cs.LG] |
| (or arXiv:2610.29965v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29965 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenhao Yu [view email]
[v1]
Wed, 21 Oct 2026 16:19:22 UTC (1,431 KB)
Source: Agent Control Flash News · arxiv.org
Related Products & Analysis
Cursor Agent 架构全景拆解：从影子工作区到多文件推演引擎
全面剖析 Cursor 在 Agent 模式下的底层技术实现，解析影子分支隔离、LSP 上下文剪枝以及预测性补全的核心架构细节与工程实践经验。
Vercel v0 推出全新多 Agent 协作画布与实时全栈部署预览
动态速评：v0 正在从单一组件生成器演进为全栈应用搭建协作台，尤其是与 Neon/Postgres 的直连极大降低了全栈原型验证成本。