Dream-RSI: Recursive Self-Improvement through Evolving Worlds
Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng, Yunsong Guo · 2026
Dream-RSI usa el historial de búsquedas pasadas de un agente como simulador para mejorar, sin experimentos nuevos, la política que decide qué explorar. En Lasso con Gemini-3.1-Pro, baja el tiempo promedio en seis datasets de 3587.1 a 2931.0 ms con 317 llamadas al agente en vez de 550 (Zheng et al., 2026, Google).
Lee la explicación en español: Dream-RSI: automejora recursiva de agentes →Crea tu cuenta gratis para descargar. Por su licencia, la descarga viene del sitio original del paper.
Abstract original
Recursive self-improvement is becoming essential for autonomous AI agents, whose progress depends on discovering high-value solutions across complex domains. Effective exploration drives this process, yet managing and improving exploration strategies remains a major bottleneck. Current systems face a fundamental dilemma: fixed strategies fail to adapt as search spaces scale, while online policy optimization must navigate vast meta-search spaces under delayed, expensive feedback from long-horizon rollouts. We introduce Dream-RSI, a framework for scalable, recursively self-improving exploration. A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying base agent unchanged. Our key insight is that accumulated discovery history can act as a replay simulator over the realized search space. By dreaming within this simulator built from historical discovery trees, Dream-RSI obtains immediate, low-cost off-policy feedback to evaluate and refine exploration policies without repeated, expensive online evaluation. The improved policy is then redeployed online to drive further discovery, continuously expanding the simulator pool in a self-improving loop. Across 9 tasks in 4 domains, Dream-RSI achieves competitive quality and improves discovery efficiency in several settings.