Rolling-WAM

World Action Models
with Rolling Imagination

Yinghua Zhou1,2,*Junjie Ye1,*Yiqi Zhao1Hao Dong1Celina Shiyu Wang1Ruohai Ge1Tingyi Yang1,3Basile Van Hoorick4Gaurav Sukhatme1Vitor Guizilini4,†Yue Wang1,†
1University of Southern California 2Brown University 3Fudan University 4Toyota Research Institute

* Equal contribution† Equal advising

arXiv Coming soon Code

Abstract

World Action Models (WAMs) couple action generation with future visual prediction for robotic manipulation. However, completing the joint video-action denoising process at each replanning cycle incurs substantial latency, delaying action updates and limiting closed-loop responsiveness. We present Rolling-WAM, a formulation that distributes joint denoising across successive replanning cycles. Our method maintains a sliding window of video-action chunks at staggered noise levels. At each step, a rolling noise schedule fully denoises the imminent action chunk for execution, while partially refining farther-future chunks. As the window advances with new camera observations, the retained future chunks continue their denoising process. This distributes the computational cost over time while carrying an evolving visual-action context across chunk boundaries. Evaluations on LIBERO, RoboTwin, and a real-world Unitree G1 humanoid show that Rolling-WAM achieves competitive manipulation performance. By removing the need to denoise the entire prediction horizon from scratch, it delivers a 4.5× steady-state replanning speedup over standard joint WAMs.

4.5×

faster replanningvs. Joint-WAM · steady state

98.1%

LIBEROaverage success

93.3%

RoboTwin 2.0average success

85.0%

real-world G1average success

Method Overview

Rolling-WAM jointly refines video and action predictions in a sliding window. The next action becomes ready to execute while future chunks remain partially denoised.

Prediction windowClean Noisy
01 / REFINE WINDOW PREDICTIONS

Jointly denoise video and action predictions across the window.

01

Refine Window Predictions

Jointly denoise predictions across the window until the next action chunk is ready.

02

Execute and Observe

Execute the action chunk and acquire a new observation.

03

Roll the Window

Retain the future predictions and append a new noisy chunk.

Rolling-WAM architecture and rolling inference: video and action experts share visual predictions, while the denoising window advances after each action chunk.

Our model uses a Mixture-of-Transformers (MoT) architecture, pairing a pretrained video Diffusion Transformer with a lightweight action Transformer. The video expert processes the current camera observation and noisy future video, while the action expert processes noisy robot actions. Language instructions and robot state condition both experts.

Masked joint attention lets future video tokens attend to the current observation and all predicted video chunks, but not to actions. Action tokens attend to all visual tokens, while action-to-action attention stays within each chunk. Per-chunk noise conditioning lets both experts refine the rolling window at different denoising stages, completing the next action chunk while future predictions remain partially noisy.

Real-World Humanoid Experiments

Qualitative Rollouts

Original playback speed
Rolling-WAM Ours
Joint-WAM
Fast-WAM

Experimental Results

Success rate (%) · higher is better

Replanning Latency

215ms

steady-state replanning

4.5× faster than Joint-WAM

REPLANNING LATENCY Lower is better ↓
Joint-WAM
978
Fast-WAM
548
π₀.₅
296
GR00T N1.7
285
Rolling-WAM
215 ms

Single NVIDIA A100 · Controlled RoboTwin 2.0 setup
visual encoding + denoising; warm-up and initialization excluded.No torch.compile, TensorRT, or custom CUDA kernels

BibTeX

@unpublished{zhou2026rollingwam,
  title = {{Rolling-WAM: World Action Models with Rolling Imagination}},
  author = {Zhou, Yinghua and Ye, Junjie and Zhao, Yiqi and Dong, Hao
            and Wang, Celina Shiyu and Ge, Ruohai and Yang, Tingyi
            and Van Hoorick, Basile and Sukhatme, Gaurav
            and Guizilini, Vitor and Wang, Yue},
  year = {2026},
  note = {Manuscript; arXiv identifier forthcoming}
}