Researchers have developed a model-system framework named Long-WAM that enhances real-time robot control by enabling the use of extended visual histories. The framework's core finding is that simply providing more visual data is not enough; the way this data is processed and learned from is critical. Long-WAM demonstrates that longer visual contexts yield substantial performance gains when the underlying video foundation model is pretrained using an autoregressive (AR) approach.
The researchers first train the model to predict causal relationships from robot and egocentric videos without requiring action labels. This history-to-future structure is then preserved during the world-action adaptation phase. On the RoboCasa GR-1 benchmark, increasing the visual context from zero to 19.2 seconds improved task success rates from 63.3% to 78.7%. In contrast, a model initialized with bidirectional pretraining showed no significant improvement with extended context. Further gains were observed with robot-domain AR pretraining on both the GR-1 and LIBERO-Long benchmarks.
Long-WAM also achieved top results on other benchmarks, including LIBERO-Long, RoboTwin 2.0, and DOMINO, outperforming compared methods. The system employs streaming observation encoding, asynchronous execution, and hardware-specific acceleration to maintain real-time performance. On an RTX 5090 GPU, each action chunk, including future video latent prediction, is processed in 107.4 milliseconds. This enables real-time deployment on platforms like the Unitree G1 and YAM robots, facilitating dynamic and long-horizon manipulation tasks. For example, Long-WAM achieved a 95% success rate on a dynamic cup stacking task, where other methods like Pi0.5 and Fast-WAM failed in all 20 trials. As a memory-informed executor, Long-WAM can also complement higher-level planning systems for complex tasks.
This work builds upon research in autoregressive models for robotics, such as ARM4R, which also leverages autoregressive pretraining on human video data to improve robotic control by learning low-level 4D representations. While ARM4R focuses on lifting 2D representations to 3D for better geometric understanding, Long-WAM specifically addresses scaling the context window for world-action models within real-time constraints. The development of Long-WAM is part of a broader infrastructure for long-context world-action models, including robot-video pretraining and deployment demos.