A new research paper published on arXiv details RoboJEPA, a world model built on the Joint Embedding Predictive Architecture (JEPA) and trained on a large dataset from 12 different robotic systems. The work establishes scaling laws for multi-embodiment robotic world models trained using real robot data. RoboJEPA, with 8 billion parameters, is currently the largest JEPA predictor model developed.
The core finding is that RoboJEPA's imagination error, which quantifies the accuracy of its latent rollouts, adheres to a second-order power law in relation to compute. This mathematical relationship enables researchers to forecast the model's quality even at scales significantly beyond the initial training parameters. Furthermore, the study indicates that robotic planning performance improves predictably with increased compute, and this improvement correlates strongly with the reduction in imagination error. This correlation positions imagination error as a reliable proxy for evaluating real-robot performance.
Joint Embedding Predictive Architectures (JEPAs) represent a self-supervised learning approach that predicts abstract representations of inputs rather than reconstructing raw data like pixels or tokens. This method allows the model to focus on essential structural information while disregarding unpredictable details. Yann LeCun proposed JEPA in 2022 as a foundational step toward developing AI systems that construct internal models of the world, akin to human and animal learning processes. Unlike generative models that predict every detail, JEPAs operate in a latent embedding space, predicting the embedding of a target from a context. This approach aims to enhance computational efficiency and the quality of learned representations.
Previous JEPA models include I-JEPA for images and V-JEPA and V-JEPA 2 for video, with V-JEPA 2 demonstrating zero-shot robot control in new environments. These models learn by predicting representations of masked spatiotemporal regions in videos, enabling them to focus on motion, objects, and relationships relevant to scene changes. RoboJEPA extends these capabilities to a multi-embodiment robotic context, demonstrating that latent world models can function as zero-shot robotic agents. These agents can plan towards a single goal image to complete tasks requiring long-horizon planning on physical hardware.
The researchers have made all model checkpoints, along with their training and robot deployment code, publicly available. This release aims to facilitate further research and development in scaling robotic world models. The establishment of these scaling laws provides a framework for understanding and predicting the performance of increasingly complex robotic AI systems.