The CLAP framework, detailed in a paper published on arXiv, addresses a limitation in current action-conditioned video models, which are typically restricted to a single robot embodiment. This restriction prevents them from fully utilizing the extensive volume of heterogeneous video data available online. CLAP aims to bridge this gap by enabling training on diverse videos featuring both human and robotic agents. The core insight behind CLAP is that universal physical laws govern spatiotemporal dynamics irrespective of the actor.
Cross-embodiment learning presents a challenge because action representations differ significantly across robot platforms and are often absent in human videos. CLAP tackles this by aligning visual dynamics from videos with proprioceptive dynamics derived from robot trajectories through a process of contrastive learning. This alignment maps unlabeled human videos into a physically grounded latent action space. Unlike methods that focus on reconstructing visual appearance, CLAP emphasizes capturing meaningful motion patterns and underlying skills.
The framework employs a dual-formulation Vision-Language-Action (VLA) approach, offering both CLAP-NTP and CLAP-RF. CLAP-NTP is an autoregressive model designed for instruction following and object generalization. CLAP-RF, on the other hand, is a Rectified Flow-based policy built for high-frequency, precise manipulation. This dual approach allows CLAP to handle both complex reasoning tasks and rapid, accurate physical interactions.
Previous research has explored various methods for cross-embodiment transfer. For example, the LAC-WM model, introduced in February 2026, also aimed to create a unified latent action space across diverse robot embodiments to improve generalization. Similarly, the DreamZero World Action Model, presented in February 2026, demonstrated cross-embodiment transfer by learning physical dynamics from heterogeneous robot data, including human videos, leading to improved performance on unseen tasks. Another approach, ContactFlow, proposed in July 2026, uses an embodiment-agnostic action representation that encodes manipulation through the trajectory of 3D contact points between an actor and an object, allowing a shared conditioning signal for human and robotic demonstrations.
The development of frameworks like CLAP, DreamZero, and ContactFlow highlights a broader trend in robotics research: moving beyond single-embodiment models to leverage the vast and diverse data available from both human and robotic interactions. This shift aims to enable robots to learn more generalizable physical priors, which are crucial for performing novel tasks in unstructured real-world environments without extensive, embodiment-specific retraining.
The ability to train on internet-scale videos of human actions is particularly valuable because human videos offer a rich source of information about physical interactions and object manipulation that is difficult to replicate with robot-specific data collection. By effectively integrating this human-centric data, CLAP allows robotic systems to acquire a more intuitive understanding of how objects behave under various forces and movements.
The researchers from Princeton University, who developed CLAP, assert that the framework achieves superior or competitive performance compared to single-embodiment models. It also facilitates data-efficient adaptation to both known and novel robot morphologies. This efficiency in adaptation is a key factor for deploying robots in diverse applications, as it reduces the need for extensive, costly data collection for each new robot design or task.
The CLAP framework also incorporates a Knowledge Matching (KM) regularization strategy to mitigate catastrophic forgetting during fine-tuning. This strategy helps preserve the semantic knowledge acquired during pretraining while adapting the model to new tasks, ensuring that the robot retains its broader understanding of physics even as it specializes in specific operations.
Looking ahead, the ability of models like CLAP to learn from varied data sources across different embodiments could accelerate the development of more versatile and adaptable robotic systems. The focus on generalizable physical priors, rather than embodiment-specific actions, represents a foundational step toward robots that can operate effectively in a wider range of complex, dynamic environments.