A new framework called "Learning While Deploying" (LWD) allows generalist robot policies to achieve multi-task improvement in real-world settings, according to a paper published on arXiv. The LWD system facilitates continuous post-training of Vision-Language-Action (VLA) policies, which are initially developed using large-scale pretraining. This method addresses the challenge that offline training alone cannot fully prepare robots for the varied conditions, unexpected failures, and human interactions encountered in deployment.LWD operates by creating a feedback loop where experience gathered from deployed robots, including both successful operations and failures, along with optional human corrections, is used to refine the shared policy. This collected data then feeds into an online replay system and is combined with existing offline data for further training. The improved policy is periodically redistributed to the robot fleet. This contrasts with earlier methods that often focused on specialist policies for individual tasks or had inconsistent training objectives between offline and online stages.To maintain stable learning from the diverse and often sparse-reward data collected from a fleet, LWD integrates two key algorithmic components. It employs Distributional Implicit Value Learning (DIVL) for robust value estimation and Q-learning via Adjoint Matching (QAM) for extracting policies from flow-based VLA action generators. This unified training approach across both offline and online phases enhances stability and scalability.The researchers validated the LWD framework on a fleet of 16 dual-arm robots. These robots performed eight distinct real-world manipulation tasks, including grocery restocking and longer tasks lasting three to five minutes. The tasks often involved sparse rewards, meaning success was only credited at the end of an episode rather than for individual steps. Through the accumulation of fleet experience, the single generalist policy improved, achieving an average success rate of 95%. The most significant performance gains were observed in the long-horizon tasks.The LWD framework represents an advance in enabling robots to learn and adapt continuously in dynamic environments. By making deployment an integral part of the training process, it allows generalist policies to overcome limitations of fixed demonstration datasets and adapt to real-world complexities.