A new research paper details Chain-of-Experience (CoE), a method designed to enable large language models (LLMs) to learn and improve continuously during inference. Unlike traditional evaluations that assess LLMs in a static, zero-shot manner, CoE allows models to accumulate "experiential traces" through repeated interactions, using both self-generated and external feedback. This iterative learning process has demonstrated consistent performance improvements over baselines that do not incorporate feedback.
The CoE framework was evaluated across eight LLMs, including GPT-5, Gemini-2.5 Pro, and Claude-4.5 Sonnet, across domains such as mathematics, coding, and general knowledge. The study found that leveraging iterative experience led to a 5.6% overall performance improvement. Furthermore, the method resulted in a 19% reduction in API costs across the tested tasks and models. These gains were observed even with self-feedback alone. The researchers also noted that combining different feedback channels, such as model-generated feedback and correctness signals, yielded additional improvements.
The paper, titled "Chain-of-Experience for Continual LLM Improvement," was submitted to arXiv by Haoqin Tu, Yunhao Fang, Yizhong Wang, Cihang Xie, and Shen Yan. Their work addresses a limitation in conventional LLM evaluation, which typically overlooks a model's capacity to enhance its abilities through interaction during the inference phase. Humans learn continuously from their experiences, and the CoE method aims to replicate this dynamic learning process in LLMs.
The researchers instantiated CoE with various feedback mechanisms. These included feedback generated by the model itself and environmental signals, such as indicators of correctness or public coding test pass rates. The study also indicated a positive correlation between an LLM's foundational ability and its capacity for improvement within the CoE framework. Models demonstrated robustness even when exposed to weak or spurious feedback, with most improvements occurring early in the iterative process. The CoE approach also delivered higher accuracy per token compared to existing test-time strategies.
The concept of continual learning in LLMs is gaining increased attention in the research community. This area of study explores how AI systems can stay current, adapt to specific domains or users, learn from experience, and improve after deployment without losing previously acquired knowledge. This challenge is often referred to as mitigating "catastrophic forgetting," where a model forgets old information when learning new tasks. Prior research has explored continual learning through various methods, including continual pre-training, fine-tuning, and alignment. Some studies suggest that reinforcement learning (RL) may be robust to forgetting in continual learning contexts. Other approaches involve memory-augmented LLM agents that accumulate experience in external memory.
The IOL-AI Challenge, an open-science competition, recently benchmarked frontier and open models on unseen problems from the International Linguistics Olympiad. In this challenge, Claude Opus 4.8 achieved a jury score equivalent to a gold medal under strict compute budgets. While the CoE paper focuses on a different aspect of continual learning, inference-time improvement rather than competitive benchmarking, both highlight the ongoing efforts to enhance LLM capabilities beyond static training. Claude Opus 4.8 has also shown advancements in scientific and academic reasoning, leading on benchmarks like Humanity's Last Exam and CritPt, a physics evaluation. The model also demonstrated a higher score on the Legal Agent Benchmark.
The CoE research suggests that allowing LLMs to learn from their own iterative experiences can lead to more efficient and capable models. This has implications for how LLMs are deployed and maintained, potentially enabling them to adapt and improve over time without requiring extensive retraining. The findings contribute to the broader field of continual learning for LLMs, an area where fundamental challenges in achieving seamless knowledge integration still exist.