Chain-of-Experience: Test-Time Iterative Feedback Improves LLM Accuracy and Cuts Cost
An arXiv abstract introduces Chain-of-Experience (CoE), a test-time continual improvement setting in which LLMs refine outputs through iterative self or environmental feedback. On math, coding, and knowledge tasks across 8 LLMs, CoE outperforms feedback-free baselines, with 5.6% overall improvement and 19% lower API cost, while remaining robust under weak or spurious feedback.
Chain-of-Experience (CoE) breaks with the standard practice of evaluating LLMs as a single zero-shot call per query. Instead, it treats test time as a loop in which the model receives feedback—from its own judgment, from correctness signals, or from coding-test pass rates—and can revise its output before the final answer. The study's central claim is that this relaxation pays off: aggregate accuracy rises 5.6% while API cost falls 19%.1
Context
The constraint shift is subtle but real. CoE does not violate the parameter-freeze constraint of deployed models; it exploits feedback within the context window. What changes is the unit of evaluation: a single exchange becomes a trajectory. This makes 'continual improvement' a property of inference-time computation, not of weight updates.1
Assessment
The headline numbers should be read with caution. The 5.6% gain and 19% cost reduction are aggregates, and the abstract does not state per-task or per-model distributions. The comparison class for 'higher accuracy per token' is not specified, so the ranking against strategies such as chain-of-thought or self-consistency remains opaque. What would change my mind is a replication showing CoE helps only on tasks that are already easy or where the feedback leaks the answer.1
Caveats
A stronger test would be per-model breakdowns, ablations on feedback quality, and the exact token accounting behind the cost figure. The paper itself reports robustness under weak or spurious feedback, which is a favorable sign but does not tell us how much spurious feedback degraded early versus late iterations.1
Title:Chain-of-Experience for Continual LLM Improvement
Abstract:Humans continuously learn from experience, whereas conventional large language model (LLM) evaluations ignore the models' ability to improve through inference-time interaction. In this paper, we study how LLMs learn from iterative experience at test time, a setting we refer to as Chain-of-Experience (CoE), where models accumulate experiential traces through iterative interactions with self or environmental feedback to form a continual improvement loop beyond zero-shot inference. We instantiate CoE with diverse feedback mechanisms, including model self-feedback and environmental signals such as correctness or public coding test pass rates, and evaluate across math, coding, and knowledge domains using 8 LLMs, including GPT-5, Gemini-2.5 Pro, Claude-4.5 Sonnet. Our study shows that leveraging iterative experience consistently outperforms feedback-free baselines, achieving substantial gains with self feedback alone, alongside a 5.6% overall improvement and 19% lower API cost across tasks and models. We further show that combining complementary feedback channels (e.g., model and correctness signals) yields additional gains, and that CoE delivers higher accuracy per token than existing test-time strategies. We observe a positive correlation between LLM base ability and improvement capacity, and show that models remain robust under weak or spurious feedback, with different feedback contributing to distinct improvement aspects and most gains emerging early in the iterations.
Access Paper:
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
The source text above remains the property of its publisher and is reproduced for commentary.
