CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval

CaSKG calibrates graph edges with counterfactual probes before retrieval, improving LLM agent skill retrieval across six backbones and two benchmarks while keeping the downstream policy fixed.

August 29, 20261 source60% confidence

68Strategic

Strategic score

Evidence strength45

Technical novelty78

Commercial potential55

CaSKG introduces a calibration layer for procedural skill graphs: instead of retrieving over raw edges, it scores edge direction and reliability before task-conditioned expansion. The novelty is concentrated in the offline pipeline that fuses semantic, lexical, I/O, structural, and repair evidence, then applies direction-conditioned counterfactual probes that remove, substitute, and reorder skill pairs, producing a state-filtered weighted graph.¹

Context

The constraint CaSKG removes is the dependence of retrieval on edges that are already reliable. Where full-library prompting preserves coverage at high context cost and vector retrieval treats skills as independent texts, CaSKG shifts the reliability computation to graph construction time; the downstream agent policy and task interface are unchanged, so an agent can use the new retrieval without retraining.¹

Assessment

The quantitative claim is broad: across six LLM backbones and two benchmarks, CaSKG obtains the highest task score in all twelve combinations, improving GoS macro-average ScienceWorld from 72.62 to 80.50 and ALFWorld success from 80.01% to 86.79% with fewer mean environment steps. This is evidence that edge calibration helps, but the abstract provides no confidence intervals, per-task breakdowns, or probe-level ablations, so the size of the effect is not firmly established.¹

Caveats

AI hypothesisThe strongest alternative explanation is that the gains come from the high-recall candidate graph or the LLM judge rather than from the counterfactual calibration itself. A change of mind would therefore require an ablation that fixes the candidate graph and removes only the counterfactual probes, plus a comparison on held-out tasks not used to tune the graph.¹

Source text

Hugging Face Daily Papers — CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval · August 26, 2026

Computer Science > Artificial Intelligence

Title:CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval

Abstract:Reusable skill libraries allow large language model (LLM) agents to reuse procedural knowledge across tasks, but they also turn memory access into a challenging retrieval problem. Full-library prompting preserves coverage at high context cost, vector retrieval returns compact neighborhoods but treats skills as independent text, and graph-based retrieval can recover workflow context only when the edges that carry relevance are reliable. We propose CaSKG, a counterfactual-causal skill graph framework that calibrates procedural relations before retrieval. CaSKG first builds a high-recall directed candidate graph from semantic, lexical, input/output, and structural evidence, with repair evidence and an optional LLM judge further refining candidate scores. It then applies direction-conditioned textual counterfactual probes that remove, substitute, and reorder skill pairs, aggregates the evidence with Bayesian smoothing, and publishes a state-filtered weighted graph for task-conditioned expansion. The graph is constructed offline and used without changing the downstream agent policy or task interface. Across six LLM backbones on ALFWorld ID-140 and ScienceWorld U211, CaSKG achieves the highest task score in all twelve combinations of model and benchmark. Relative to Graph-of-Skills (GoS), it improves the six-model macro-average ScienceWorld score from 72.62 to 80.50 and ALFWorld success from 80.01\% to 86.79\%, while reducing mean environment steps on both benchmarks. Qualitative and ablation analyses further show that calibrated edges help retrieval preserve prerequisites, state-changing actions, verification routines, and final completion steps. These results position edge-confidence calibration as an effective route to compact and executable skill retrieval at scale\footnote{Code is available at: this https URL }.

Access Paper:

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

The source text above remains the property of its publisher and is reproduced for commentary.

↑ Back to top