T1: Terminal Agent RL for Long-Horizon Tasks with TITO and R3

T1 is a 122B-parameter Mixture-of-Experts model trained with reinforcement learning to operate a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier. The paper reports 64.0% resolved on Terminal-Bench 2.1 and 27.9% on Long-Horizon Terminal Bench, surpassing GPT-5.4 and GLM-5.1, with TITO and R3 reducing the training-to-inference log-probability difference from 0.021 to 0.013.

Hugging Face Daily Papers — T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

September 11, 20261 source40% confidence

72Strategic

Strategic score

Evidence strength35

Technical novelty78

Commercial potential75

T1 is a 122B-parameter Mixture-of-Experts model trained with reinforcement learning to operate a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier. The paper reports a post-training pipeline that raises a base model from 43.8% to 64.0% resolved on Terminal-Bench 2.1, and reaches 27.9% on Long-Horizon Terminal Bench, surpassing GPT-5.4 and GLM-5.1.¹

Context

The stated motivation is a shift in agent usage toward long-horizon tasks such as coding and scientific discovery, with terminal tasks especially important. The training recipe combines an aggressively warm-started actor-critic, a dense process reward scoring trajectories by the absolute number of passing verifiers, TITO construction with drift repair at turn boundaries, rollout routing replay (R3) of per-token expert choices at every MoE layer, and an out-of-distribution training corpus disjoint from Terminal-Bench 2.1.¹

Assessment

The central technical claim is that TITO and R3 reduce the training-to-inference log-probability difference from 0.021 to 0.013 with exactly aligned zero token drift in the loss region. That addresses a known instability in RL for long-horizon agents: the mismatch between sampled token identifiers and re-tokenized training sequences. The reported benchmark gains are large enough to be material if they replicate, but the abstract provides no variance, baseline scores, or ablation isolating the OOD corpus.¹

Caveats

Confidence is limited by the evidence available: a single arXiv abstract, no peer review, no released code or data, no baseline numbers for GPT-5.4 or GLM-5.1, and no active-parameter count for the 122B total MoE. What would change the assessment: an independent replication of the Terminal-Bench 2.1 and Long-Horizon Terminal Bench numbers, full baseline scores, an ablation separating TITO, R3, dense process reward, and OOD corpus effects, and the active expert count to estimate inference cost.¹

Source text

Hugging Face Daily Papers — T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks · September 10, 2026

Computer Science > Machine Learning

Title:T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

Abstract:Agent usage is shifting toward long-horizon tasks such as coding and scientific discovery, among which terminal tasks are especially important. We introduce T1, a Mixture-of-Experts model of 122B total trained with reinforcement learning, operating a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier. We provide a comprehensive recipe: First, an aggressively warm-started to stabilize actor-critic training, with a dense process reward scoring trajectories by the absolute number of passing verifiers. Second, stable optimization through TITO construction, training on the exact sampled token identifiers with drift repair at turn boundaries, and rollout routing replay, recording the sampler's per-token expert choices at every MoE layer and replaying them during training. Third, fully out-of-distribution training corpus: isolated seeds and synthesized tasks disjoint from Terminal-Bench 2.1 ensures gains reflect genuine capability transfer over benchmark overfitting. Together, TITO and R3 cut the training-to-inference log-probability difference from 0.021 to 0.013, with exactly aligned zero token drift in the loss region. On Terminal-Bench 2.1, our post-train pipeline raises initial base model from 43.8% to T1 with 64.0% resolved. On Long-Horizon Terminal Bench, T1 reaches 27.9% and surpasses GPT-5.4 and GLM-5.1.

Access Paper:

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Passage: 'Mixture-of-Experts model of 122B total.' The 'total' qualifier is doing important work: it does not state active parameters per token or the number of experts routed per layer. Inference cost, latency, and serving feasibility depend on active parameters, not total parameters. The abstract's omission means the practical deployment constraint cannot be assessed from this text alone.

Passage: 'TITO and R3 cut the training-to-inference log-probability difference from 0.021 to 0.013, with exactly aligned zero token drift in the loss region.' The numbers are presented without units, baselines, or variance. 'Exactly aligned zero token drift' is a strong claim about numerical identity between sampling and training token paths; it is plausible only under the described replay and drift-repair mechanism, and it is not accompanied by an independent check or error bars. The basis for the 0.021 and 0.013 figures is therefore not evaluable from the abstract.

Passage: 'fully out-of-distribution training corpus: isolated seeds and synthesized tasks disjoint from Terminal-Bench 2.1 ensures gains reflect genuine capability transfer over benchmark overfitting.' The verb 'ensures' overstates what a corpus design can establish. Disjoint training data removes one source of contamination, but it does not by itself rule out overfitting to the verifier distribution, reward hacking, or incidental similarity between synthesized tasks and the benchmark. An ablation or held-out transfer suite would be needed to carry the causal claim.

The source text above remains the property of its publisher and is reproduced for commentary.

Hugging Face Daily Papers — T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

September 10, 2026endistinct angle

T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

↑ Back to top