ENTLORE introduces a benchmark construction framework for enterprise question answering that explicitly tests whether systems can recover organizational relations absent from the document corpus. Unlike prior multi-source benchmarks, it does not materialize a predefined answer path; instead it reconstructs an audited enterprise world and withholds the target structure. The central claim is that latent organizational reasoning remains difficult even when gold documents are supplied.1
Context
The benchmark draws on 2,341 routine records from three source types and frames 907 questions across explicit lookup, cross-source composition, and latent reasoning. A versioned truth graph certifies derived relations, producing complete golden answers and proof certificates. This design moves the constraint from compositional retrieval of stated facts to recovery of implicit relations across heterogeneous enterprise records.1
Assessment
Evaluation across 56 model and access configurations shows that induced entity graphs or navigable knowledge bases yield the strongest deployable results. However, with gold documents provided, 30.4% of latent questions remain unanswered, versus 12.6% for explicit and 6.2% for compositional questions. The gap indicates that document recall is not the bottleneck; making implicit organizational relations usable is the unresolved constraint.1
Caveats
AI hypothesisThe results come from a single benchmark paper with no external replication or human baselines reported. Evidence of reasoning gains would be strengthened by future work showing methods that improve latent accuracy beyond graph or knowledge-base access. If later analyses show that a large fraction of latent questions can be solved by simple lexical variation of explicit queries, the benchmark's distinctness would be called into question.1
