The Right Fact, the Wrong Decision: SciValidityBench for Scientific Agent Memory
- Published
- Source
- arXiv
- Paper number
- 1397
- Field
- Agents / Memory
- arXiv ID
- 2610.34767
Key points
- The paper defines scientific use validity: the same p = 0.03 value changes its evidential status depending on whether the plan was fixed in advance.
- The authors built SCIVALIDITYBENCH, an executable benchmark factory whose 64 TaskSpecs cover five validity modules and six scientific domains.
- The SCISTATEFORGE compiler–reader pipeline turns distributed experimental histories into explicit decision-governing state.
- Across four model families, compiled scientific state raised valid decisions by 9.2–18.8 percentage points over a generic Event Graph.
- BM25 retrieval alone did not reproduce this cross-family gain.
- Exact repair of the decisive state recovered 18.6 validity points more on average than a matched irrelevant repair.
Paper links
External research summaries. These are not HDATF publications or measured product results.