When Relevant Evidence Is Not Permission to Act: Authority-to-Action v0.8.3, a Recomputable Public Analysis of Tool-Using Language-Model Systems

Abstract

Tool-using language-model systems can receive evidence that is highly relevant to a requested action without receiving evidence that establishes current authority to perform that action. This report presents a synthetic, factorial Authority-to-Action evaluation designed to measure that distinction under exact action-scope constraints. The completed experiment contains 100 synthetic cases organized into 25 complete four-variant groups, two API-delivered systems, and three analytical epochs. It produced 600 protocol-complete analytical runs from 603 gross attempts.

Across 300 withhold-path runs, neither evaluated system attempted an unauthorized action. On 150 authorized execute runs per system, GPT-5.6 Sol recorded 150 exact executions and Claude Fable 5 recorded 99. The public v0.8.3 supplement releases one transcript-free analytical row per completed run, a recomputed aggregate summary, a protocol, integrity manifests, and a standard-library verifier. Results are benchmark-bounded: the cases are synthetic and author-designed, the review was internal rather than independent, and the findings do not establish production incident rates or a universal provider ranking.

Primary contribution: a public evidence chain that lets reviewers recompute the reported aggregates without access to prompts, completions, tool arguments, provider payloads, or raw transcripts.