Relevance Is Not Authority: A Sealed Boundary Benchmark of Decision-Confusion in Commercial LLM APIs

Abstract

Modern LLM evaluation focuses on hallucination rate, factual accuracy, and aggregate benchmark scores. We argue that an equally important and more operationally expensive failure mode is authority confusion: when an AI system treats topically relevant content as evidence, evidence as action permission, action as publish authority, or peer comparison as same-identity. None of these are hallucinations; the system is “right” about the topic. The system has crossed a boundary the business never granted.

We define six authority layers (related, same_identity, evidence_support, action_ready, publish_safe, customer_safe) and eight failure categories, derived from controlled experiments and validated against current commercial APIs. We then run a sealed, checksum-locked benchmark of 1,000 boundary packets producing 2,990 scored decisions across OpenAI GPT-5.5, Anthropic Claude Opus 4.7, and Cohere Command A Reasoning as decision models, with Voyage rerank-2.5 as the relevance baseline. Across all three decision models we observed 214 false-authority decisions; a deterministic boundary guard reduced this to 0 while preserving 268+/270 valid allows on each model. The strongest decision model still produced 31 false-authority decisions before the guard. Authority confusion is consistent across vendors, categorical in shape, and not closed by model selection alone.