Adam Fisch, Shubhendu Trivedi, Fantine Huot +5cs.AI
Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost. Routing requires estimating each specialist's expected return, but this value estimation has a cost. Cheap estimators (e.g., embedding-based predictors) are fast but noisy, while accurate estimators (e.g., fine-tuned models with access to retrieval results or partial reasoning traces) are expensive. We formalize this tradeoff as an instance of Pandora's Box, the classical problem of optimal search with costly inspection. Under a Gaussian signal model, the resulting policies have closed-form value-of-information expressions that determine, for each specialist and input, whether refining the value estimate is worth its cost. We call the centralized policy Pandora's Router. We extend this to a decentralized setting, Pandora's Bidder, where specialists independently decide whether to invest in self-assessment before accepting an offered price to claim a query. Experiments across three domains---a standard multi-LLM benchmark, retrieval-augmented specialists, and LLMs with variable inference-time reasoning---show that Pandora's Router matches the routing quality of exhaustive estimation, while querying the expensive estimator far less often. In the decentralized setting, value-of-information reasoning improves allocative efficiency when competing estimates are accurate; when competing estimates are noisy, however, it can increase the strategic specialist's utility at the expense of others.
Mixtures of low-rank adaptation experts increase parameter-efficient capacity by routing each input through a subset of adapters. Recent dynamic routers activate more experts when the router or prediction is uncertain. This rule silently equates uncertainty with useful additional computation: an uncertain example may contain complementary, unqueried expert evidence, but it may instead remain ambiguous after every expert agrees. We formulate routing as certified value-of-information allocation. VI-MoLE learns the counterfactual risk remaining after each expert prefix, converts these predictions into simultaneous upper-risk certificates on held-out calibration data, and spends a global adapter budget on the token--layer action with the largest certified marginal risk reduction per unit cost. A terminal certificate then decides whether to answer or abstain. Unlike an uncertainty gate, this procedure distinguishes present ambiguity from recoverable and residual risk. We prove simultaneous certificate validity, optimal greedy allocation under diminishing certified gains, and allocation regret under value-estimation error. The evaluation protocol tests matched-compute accuracy, certificate coverage, risk--coverage, distribution shift, and tail latency against fixed and dynamic MoE-LoRA routers.
Value-of-information (VOI) analysis is usually conducted under a single probability measure. However, in practice, the available evidence often pins the measure down only to a set. Consequently, under a set of probability measures, VOI requires different formulations. First, we explicate a rule-specific VOI that fixes a decision rule for acting under imprecision (such as Gamma-maximin) and measures what the information is worth to a decision maker who uses that rule. Second, we derive a fixed-measure envelope that evaluates the classical VOI functional over all admissible precise measures. We formalize this distinction and explicate its consequences for the expected perfect, partial, and sample information. The expected value of perfect information is concave over the credal set. Hence, when the set is generated by finitely many measures, its lower envelope endpoint is obtained exactly from the generators, while its upper endpoint may be interior and is computed by a finite linear program. The Gamma-maximin value, in contrast, can exceed the entire envelope, so a rule-specific value is not recovered from the envelope's endpoints. A continuity bound limits how much the VOI can change as the measure varies, and we identify when the partial- and sample-information endpoints can still be obtained from the generators. Because the single-measure VOI must itself be estimated, the procedure we give combines standard estimators for it with a search over the credal set. By using a worked decision problem, we show how the two quantities separate conclusions that hold across every admissible measure from conclusions that depend on one unidentified choice of measure.
In a world of generative AI, candidate insights are abundant; what is scarce is the capacity to discern which matter, to act on them in the right amount and order, and to forget the rest so the system can adapt. We argue these scarcities are governed by one object and build a framework around it. We define an insight strictly as a lever with an identified, measurable effect on an objective, and rank candidates by decision-relevance via the expected value of information rather than novelty. We show action carries an order, not only a size: under realistic belief dynamics, content "touches" are non-commuting operators, so a fixed plan delivered in different orders yields different outcomes, defining a sequence premium. We observe that the value of any lever is a shadow price, unifying pharmaceutical marketing, equity selection, and manufacturing as one leverage-discovery problem. Most speculatively, we propose APOHA, a theory in which forgetting is not the disposal of knowledge but the operator by which value is learned: the value of a retained item is the counterfactual cost of forgetting it, a learning system is the residue of maximal forgetting subject to preserved value, and higher-order value is the structure that survives repeated forgetting (a renormalisation-relevant invariant), with consolidation as its conjugate. We state the central open problem (a non-trivial attractor with a spectral gap) and test the forgetting theory: operationalising APOHA as an agent on a non-stationary obesity-treatment decision world over 30 seeds, adaptive forgetting cut cumulative decision-regret by 24-32% against never-forget and a fixed half-life, kept a ~6x smaller, cleaner memory, and converged stably; notably, blind forgetting was worse than never forgetting, so the benefit is specific to value-aware forgetting. A multi-disciplinary critique stress-tests the whole.