Large language models are increasingly used as evolutionary engines for scientific discovery: generate candidates, select winners, feed them back as parents, and repeat. We audit whether this loop actually compounds discovery in scientific equation discovery, a setting where finite samples make structure underdetermined and interpolation easy. Under matched LLM-call budgets, parent-conditioned evolution is indistinguishable from fresh independent sampling: median OOD NMSE is 0.045 vs. 0.049, instructed multi-parent crossover is worse, final success is predicted by initial proposal quality, and multiple iteration schemes fail to add solved problems. Operationally, the loop reduces to what it produces: a dictionary of candidate terms. We turn that diagnosis into PTB-Search, a one-generation method for componentized scientific discovery. PTB-Search samples independent LLM proposals once, extracts reusable terms into a per-problem dictionary, and performs train-only set-level sparse selection with least-squares coefficients. Its central principle is that underdetermined data identifies the joint behavior of term sets, not reliable per-term credit. On identical dictionaries and zero additional LLM calls, set-level selectors solve 165--169 of 717 cells, while single-term reductions solve only 74--78. On the official 239-problem LLM-SRBench split, PTB-Search reaches 73.2% Acc0.1 with Llama-3.1-8B and 77.0% with a single-seed DeepSeek-V4 anchor, versus 49.2% for the best reported baseline, using one tenth of the standardized call budget. A program-domain stress test gives a scoped boundary: generation count remains unreliable, while retained external state can help in harder non-linear spaces. Across these results, LLMs are best understood as material suppliers; discovery is carried by external set-level selection over reusable components.
Neural Combinatorial Optimization (NCO) achieves strong performance, yet its black-box nature remains a key roadblock to deployment and scientific diagnosis. Standard interpretability tools, such as Concept Bottleneck Models (CBMs), are ill-equipped for NCO, whose decisions are dynamic, state-dependent, and lack proper concept vocabulary definition. To close this gap, we introduce Evolving Programmatic Bottlenecks (EPB), to our knowledge, the first framework for interpreting NCO policies by distilling black-box NCO models into human-readable program portfolios. EPB employs an LLM to autonomously evolve a bank of programs, where each program's per-step action distribution serves as the bottleneck. EPB works through an iterative framework: Block I fixes program bank capacity and introduces a hybrid textual-numerical gradient descent scheme that couples numerical gradients for student router updates and textual gradients for LLM-based program revision; Block II dynamically adapts bank capacity via fault-targeted expansion and redundancy pruning. Extensive experiments demonstrate EPB's effectiveness and broad applicability, where the distilled program portfolios largely match original performance. EPB also reveals that NCO behavior shifts across optimization stages and can be approximated as a composition of classic heuristic variants. Our work advances interpretable NCO and establishes EPB as a promising tool for interpreting sequential decision-making models.