Learning the Geometry of Admissible Hypotheses through Inductive Bias in Training Distributions

Scientific discovery often requires reasoning over competing hypotheses that are consistent with experimental observations. For mixed-variable and combinatorial hypothesis spaces, however, constructing probabilistic representations remains challenging because both the active model components and their associated parameters are unknown. In this work, we present a framework for learning continuous latent representations of admissible partial differential equations (PDEs) by embedding a scientific inductive bias directly into the training distribution. Progressively richer structural principles (e.g., sparsity, logical dependencies, common PDE families, and physical admissibility) are used to generate a structured distribution of hypotheses from which a gated variational autoencoder learns a continuous latent manifold. Experimental results show that the resulting 11-dimensional representation accurately reconstructs a broad collection of representative PDEs, while exhibiting smooth geometric transitions both within and across equation families. Through an ablation study we further demonstrate that introducing scientific principles reduces both structural misclassifications of equation forms and parameter estimation errors when reconstructing a representative benchmark set of admissible partial differential equations. These results show that embedding a scientific inductive bias in the training distribution enables the learning of compact and geometrically meaningful hypothesis manifolds, providing a principled foundation for future inference over competing governing equations.

Physics-Informed Error Field Learning: A Post-Training Optimization Framework for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have emerged as an important class of numerical methods for solving partial differential equations (PDEs). However, during the late-stage optimization process, further parameter updates often yield diminishing accuracy improvements while increasing computational costs. To address this issue, this paper proposes a Physics-Informed Error Field Learning (PIEFL) framework for PINNs. Unlike conventional approaches that continuously approximate the solution field using a single network, PIEFL introduces an auxiliary error network after the primary network achieves satisfactory accuracy and shifts the learning objective from the solution field to the error field. By deriving error control equations under physical constraints, the error network learns the discrepancy between the current approximation and the exact solution, and the learned error correction is combined with the primary prediction to improve solution accuracy. The proposed framework avoids continuous optimization of the entire solution space and focuses computational resources on correcting existing prediction errors. Moreover, PIEFL requires no modification to the primary network architecture, making it compatible with existing PINN models and applicable as a general post-training optimization strategy. Numerical experiments on representative PDEs demonstrate that PIEFL achieves higher solution accuracy under the same computational budget, validating its effectiveness in improving the performance of PINNs.

A Two-Stage Learning PINN Approach for Solving the Inverse Problem of the 1D Porous Medium Equation

The Porous Medium Equation (PME), given by $u_t = Δ(u^m)$ for $m > 1$, is a degenerate nonlinear parabolic partial differential equation that arises in various physical applications such as fluid flow in porous media, heat transfer in plasmas, and population dynamics. It is known for its nonlinear diffusion and finite propagation speed. In this paper, we study numerical solutions of the one-dimensional direct and inverse PME using Physics-Informed Neural Networks (PINNs), and compare them with classical numerical methods and available analytical and manufactured solutions. While PINNs provide a flexible framework for solving both forward and inverse problems, we show that the standard inverse formulation suffers from a strong sensitivity to the initial guess, leading to only local convergence. To address this issue, we propose a novel two-stage PINN training framework for the inverse problem, which significantly improves convergence stability and allows reliable recovery of the unknown parameter even for poor initial guesses. Overall, the proposed approach demonstrates that PINNs are a flexible and accurate alternative to classical methods for the 1D PME, and the introduced two-stage training strategy substantially improves their robustness in inverse problems, providing a solid basis for extensions to more complex geometries and higher-dimensional cases.

A Novel Fourier Feature Network for Solving Partial Differential Equations

Building on the foundation of single-hidden-layer neural networks, Fourier Feature Networks (FENs) are proposed, which incorporate Fourier features using $\cos$, $\sin$, or a combination of both. Similar to Extreme Learning Machines (ELMs), FENs employ a single-hidden-layer architecture to generate a set of basis functions. The target function is then approximated as a linear combination of these basis functions, with the coefficients determined using the least squares method. However, unlike ELMs, which often rely on affine transformations to improve representational power, FENs can achieve high-precision solutions without requiring such transformations on the input variables. To evaluate the representational capacity of these networks, we search for an optimal scaling factor within a predefined range for the randomly initialized and fixed weights and biases. By adjusting this scaling factor, we ensure a fair comparison between FENs and ELMs using various activation functions, such as $\text{sigmoid}$, $\tanh$, and $\text{swish}$. Our numerical experiments demonstrate that FENs consistently achieve higher accuracy than ELMs.

Continual-Learning Physics-Informed Neural Networks for Parameterized Partial Differential Equations

Physics-informed neural networks (PINNs) incorporate governing equations into neural-network training and can approximate PDE solutions without requiring large observational datasets. Parameterized PINNs (ParamPINNs) further take physical parameters as inputs, allowing a single model to represent a family of PDE solutions over a parameter domain. Existing ParamPINNs, however, still face inefficient training, uneven accuracy across parameters, and overfitting to a limited set of sampled parameter tasks, which can impair generalization to unsampled parameters. To address these issues, we propose a continual-learning physics-informed neural network (CL-PINN), which treats PDE instances at different parameter values as related tasks and learns them sequentially. CL-PINN combines Bayesian-optimization-based active parameter selection, task-wise dynamic loss weighting, sparse physics-constrained replay, and an optional parameter subnetwork to improve task allocation and knowledge retention under bounded active-task capacity. It requires no observational data and is designed to solve parameterized PDEs over relatively broad parameter domains under limited computational resources. Multi-seed evaluations on five benchmarks, including one continuous function and four parameterized PDEs, show that Bayesian selection substantially reduces objective-loss queries relative to grid-greedy search, while sparse replay mitigates forgetting of earlier tasks. Under the prescribed within-case resource protocols, CL-PINN generally provides higher and more balanced solution accuracy than fixed-sampling and grid-greedy baselines. CL-PINN offers a practical route toward learning PDE solutions that generalize across physical parameters and has the potential to support reusable physics-informed surrogates for large-scale engineering parameter studies.

Physics-Informed Broad Learning System: An Efficient Backpropagation-Free Framework for Solving Partial Differential Equations

Physics-informed neural networks (PINNs) have emerged as a powerful paradigm for solving partial differential equations (PDEs) by embedding governing physical laws into deep neural networks. However, their reliance on computationally expensive gradient-based optimization and deep architectures often results in slow training, high computational cost, and limited scalability. In this work, we propose a novel physics-informed broad learning system (PI-BLS), the first physics-informed learning framework based on broad RdNNs. The proposed formulation embeds the governing differential operator and the associated initial and boundary constraints directly into a linear output-layer optimization problem, thereby replacing nonlinear gradient-based training with a deterministic least-squares solution obtained via the pseudoinverse. Consequently, the entire learning process is reduced to a single linear optimization stage while preserving the underlying physical constraints. As a result, PI-BLS offers an efficient learning paradigm for a physics-informed learning framework for solving PDEs that eliminates iterative backpropagation while preserving the underlying physical constraints. Experimental results on representative forward PDE benchmarks demonstrate that PI-BLS achieves competitive and often superior performance with reduced training time and model parameters compared with conventional PINNs.

LRX-PINN: A Layer-Resolving XNet Physics-Informed Neural Network with Integrated Cauchy Activations for Convection-Dominated Problems

Convection-dominated convection-diffusion problems often develop thin layers, where the solution has sharp transition profiles and its derivatives are highly localized. This creates a structural mismatch for standard physics-informed neural networks (PINNs), whose trial spaces are not designed to match the value--derivative structure of such layers. We propose a Layer-Resolving XNet Physics-Informed Neural Network (LRX-PINN) based on integrated Cauchy activations. The proposed basis is transition-type at the solution level, while its derivative recovers a localized Cauchy kernel. We show that this structure matches the scaling of convection-dominated layers, inherits the Cauchy approximation mechanism at the derivative-profile level, and identifies \(d/\|w\|\) as the effective physical width of a ridge neuron. For analytic layer profiles, this yields derivative-stable exponential approximation in the stretched coordinate and a layer-scaled estimate for the strong residual of the singularly perturbed operator. Numerical experiments on several convection-dominated benchmarks show that LRX-PINN achieves higher accuracy than PIKAN and Fourier-feature PINNs while using less than \(30\%\) of their trainable parameters. On more challenging benchmarks, embedding the proposed representation into hp-VPINN-based frameworks further improves the best results obtained by existing hp-VPINN-based baselines without changing their original loss functionals or stabilization strategies. These results show that neural representations aligned with layer structure provide a compact and effective approach for convection-dominated problems.

An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have emerged as a promising route to solve partial differential equations, yet they have struggled to reach the precision of classical solvers. The obstacle is increasingly understood to be one of optimisation, owing to the severely ill-conditioned loss landscape. We present $\textbf{DSGNAR}$: Doubly-Sketched Gauss-Newton with Adaptive Ratio, a scalable second-order optimisation framework that confronts this ill-conditioning and, in doing so, obtains unprecedented accuracy and speed. $\textbf{DSGNAR}$ couples a doubly-sketched Gauss-Newton model with a novel strategy that carefully controls both regularisation and step length. Across a suite of problems spanning nonlinear, chaotic, multi-scale, high-dimensional, and Navier-Stokes, the framework greatly improves on the state of the art: able to attain relative $\ell_2$ errors as low as $3\times10^{-16}$ in double precision, improve contemporary results by five orders of magnitude on the canonical Burgers' equation, and as much as eight orders on a high-dimensional Poisson problem, while remaining markedly faster. We further show that, in single precision, solutions at the limit of round-off error can be obtained very quickly: Burgers' equation to $\ell_2^{\text{rel}} = 4.75 \times 10^{-7}$ in under ten seconds. The framework is also robust to the choice of architecture, arithmetic precision, and initial hyperparameters. The code is available at https://www.github.com/wephy/physics-informed-neural-networks

Reliability-Aware Hard--Soft Physics-Informed Neural Networks for Robust Learning of Challenging Partial Differential Equations

Physics-informed neural networks (PINNs) provide a mesh-free framework for solving partial differential equations, but their training is often affected by loss imbalance, optimization stiffness, and difficulty in capturing localized or multi-mode solution structures. Hard-soft PINNs (HSPINN) alleviate part of this difficulty by embedding Dirichlet or periodic constraints directly into the trial space, but the resulting fixed admissible representation can still be poorly conditioned for sharp or heterogeneous residual fields. This paper proposes a reliability-aware hard-soft PINN (RA-HSPINN) that preserves exact embedded constraints while introducing a bounded learnable reliability field to modulate the interior representation. The method combines this reliability-aware ansatz with inverse-EMA global loss balancing and lightweight regularization, while retaining the standard mean-square residual form. The reliability field is a numerical modulation variable, not a physical parameter or calibrated probability. RA-HSPINN is evaluated on nonlinear Burgers equations, periodic convection, a mixed-boundary Poisson problem, and a mixed first-order Poisson system. Compared with HSPINN, it reduces the relative error by $98.65%$ for sharp-gradient Burgers, $72.42%$ for Burgers data with noisy and incompatible initial conditions, $61.18%$ for smooth periodic convection, $60.02%$ for localized periodic convection, $29.36%$ for mixed-boundary Poisson, and $82.17%$ for a multi-mode mixed first-order Poisson system. The results show that reliability-aware modulation is most beneficial when hard-soft trial spaces are admissible but difficult to optimize, especially in localized, unreliable-data, and multi-mode PDE regimes.

Domain-Decomposed Randomized Neural Networks for Partial Differential Equations in Unbounded Domains

Partial differential equations on unbounded domains are challenging because the exterior region must be represented without excessive truncation error. Truncation-based methods often require problem-dependent artificial boundary conditions, while global spectral bases may be inefficient for localized structures, irregular geometries, or solutions with different near-field and far-field behaviors. We propose a domain-decomposed randomized neural network framework for such problems. Different randomized subnetworks are assigned to different spatial regimes: a near-field subnetwork captures local and geometric features, whereas a far-field subnetwork represents exterior decay. The subnetworks are coupled by boundary and interface conditions, and only the output-layer coefficients are solved from linear least-squares systems arising from Petrov--Galerkin or collocation formulations. We develop a Petrov--Galerkin method for semi-unbounded elliptic problems and a collocation method for fully unbounded, perforated, and time-dependent problems. A conditional bounded-parameter approximation result is proved in a broken Sobolev norm, together with an error decomposition covering approximation, empirical-consistency/quadrature, and least-squares optimization errors. Numerical experiments for Poisson and time-dependent Schrödinger equations demonstrate the accuracy and flexibility of the proposed method.

A Physics-Informed Neural Network with a Modified Lorentzian Activation for Nonlocal Gradient-Flow Equations in Dynamic Density Functional Theory

We develop a physics-informed neural network (PINN) framework for nonlocal partial differential equations arising in dynamic density functional theory (DDFT). Such equations are challenging for standard PINN methods because they involve nonlinearities, nonlocal interaction terms, and an underlying gradient-flow structure, often leading to slow convergence and difficult optimization. We adapt the PINN methodology to DDFT gradient-flow equations and introduce two computational components: a modified Lorentzian activation function that behaves approximately linearly for small inputs and decays toward zero as the input magnitude increases, and a precomputed discrete operator for evaluating the nonlocal convolution term efficiently during training. The method is tested on four examples in one and two space dimensions. In the first example, the exact stationary solution is known, while in the remaining cases the neural-network approximations are validated against reference solutions computed using continuous and discontinuous Galerkin finite element discretizations. Accuracy and physical consistency are assessed through $L^1$, $L^2$, and $L^\infty$ errors, together with mass conservation and free-energy dissipation. The results show that the proposed activation function accelerates convergence relative to the standard $\tanh$ function, while the overall framework maintains good agreement with the reference solutions and captures the expected gradient-flow behaviour. These findings demonstrate the potential of the proposed PINN framework for solving nonlocal gradient-flow equations arising in DDFT.

IKNO: Infinite-order Kernel Neural Operators

Neural operators have achieved significant success in modern scientific computing due to their flexibility and strong generalization capabilities. Existing models, however, primarily rely on first-order kernel integral approximations, which severely limit their expressivity. To address this, we propose the Infinite-order Kernel Neural Operator (IKNO), which constructs neural operators via infinite-order kernel integrals and admits an elegant closed-form finite approximation. We develop two complementary infinite-order neural operator constructions: IKNO-Vanilla, which applies the full-kernel resolvent on the product grid via Kronecker eigendecomposition, and IKNO-TP, an alternative tensor-product operator that composes per-axis resolvents. Furthermore, we develop fast computation schemes for both variants of IKNO, which achieve outstanding global information aggregation while maintaining high computational efficiency. Empirically, we evaluate our IKNO on both time-dependent and time-independent benchmarks with arbitrary input shapes, including large-scale industrial datasets. Extensive experiments demonstrate that the IKNO method consistently achieves the SOTA accuracy with significant improvements on nearly all benchmark datasets while maintaining scalability to very large point clouds.