Physics-informed neural networks (PINNs)
A neural network learns the solution of a differential equation with no training data, only the equation itself, by minimising the residual of the PDE at random points. Everything runs in your browser on a small automatic-differentiation engine written from scratch, so you can watch the network learn the physics.
Horizontal axis: x ∈ [0, 1]. Vertical axis: t ∈ [0, T], increasing upward. White dots: data points (inverse mode).
Profiles u(x) at fixed times: PINN (solid) vs exact (dashed)
Training loss (log scale)
How a PINN works
A tanh network \(u_\theta(x,t)\) is trained to minimise
$$ \mathcal L(\theta) = \underbrace{\frac1{N_f}\sum_i \big|u_t - \alpha u_{xx}\big|^2_{(x_i,t_i)}}_{\text{PDE residual}} + \lambda_{IC}\,\frac1{N_0}\sum_j \big|u(x_j,0) - u_0(x_j)\big|^2 + \lambda_{BC}\,\frac1{N_b}\sum_k \big|u(0|1,t_k)\big|^2 $$The collocation points are random and resampled every 25 steps, so no mesh is needed. \(u_t\) and \(u_{xx}\) are exact derivatives of the network, not finite differences. Writing \(a=\tanh z\), they are propagated forward through each layer as
$$ z' = Wa',\quad a' = (1-a^2)\,z',\quad a'' = (1-a^2)\,z'' - 2a(1-a^2)\,z'^2 ,$$and the gradient of \(\mathcal L\) with respect to every weight is computed by reverse-mode automatic differentiation. The test suite checks it against finite differences to 10⁻⁶.
The exact reference
$$ u(x,t) = \sum_n b_n\, e^{-\alpha n^2\pi^2 t}\sin(n\pi x),\qquad u(0,t)=u(1,t)=0 .$$Inverse problems: where PINNs shine
Make α a trainable parameter and add a data term \(\frac1{N_d}\sum|u_\theta - u_{meas}|^2\) for noisy sensor readings. The network then fits the data while obeying the physics, and the unknown diffusivity is recovered as a by-product. Classical solvers would need an outer optimisation loop around many forward solves to do this.
Honest performance note
For a 1D forward problem like this one, a classical finite-difference solver is far faster and more accurate. The readout shows FTCS solving it in milliseconds at about 0.01% error, while training takes seconds to reach about 0.1–1%. The claim that PINNs are faster than numerical methods does not hold for small forward problems. Their strengths are elsewhere: no mesh, a continuous and differentiable solution, inverse and data-assimilation problems, and amortised inference once trained (one forward pass replaces a solve).
Failure modes to explore
- Oscillator with soft constraints: untick “hard initial conditions”. The network often settles on a trivial, too-smooth solution. With the trial solution \(x = x_0 + v_0t + (t/T)^2N(t)\), the initial conditions hold exactly and training converges.
- Long time spans: set T = 12. PINNs struggle to propagate information forward in time (spectral bias and causality).
- Sharp data: the tent initial condition has a kink, and smooth tanh networks need more steps to resolve it.