Numerical Methods

Numerical methods give approximate answers when a closed-form solution is impractical. The FE tests which method applies, its convergence/error order, and a single hand iteration — rarely a long computation. Know the trade-offs: guaranteed-but-slow versus fast-but-fragile.

Root-Finding

Bisection Method

If f(a) and f(b) have opposite signs (f(a)·f(b) < 0), the Intermediate Value Theorem guarantees a root in [a, b]. Repeatedly take the midpoint c = (a+b)/2 and keep the half-interval that still brackets the sign change.

Convergence: guaranteed but slow — the bracket (and thus the error bound) halves each step: error ≤ (b−a)/2ⁿ. It does not require f to be differentiable.

Newton-Raphson Method

x_{n+1} = x_n − f(x_n)/f′(x_n). Fast (quadratic) convergence near a root, but it requires the derivative, can diverge from a poor start, and fails if f′(x_n) = 0.

Worked example — Newton-Raphson. Find a root of f(x) = x² − 4 starting at x₀ = 3, with f′(x) = 2x. First step: x₁ = 3 − (3²−4)/(2·3) = 3 − 5/6 = 3 − 0.833 = 2.167. A second step gives x₂ = 2.167 − (2.167²−4)/(2·2.167) = 2.167 − 0.696/4.334 ≈ 2.006, rapidly approaching the exact root x = 2.

Secant Method

x_{n+1} = x_n − f(x_n)·(x_n − x_{n−1})/(f(x_n) − f(x_{n−1})). It approximates f′ with a finite difference, so it needs no analytic derivative; convergence is superlinear (order ≈ 1.618).

Choosing a Root-Finder

The FE often frames this as a trade-off: bisection is robust (cannot fail once a bracket exists) but needs many iterations; Newton-Raphson is fast but can shoot off to a different root or diverge if f′ is small near the start; the secant method is a middle ground that keeps Newton-like speed without computing a derivative. A practical engineering strategy — and a defensible FE answer — is to bracket coarsely with bisection, then switch to Newton-Raphson once you are near the root.

Numerical Integration

With n equal subintervals of width h = (b−a)/n and nodes xᵢ = a + ih:

Simpson's 1/3 is far more accurate than the trapezoidal rule for smooth functions because it fits parabolas instead of straight lines.

Worked example — trapezoidal rule. Estimate ∫₀² x² dx with n = 2 (so h = 1, nodes x = 0, 1, 2; f = 0, 1, 4). Trapezoidal: (1/2)[0 + 2(1) + 4] = (1/2)(6) = 3.0. The exact value is [x³/3]₀² = 8/3 ≈ 2.667, so the trapezoidal estimate is high by ~0.33 — straight-line segments overshoot a convex curve.

Worked example — Simpson's 1/3. Same integral, same nodes: (1/3)[0 + 4(1) + 4] = (1/3)(8) = 2.667, which equals the exact value. Simpson's rule integrates any cubic (and hence x²) exactly, demonstrating its O(h⁴) accuracy advantage.

Numerical ODE Solvers

For dy/dx = f(x, y) with y(x₀) = y₀ and step size h:

RK4 evaluates the slope four times per step (k₁ at the start, k₂ and k₃ at the midpoint, k₄ at the end) and is the standard workhorse for its accuracy-vs-effort balance.

Worked example — Euler's method. Solve dy/dx = x + y, y(0) = 1, with h = 0.1; find y at x = 0.1. y₁ = y₀ + h·f(x₀, y₀) = 1 + 0.1·(0 + 1) = 1.10. The exact solution y = 2e^x − x − 1 gives y(0.1) = 1.1103, so Euler's first-order O(h) error already shows a small lag.

Error Analysis

Significant figures: in multiplication/division the result carries as many sig figs as the least-precise factor; in addition/subtraction align by decimal place. Worked example — relative error. If a method yields 2.167 for a true value of 2.000, the absolute error is 0.167 and the relative error is 0.167/2.000 = 0.0835 = 8.35%.

Step Size, Stability, and the Speed-Accuracy Trade

Shrinking the step size h reduces truncation error — halving h cuts a trapezoidal estimate's error by ~4× (O(h²)) and a Simpson or RK4 estimate by ~16× (O(h⁴)). But arbitrarily small h is not free: more steps mean more arithmetic and more accumulated round-off error, and some explicit ODE methods become numerically unstable if h is too large for the problem's fastest-changing mode. The practical takeaway the FE tests is that total error has two competing parts, and the optimal h balances truncation against round-off rather than driving h toward zero. When a question gives you the error order, you can predict how the error scales without re-running the computation — e.g., a method labeled O(h⁴) that already gives 1% error will give roughly 0.06% error if you halve h.