A Unified Calculus-Based Analysis of Polynomial Interpolation and Least-Squares Regression: Theory, Conditioning, and a Novel Stability Result

This paper presents a mathematically rigorous comparison between polynomial interpolation (Lagrange form) and least-squares approximation through the framework of calculus and classical approximation theory. From first principles, interpolation polynomials and the normal equations are derived via stationary conditions on the sum-of-squared-residual functional, highlighting interpolation's exactness at data nodes versus regression's minimization of global L 2-error. Stability and error behavior are contrasted: Runge's phenomenon and ill-conditioned Vandermonde systems for interpolation versus the stabilizing role of orthogonal polynomials in least squares. A new theorem is proved showing that, with quasi-uniform nodes, the discrete Gram matrix in a continuous orthogonal basis approximates the identity with error O(1/n 2), explaining the enhanced stability of least-squares under such distributions. Numerical experiments complement the theory, documenting condition numbers and errors across node choices. The paper concludes by noting how these principles extend to modern statistical and machine-learning settings.