Geometric Interpretation of Least Squares (OLS): Projection onto Column Space

This work presents a geometric interpretation of ordinary least squares (OLS) as a projection of a data vector onto the column space of a design matrix. The vector y is decomposed into two orthogonal components: y∥ in col(X) and y⊥ orthogonal to col(X). The fitted values are given by ŷ = Xβ, while the residual y − ŷ lies in the orthogonal complement of the column space. A concrete visualization in ℝ³ (three observations) is used to illustrate the geometry of least squares, where the column space forms a plane and the solution corresponds to the orthogonal projection of y onto that plane. This representation emphasizes the structural relationship between data, model space, and residual, rather than focusing on computational procedures. The material is part of the GraphMath project, which focuses on geometric and structural understanding of linear algebra.