Orthogonal Projection Background

The notion of projecting a vector onto a subspace lies at the heart of linear algebra and has wide-reaching consequences in geometry, statistics, and numerical methods. Given a subspace \(S\) of \(\mathbb{R}^n\) spanned by basis vectors \(\mathbf{b}_1, \mathbf{b}_2, \ldots, \mathbf{b}_k\), the orthogonal projection of a vector \(\mathbf{v}\) onto \(S\) is the unique vector \(\mathbf{p} \in S\) such that the difference \(\mathbf{v}-\mathbf{p}\) is orthogonal to every vector in \(S\). In other words, \(\mathbf{v}-\mathbf{p}\) belongs to the orthogonal complement \(S^\perp\). This decomposition \(\mathbf{v} = \mathbf{p} + (\mathbf{v}-\mathbf{p})\) is fundamental in many algorithms, from least squares regression to the Gram-Schmidt process.

When the basis vectors are assembled into a matrix \(A = [\mathbf{b}_1\ \mathbf{b}_2\ \ldots\ \mathbf{b}_k]\), the projection matrix onto \(S\) is given by

This formula arises by solving the normal equations \(A^T A \mathbf{c} = A^T \mathbf{v}\) for the coefficients \(\mathbf{c}\) of the projection in the basis of \(S\). The projected vector is then \(\mathbf{p} = A \mathbf{c}\). Computationally, it involves forming the Gram matrix \(A^T A\), inverting it, and combining the pieces. The matrix \(P\) is idempotent (\(P^2 = P\)) and symmetric, properties that uniquely characterize orthogonal projections.

For illustrative purposes, suppose \(\mathbf{v} = (1,2,3)\) and we project it onto the plane spanned by \(\mathbf{b}_1 = (1,0,0)\) and \(\mathbf{b}_2 = (0,1,0)\). In this case, \(A\) is the identity matrix in the first two columns. The Gram matrix \(A^T A\) is the 2×2 identity, whose inverse is itself. Thus the projection matrix effectively zeroes the third component, yielding \(\mathbf{p} = (1,2,0)\). The orthogonal component is \((0,0,3)\). Geometrically, this corresponds to dropping a perpendicular from the point \((1,2,3)\) onto the \(xy\)-plane.

The procedure generalizes seamlessly. When the basis vectors are not orthonormal, the Gram matrix accounts for their mutual angles and lengths. The calculator performs these matrix calculations numerically, allowing arbitrary basis vectors. As long as the basis vectors are linearly independent, \(A^T A\) is invertible. If the vectors are not independent, the determinant of \(A^T A\) becomes zero, meaning the subspace is ill-defined; the calculator detects this and warns the user.

Orthogonal projection underlies least squares approximation. Given an overdetermined system \(A\mathbf{x} \approx \mathbf{b}\), the least squares solution is exactly the projection of \(\mathbf{b}\) onto the column space of \(A\). The residual \(\mathbf{b}-A\mathbf{x}\) is orthogonal to that column space. This geometric view clarifies why normal equations emerge and why orthogonality ensures minimal error.

From a geometric standpoint, the projection splits space into complementary pieces. Every vector can be uniquely expressed as the sum of its projection onto \(S\) and a component orthogonal to \(S\). This decomposition forms the basis for Fourier series expansions, where functions are projected onto subspaces spanned by sine and cosine functions. In multivariate statistics, projecting data onto lower-dimensional subspaces (as in principal component analysis) reveals patterns and reduces noise.

Orthogonal projections also play a role in solving differential equations, optimization, and computer graphics. In optimization, constraints are often linear subspaces, and gradient methods project steps back into feasible regions. In computer graphics, projecting points onto planes or lines is essential for rendering, collision detection, and camera transformations.

The mathematics extends to abstract inner product spaces, where projections are defined similarly but rely on inner products rather than dot products. In Hilbert spaces, projection operators are central to functional analysis and quantum mechanics, where states are projected onto subspaces representing measurement outcomes.

The table below outlines the computation steps performed by the calculator:

By exploring various basis vectors and input vectors, you can build intuition for how projections behave. Try non-orthogonal bases or higher-dimensional scenarios by considering three basis vectors. Observe how the projection shifts as the subspace rotates or skews. These experiments reflect the rich geometry encoded in linear algebra.

Ultimately, understanding orthogonal projection equips you to tackle problems across mathematics and engineering. Whether fitting models to data, decomposing signals, or navigating geometric spaces, projection operators provide clarity and structure. Their properties—linearity, idempotence, and symmetry—make them powerful tools, and this calculator aims to demystify the computational steps involved.