Understanding the Kolmogorov–Smirnov Test

The Kolmogorov–Smirnov (KS) test is a versatile nonparametric method for comparing empirical distributions. It gauges whether two samples originate from the same continuous distribution without assuming any particular functional form. Unlike parametric tests that rely on specific distributional assumptions, the KS test operates solely on the ordering of data values, making it especially useful when little is known about the underlying populations. It examines the maximal vertical distance between the empirical cumulative distribution functions (ECDFs) of the samples and uses that distance to infer whether the observed discrepancy is likely under the null hypothesis of identical distributions.

To appreciate the logic behind the KS test, imagine plotting the ECDF of each sample on the same axes. These step functions start at zero and jump upward at each observed data point until they reach one. If both samples arise from the same population, the two ECDFs should trace similar paths, though random sampling variation will cause slight deviations. The KS statistic, denoted $D$, captures the largest of those deviations. A large $D$ suggests that one sample tends to accumulate probability mass faster than the other in some region, indicating a systematic difference between the distributions.

The intuition for the test dates back to work by Andrey Kolmogorov and later Nikolai Smirnov in the mid‑20th century. They derived the exact distribution of the test statistic under the null hypothesis, revealing a surprising universality: after appropriate scaling, the distribution of $D$ depends only on the combined sample sizes, not on the particular form of the common distribution. This property allows the KS test to function as a distribution‑free goodness‑of‑fit tool, applicable to both theoretical comparisons and practical data analysis.

Computation of the KS Statistic

Suppose we have two independent samples $X_1,X_2,...,X_n$ and $Y_1,Y_2,...,Y_m$. Their ECDFs are defined by

$F_n\left(x\right)=\frac{{\#}_{X\le x}}{n}$ and $G_m\left(x\right)=\frac{{\#}_{Y\le x}}{m}$.

The KS statistic is then $D={sup}^x\left|F_n\right(x)-G_m(x\left)\right|$. Our calculator constructs both ECDFs by sorting the samples and scanning through unique data values, updating cumulative counts at each step. This algorithm runs in linear time after the initial sort and yields the exact $D$.

Approximating the p‑Value

The null distribution of the two-sample KS statistic can be expressed through the complementary cumulative distribution function of the Kolmogorov distribution. When sample sizes are moderate, a useful approximation for the p‑value is

$p=2{\sum }_j^{=}{-}^{j-1}{e}^{-2j^2{\Lambda }^2}$ where $\Lambda =(\sqrt{\frac{\mathrm{nm}}{n+m}}+0.12+\frac{0.11}{\sqrt{\frac{\mathrm{nm}}{n+m}}})D$.

By summing terms in this alternating series until they become negligibly small, the calculator estimates the tail probability that a difference at least as large as the observed $D$ would arise under identical parent distributions. Small p‑values—typically below 0.05—indicate that such a difference is unlikely by chance alone, leading to rejection of the null hypothesis.

Worked Example

Consider two manufacturing processes that produce slightly different measurements for a critical component. Suppose sample A consists of the diameters [5.01, 4.98, 5.03, 5.00, 5.02] millimeters, while sample B produces [4.97, 4.99, 5.04, 5.01, 4.96]. When these values are entered into the calculator, the sorted ECDFs reveal a maximum deviation of $D=0.4$. With both samples containing five observations, the approximation above yields a p‑value around 0.70. This high p‑value implies that the observed difference could easily result from random variation, so we fail to reject the hypothesis that both processes share the same distribution.

Algorithmic Steps Summarized

Step	Description
1	Parse the comma-separated samples and sort each in ascending order.
2	Scan through the ordered values to build ECDFs for both samples.
3	Track the maximum absolute difference between the ECDFs to find $D$.
4	Compute the scaling factor $\Lambda$ and evaluate the infinite series for the p‑value.
5	Report $D$, the p‑value, and sample sizes.

Interpreting Results

It is important to remember that the KS test evaluates differences in cumulative distributions, not just in means or variances. As a result, it is sensitive to shifts in location, changes in dispersion, or any other distributional discrepancy. However, it is less powerful than specialized tests if a specific alternative hypothesis is known. For instance, if you expect only a difference in means, a t‑test may detect smaller shifts than the KS test, because the latter accounts for all possible differences.

The two-sample KS test is symmetric: swapping the sample labels yields the same $D$ and p‑value. It also handles samples of different sizes with ease. When sample sizes are very small, p‑value approximations may be less accurate, so exact tables or permutation tests provide better control of Type I error. For large samples, the asymptotic formula used here performs well.

Beyond hypothesis testing, the KS statistic offers a convenient measure of distance between distributions. In machine learning and data mining, it can quantify how well synthetic data mimic real-world observations. In finance, analysts compare historical return distributions across market regimes using KS distances, highlighting structural changes that may affect risk assessments.

Although the classic KS test applies to continuous data, variants exist for discrete distributions and for one-sample goodness-of-fit assessments. The one-sample version compares the empirical distribution to a fully specified theoretical CDF. Extensions also handle cases with tied observations by applying mid‑rank adjustments or by adding small random jitter.

The KS test's reliance on empirical distributions means it does not rely on parametric assumptions, but it still assumes independent observations. Correlated data, such as time series, can inflate the probability of false positives because neighboring values influence each other. When analyzing such data, consider block bootstrap methods or other techniques that account for dependence.

Historically, the test gained prominence in the mid-20th century as statisticians sought methods that were robust to unknown distribution shapes. Its use has expanded with modern computing power, enabling practitioners to evaluate complex models by comparing simulated and observed data. The KS statistic also underlies metrics such as the Earth mover's distance and the Cramér–von Mises criterion, situating it within a broader family of distributional comparisons.

To deepen your understanding, experiment with the calculator by entering samples drawn from known distributions. Try comparing normal and uniform samples, or two normal samples with different means. Observe how the KS statistic reacts to shifts, spread changes, and outliers. Such experimentation builds intuition for how cumulative differences manifest in practice.

Because every computation occurs in your browser, no data are transmitted elsewhere. You can save this page and use it offline to analyze sensitive datasets securely. The algorithm scales well for small to moderate sample sizes, though extremely large datasets may incur noticeable sorting time. Nonetheless, for educational purposes and quick analyses, the KS test's simplicity and generality make it an invaluable addition to any mathematical toolkit.