Estimating Parameters

Suppose we draw $n$ independent samples from a normal distribution with unknown mean $\mu$ and variance ${\sigma }^2$. The Fisher information quantifies how much data tell us about these parameters. More formally, it measures the curvature of the log-likelihood function at the true values. Steep curvature means small changes in the parameter drastically affect the likelihood, indicating greater information.

For the normal distribution the log-likelihood is a quadratic form. Differentiating twice and taking expectations yields a symmetric matrix with entries that depend on $n$ and $\sigma$. This matrix is called the Fisher information matrix, often written $I$.

Closed-Form Expression

When both mean and variance are unknown, the Fisher information matrix for a single observation is

$\left[\begin{array}{cc}\frac{1}{{\sigma }^2}& 0\\ 0& \frac{2}{{\sigma }^2}\end{array}\right]$

Scaling this by the sample size gives the total information for $n$ independent draws. The off-diagonal entries vanish because the mean and variance are orthogonal parameters in the normal family.

Interpreting Information

Large Fisher information indicates that an unbiased estimator of the parameter will have small variance according to the Cramér–Rao bound. Conversely, small information implies that data contain limited guidance. For the normal distribution, increasing the sample size or decreasing the variance raises the information in a straightforward manner.

Because the matrix is diagonal, its inverse is simple to compute. The variance lower bounds for unbiased estimators of the mean and variance are $\frac{{\sigma }^2}{n}$ and $\frac{{\sigma }^4}{\mathrm{2n}}$, respectively.

Example Calculation

Imagine $n=50$ and $\sigma =3$. Plugging these values into the formulas yields a Fisher information matrix with diagonal entries $\frac{50}{3^2}=\frac{50}{9}$ and $\frac{100}{3^2}$. This illustrates how easily the information scales with sample size.

Broader Impact

Fisher information plays a central role in statistics and information theory. It connects to maximum likelihood estimation, asymptotic normality, and Bayesian priors. In many models, high information corresponds to narrower posterior distributions, meaning the data strongly constrain the parameter values.

Understanding Fisher information helps in experiment design, where one aims to maximize expected information by choosing appropriate sample sizes and measurement precision. It also appears in physics in the study of quantum measurement and thermodynamic uncertainty relations.