Information matrix test

In econometrics, the information matrix test is used to determine whether a regression model is misspecified. The test was developed by Halbert White, who observed that in a correctly specified model and under standard regularity assumptions, the Fisher information matrix can be expressed in either of two ways: as the outer product of the gradient, or as a function of the Hessian matrix of the log-likelihood function. Consider a linear model $\mathbf{y}=\mathbf{X}\mathit{\beta }+\mathbf{u}$, where the errors $\mathbf{u}$ are assumed to be distributed $\mathrm{N}(0,{\sigma }^2\mathbf{I})$. If the parameters $\beta$ and ${\sigma }^2$ are stacked in the vector ${\mathit{\theta }}^{\mathsf{T}}=\left[\begin{array}{cc}\beta & {\sigma }^2\end{array}\right]$, the resulting log-likelihood function is

$\ell (\mathit{\theta })=-\frac{n}{2}\log {\sigma }^2-\frac{1}{2{\sigma }^2}{(\mathbf{y}-\mathbf{X}\mathit{\beta })}^{\mathsf{T}}(\mathbf{y}-\mathbf{X}\mathit{\beta })$

The information matrix can then be expressed as

$\mathbf{I}(\mathit{\theta })=\mathrm{E}\left[\left(\frac{\partial \ell (\mathit{\theta })}{\partial \mathit{\theta }}\right){\left(\frac{\partial \ell (\mathit{\theta })}{\partial \mathit{\theta }}\right)}^{\mathsf{T}}\right]$

that is the expected value of the outer product of the gradient or score. Second, it can be written as the negative of the Hessian matrix of the log-likelihood function

$\mathbf{I}(\mathit{\theta })=-\mathrm{E}\left[\frac{{\partial }^2\ell (\mathit{\theta })}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\mathsf{T}}}\right]$

If the model is correctly specified, both expressions should be equal. Combining the equivalent forms yields

$\mathrm{\Delta }(\mathit{\theta })=\sum _{i=1}^n[\frac{{\partial }^2\ell (\mathit{\theta })}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\mathsf{T}}}+\frac{\partial \ell (\mathit{\theta })}{\partial \mathit{\theta }}\frac{\partial \ell (\mathit{\theta })}{\partial \mathit{\theta }}]$

where $\mathrm{\Delta }(\mathit{\theta })$ is an $(r\times r)$ random matrix, where $r$ is the number of parameters. White showed that the elements of $n^{-1/2}\mathrm{\Delta }(\hat{\mathit{\theta }})$, where $\hat{\mathit{\theta }}$ is the MLE, are asymptotically normally distributed with zero means when the model is correctly specified. In small samples, however, the test generally performs poorly.

Further reading

• Krämer, W.; Sonnberger, H. (1986). The Linear Regression Model Under Test. Heidelberg: Physica-Verlag. pp. 105–110. ISBN 3-7908-0356-1.
• White, Halbert (1994). "Information Matrix Testing". Estimation, Inference and Specification Analysis. New York: Cambridge University Press. pp. 300–344. ISBN 0-521-25280-6.