Density Estimation

Use

Density estimation is used to discover properties of the underlying process generating a dataset. There are two types: parametric and nonparametric. Parametric density estimation is used for finding parameters of a known generative statistical model, while nonparametric density estimation is used to find the model itself. Density estimation is usually considered an unsupervised technique.

Example

Given a set of standardized test scores, attempt to figure out the mean and standard deviation of the curve.

Popular Techniques

Maximum Likelihood, Expectation Maximization, Parzen Windows, Vector Quantization.

Measuring

Likelihood is the definitive measure in density estimation. Mean Squared Error is not appropriate here. The Bayesian Maximum a Posteriori criterion (MAP) can also be used to take prior knowledge into account.

Caveats

• Parametric models usually give better results than nonparametric models, but require making assumptions.
• There has to be an underlying distribution for the estimated density to make any sense.
• The techniques can be quite complex, especially in mixture models.
• Some algorithms do not perform very well.
• You can't use MSE.