## Archive for **January 2011**

## Fun with maximum likelihood estimation

The following is a fun little exercise that most statistics students have probably worked out as a homework assignment at some point, but since I have found myself rederiving it a few times over the years, I decided to write this post for the record to save me some time the next time this comes up.

Given a probability density , we can approximate the probability of a sample falling within a region around the value by

Similarly, the probability of observing independent samples is approximated by

In the case of a normal distribution, the density is parameterized by two parameters and and we have

The probability of observing a given sample is then approximated by

The idea behind maximum likelihood estimation is that the parameters should be chosen such that the probability of observing the given samples is maximized. This occurs when the differential vanishes, i.e.

This, in turn, vanishes only when both components vanish, i.e.

The first component is given by

and vanishes when

The second component is given by

and vanishes when

where .

Note: The first time I worked through this exercise, I thought it was cute, but I would never compute and as above so the maximum likelihood estimation, as presented, is not meaningful to me. Hence, this is just a warm up for what comes next. Stay tuned…