## Archive for the ‘**Statistics**’ Category

## Weighted Likelihood for Time-Varying Gaussian Parameter Estimation

In a previous article, we presented a weighted likelihood technique for estimating parameters of a probability density function . The motivation being that for time series, we may wish to weigh more recent data more heavily. In this article, we will apply the technique to a simple Gaussian density

In this case, the log likelihood is given by

Recall that the maximum likelihood occurs when

A simple calculation demonstrates that this occurs when

and

where .

Introducing a weighted expectation operator for a random variable with samples given by

the Gaussian parameters may be expressed in a familiar form via

and

This simple result justifies the use of weighted expectations for time varying Gaussian parameter estimation. As we will see, this is also useful for coding financial time series analysis.

## Fun with maximum likelihood estimation

The following is a fun little exercise that most statistics students have probably worked out as a homework assignment at some point, but since I have found myself rederiving it a few times over the years, I decided to write this post for the record to save me some time the next time this comes up.

Given a probability density , we can approximate the probability of a sample falling within a region around the value by

Similarly, the probability of observing independent samples is approximated by

In the case of a normal distribution, the density is parameterized by two parameters and and we have

The probability of observing a given sample is then approximated by

The idea behind maximum likelihood estimation is that the parameters should be chosen such that the probability of observing the given samples is maximized. This occurs when the differential vanishes, i.e.

This, in turn, vanishes only when both components vanish, i.e.

The first component is given by

and vanishes when

The second component is given by

and vanishes when

where .

Note: The first time I worked through this exercise, I thought it was cute, but I would never compute and as above so the maximum likelihood estimation, as presented, is not meaningful to me. Hence, this is just a warm up for what comes next. Stay tuned…