Minimizing uncertainty in estimates

Today I want to address an interesting problem, the estimation of values of a “system” (such as speed of a car, etc.) given measurements. This problem as it is posed is truly a fascinating one for controls engineers, notably since it mirrors the concept of control design. At the cores, designing a controller and an estimator (also known as an observer) are intrinsically related, and many interested problems which arise in control design also curiously reappear when designing an observer. What I want to introduce (albeit quite briefly) today is the concept of the Kalman filter and what exactly one can accomplish by employing one.

Although the namesake of the filter, Kalman, derived the concepts of the filter about fifty years ago, the Kalman filter is still frequently used and studied to this day. One might wonder how such an archaic concept (in terms of today breakneck pace of innovation) remains such a useful tool with applications ranging from the space program (Martian rovers, shuttle, etc.) to the oil industry. The Kalman filter remains important due to the first principles (I’ve used this phrase before. Essentially, what I mean is concepts arising from basic knowledge) derivation. Rather than minimizing the error of the estimate as in the case of least squares estimation, we strive to minimize the Bayesian variance estimation of the estimate. Wow, that is a ton of jargon, so I’ll try to explain the gist of it all! Here we go!

Starting with the word Bayesian… what does that mean? Well, essentially there are two (main) competing views in the world of probability. The frequentest approach and the Bayesian approach. Frequentests look at probability as a field of categorization in a sense. Given a bunch of measurements for a single event, we can categorize the probabilities of that event. However, frequenter argue that these probabilities are only valid for exactly the same circumstances! For example, imagine that we know the daily rain fall measurements for an entire month. We can statistically analyze the data, but we cannot predict using our statistics the chance of rain the following day. Why? Since the day of the week is different, and the weather patterns are different. The only way, according to a frequentest, that we could predict the rainfall the next day, is that if history repeated itself, so to speak, and exactly the same weather patterns occurred. Bayesian, on the other hand, takes a more approximate approach. We can assume that “similar” events have occurred, comparisons can be made. Engineers like to take a Bayesian viewpoint for modeling because we can thus model probabilities!

Moving on to the variance… if you’ve taking a statistics course, this term should come flooding back. Variance is just the magnitude square of the standard deviation. Essentially the variance is a way of measuring how uncertain we are of a value. An estimate with a high variance is highly uncertain, while a low variance estimate is more precise. Putting the concepts together… we want to minimize the Bayesian variance estimation. For each estimate the observer gives, we attempt to minimize the variance of the estimation such that the answer is as precise as possible! Of course, as in any detailed subject matter, I’ve shaved off a good number of details… but I’d be glad to explain more at a future date!