Simple Outlier Detection in a Kalman Filter | David Gurevich<br>Simple Outlier Detection in a Kalman Filter<br>2025-12-13<br>I&rsquo;ve honestly spent a lot of time thinking about outliers and change detection in real-time data streams.<br>You can get into some pretty serious rabbit holes when thinking about this stuff.<br>If you&rsquo;re not careful, you might accidentally end up in non-parametric land<br>with quickest change detection or some other crazy ideas1.<br>I think something a bit more down-to-earth2 and maybe more applicable to our day-to-day work as scientists<br>and engineers is the Kalman filter which takes sequential<br>measurements of some underlying process (where the measurement and process errors are Gaussian) and comes up,<br>basically, with increasingly precise estimates of the underlying value.<br>I want to assume some familiarity with the Kalman filter in this post, otherwise I&rsquo;ll never actually finish<br>writing this thing.<br>What I aim to do here is to give you a tiny tool in your toolbox that you can apply when building a Kalman<br>filter in your own work.<br>The assumption that the measurement and process errors are jointly Gaussian3 is sometimes incorrect and<br>it is beyond your control to actually make the correction.<br>One instance is if the sensor you&rsquo;re using has some sort of failure mode.<br>For instance, if your sensor is some sort of positioning system, it might actually have several possible sources<br>of information from which it can determine your position, for example:<br>a GNSS system, like GPS, or<br>mulitlateration with the help of nearby cell towers, or<br>WiFi fingerprinting.<br>If you are in a place where GPS reception might be bad, which in large cities happens quite often, your phone might<br>be estimating your position based on other factors, like your proximity to known cell towers.<br>The errors here can get huge, and these failure modes might result in huge jumps.<br>I&rsquo;m sure you can imagine some other applications where your sensor might have failure modes.<br>A thermometer that&rsquo;s been reading 26 degrees celsius for hours might accidentally jump up to 255 degrees celsius for<br>a second.<br>Even if the device combusted, I think that your first intuition as an engineer would be to reject that measurement.<br>Fortunately, the Kalman filter gives us the tools necessary to perform an uncertainty calculation on incoming measurements.<br>Suppose we have a new measurement at time \(t\), call it \(z_t\).<br>The innovation in measurement \(z_t\) is given by<br>\[ y_t = z_t - H x_{t|t-1}. \]We say innovation because the idea is that \(H x_{t|t-1}\) is supposed to contain all the information that we have about the system<br>up to time \(t\).<br>You can almost think about it as \(y\) being orthogonal to the vector space of all known information, \(H x_{t|t-1}\).<br>Now, in the normal case, if you made the right sort of assumptions about your sensor, you will note that \(y\) is actually supposed to have<br>a zero mean Gaussian distribution with some covariance, \(S\).<br>We can actually calculate \(S\).<br>\[<br>\begin{align*} S &= \text{Cov}(y) \\<br>&= \text{Cov}(z) + \text{Cov}(H x_{t|t-1}) \\<br>&= R + H \Sigma_{t|t-1} H^T,<br>\end{align*}<br>\]where \(\Sigma_{t|t-1}\) is the covariance of the estimate at time \(t\).<br>Now since we have a point, \(y\), and the covariance of the innovation, \(S\), we can find the Mahalanobis distance.<br>Essentially, the distance between the point and the distribution.<br>The distance is given by<br>\[ d^2 = y^T S^{-1} y. \]What&rsquo;s nice about the Mahalanobis distance is that \(d^2\) follows a \(\chi_n^2\) distribution,<br>where \(n\) is the dimension of \(y\).<br>This proof falls out quite nicely when you try to take a multivariate approach to proving that the sum of squared residuals inversely scaled by the<br>variance also follows a \(\chi^2\) distribution.<br>What this means is that you can design a gate for highly unlikely measurements.<br>For instance if we let \(D^2 \sim \chi^2_2\), then we have \(\mathbb{P}(D^2 > 9.21) = 0.01\).<br>This means that we can reject cases where \(d^2 > 9.21\), because, given all that we know about the system at this moment in time,<br>there is less than a \(1\%\) probability of that measurement occurring.<br>In code, this might look something like the following:<br>1def update(self, z):<br>2 y = z - self.H @ self.x<br>3 S = self.H @ self.P @ self.H.T + R<br>4 K = self.P @ self.H.T @ np.linalg.inv(S)<br>6 # Mahalanobis gate<br>7 maha2 = y.T @ np.linalg.inv(S) @ y<br>8 threshold = 9.210 # chi-square 2 DOF, 99% confidence<br>10 if maha2 threshold:<br>11 self.x = self.x + K @ y<br>12 self.P = (np.eye(2) - K @ self.H) @ self.P

This code will flat-out ignore measurements that don&rsquo;t make it past the gate.<br>Depending on the expected experience, you might also want this to trigger an increase in either the process<br>error covariance or the sensor error covariance, so that you can demonstrate more uncertainty in your final estimate.<br>I think it&rsquo;s also worth noting that you shouldn&rsquo;t use this as a crutch in critical situations,<br>but I&rsquo;d hope that if...