Posted by James Youngman on June 29, 1998 at 08:29:38:
I have an application in which a least-squares fit
needs to be done online (i.e. without storing the
whole input vector). I have somple code to do this.
It turns out that I also need to perform
extreme-outlier rejection and I don't know how to
do that (online).
Details: Input data is linear, gradient may be + or
- but the sign is known in advance. N is between 10
and 500,000. The noise is normal, and the signal to
noise ratio varies between roughly 1e-4 and 1e0.
Where the noise ratio approaches 1e0 N approaches
As well as the normally distributed noise, there are
rogue measurements which are the only ones that we
must reject. They're wild. We we measure S.D. of the
successive y deltas having rejected these outliers
by eyeball, the measurements to be rejected lie at
Z values in excess of (say) 50.