An omnibus test: the Kolmogorov-Smirnov test
The Kolmogorov-Smirnov statistic: K-S = max w | Fhat1(w) - Fhat2(w) |
measures the maximum of the absolute differences between the empirical CDFs. A permutation test can then be constructed to test the omnibus hypothesis that F1(x) = F2(x), against the alternative hypothesis that the two CDFs are unequal.
Ranking the data helps us avoid the effects of outliers, heavy-tailed
distributions, etc. However, the difference between the distribution of raw data and
ranked data is very apparent.
It turns out that ranked scores are essentially expected values of observations from a uniform distribution. An interesting question is: is there a way to protect from outliers and other problems while retaining more information about the original distribution of the data? The answer is yes if we use different scoring systems.
Normal Scores and Van der Waerden scores
Exponential or Savage scores
General F0 scores