Why are we searching for the best model

B-Course searches for the single most predictively accurate model. While this goal is relatively easy to understand, it can also be criticized. Searching for the single model also has some unwanted consequences.

Why the single most predictively accurate model?

The answer is tautological: "because it is the model that in future will correctly predict the class of the unclassified data better than any other model". But the quirks are many.

Because of the restrictions in our set of models we consider and because of the finite data sample, it may well happen, that many different models predict almost equally well. Why should we then pick the single one that we estimate (based on our finite sample) to predict best? Maybe if we had gathered one more data vector some other model would appear to be the most probable. This is totally legitimate question. Actually, if B-Course would base its predictions on many models instead of one, it would probably do better. However, many models are not too nice from interpretation point of view.

A negative consequence of the single model approach

When there are many models that have approximately the same predictivity than the best model, those other models should also be consulted when we make predictions (generalizations) for the data we have not seen (i.e., when we want to say something about cases that were not in our data sample). If we pick just one model, our predictions (generalizations) are worse than if we use many models.

B-Course, version 2.0.0	CoSCo 2002