The purpose of this article is to formalize the generalization test method for model comparison. The method has the potential to provide powerful tests of complex and non-nested models that may also differ in terms of numbers of parameters. The generalization test differs from the better known cross - validation test in the following critical procedure. Although both employ a calibration stage to estimate parameters, cross - validation employs a replication sample from the same design for the validation test, whereas generalization employs a new design for the critical test. It is argued that cross - validation tends to select unprincipled oversimplified models with small sample sizes, and unprincipled overly - complex models with large sample sizes, always missing the appropriate model (correct principles but wrong details). Two examples of the generalization test method are presented that demonstrate its superiority for selecting an appropriate model out of a set of models that also contain inappropriate models (lacking correct principles) that are either overly - complex or oversimplified. The main advantage of the generalization test is its reliance on extrapolations to new conditions, which is after all, the hall-mark of a good scientific theory.