Psychological theories of categorization have generally focused on either rule- or exemplar-based explanations of categorization. We present two experiments that show evidence of both rule induction and exemplar encoding, and we present a connectionist model (ATRIUM) that specifies a mechanism for combining rule- and exemplar-based representation. In both experiments participants learned to classify items, most of which followed a simple rule although there were a few, frequently occurring exceptions. Experiment 1 examined how people extrapolate beyond the range of trained instances. Experiment 2 examined the effects of instance frequency on generalization to novel cases. We found that categorization behavior was well described by the model, in which exemplar representation is used for both rule and exception processing. A key element in correctly modeling categorization in tasks such as these was capturing the interaction between the rule- and exemplar-based representational structures using shifts of attention between rules and exemplars.