Evolutionary rule generation for classification and prediction

Classification is an essential activity in many areas of business, science, engineering and medicine. For example in the area of credit evaluation it is important to be able to predict which potential customers are likely to be a good credit risk and which are likely to be a bad credit risk. The need for classification has resulted in the development of many types of classification techniques. These techniques range from inductive to non-inductive and include knowledge driven approaches such as case based reasoning and conversely data driven approaches such as decision trees and neural networks. In general different classification problems may have different specific requirements. For example, in some situations overall classification accuracy is of prime importance; in others situations it is the case that predicting some specific classes accurately is of greater importance than overall accuracy. Misclassifications costs may also be a significant factor. For example, when attempting to predict the credit worthiness of a credit applicant an incorrect positive prediction may be significantly more costly in financial terms than an incorrect negative prediction. We can thus see that there may be many differing requirements associated with a particular classification problem, and no two problems necessarily have the same requirements or specifications. Having established the importance of classification, and the breadth of application areas for classification, specifically rule-based classification, the main aim of this thesis is firstly to investigate and determine the desirable properties of a rule based classification technique. The next is to establish what kind of classification technique would be able to incorporate all or most of these characteristics. We then proceed to develop, demonstrate and investigate this new technique. We include a literature survey of current techniques and an assessment of desirable properties. These properties include the ability to generate explicit rules, the ability to cope with missing values and the ability to generate rules for specified classes. We conclude that an evolutionary algorithm is a prime candidate for creating a rule based classification system with the properties we require. The new evolutionary technique, called EA-classifier, is then described and developed. EA-classifier is validated on two well known databases, one binary database and one multiclass database. Further experimentation is performed using both the genetic operators and the fitness function components. As a result of this experimentation, we also establish that EA-classifier may be used for both full and partial classification. We demonstrate the new classification technique for partial classification using a real database. One feature of the new methodology is that there are many parameters and each parameter has an associated weight which can be varied, thus impacting on the characteristics of the rule systems generated. We propose and illustrate a methodology which determines the optimal parameter settings for a problem. In conclusion, using our evolutionary algorithm EA-classifier, we have demonstrated the potential for evolutionary algorithms to generate rule based systems for classification including full and partial classification on both binary and multi-class databases. The viability of such a classification system has been demonstrated applying EA-classifier to a variety of data sets. In addition we have presented a response surface methodology for finding the optimal parameter settings for EA-classifier on any given data set.