STATE-OF-THE-ART IN SUPERVISED CLASSIFICATION

Speakers

Registration

Abstracts

Combination Methods in Classifier Ensembles

Classifier ensembles are deemed to be more accurate than single classifiers. While the advantages of using ensembles are provable under some (quite restrictive) assumptions, such advantages in the general case are far from being guaranteed. Much of the chance for success of a classifier ensemble depends upon the way the individual classifiers are combined. In this talk various methods for aggregating classifier votes will be summarised. The classifier fusion and classifier selection approaches will be outlined. Within the classifier fusion approach, two groups of methods will be discussed to account for the two most common types of classifier outputs:- class labels and estimates of the posterior probabilities for the classes. Optimality arguments will be presented for some of the combination methods including the weighted majority voting used by AdaBoost. Classifier selection approach will be introduced and the dynamic classifier selection method will be explained. Although the talk is planned as an overview, the intended level of detail should be sufficient for reproducing various combination algorithms and methods.

Classifier technology and the illusion of progress

Supervised classification methods are widely used in data mining. Highly sophisticated methods have been developed, using the full power of recent advances in computation. However, these advances have largely taken place within the context of a classical paradigm, in which construction of the classification rule is based on a ‘design sample’ of data randomly sampled from unknown but well defined distributions of the classes. In this paper, I argue that this paradigm fails to take account of other sources of uncertainty in the classification problem, and that these other sources lead to uncertainties which often swamp those arising from the classical ones of estimation and prediction. Several examples of such sources are given, including imprecision in the definitions of the classes, sample selectivity bias, population drift, and use of inappropriate optimisation criteria when fitting the model. Furthermore, it is argued, there are both theoretical arguments and practical evidence supporting the assertion that the marginal gains of increasing classifier complexity can often be minimal. In brief, the advances in classification technology are typically much less than is often claimed.

Rademacher Analysis and Multi-view Classification

The talk will review the frequentist analysis of classification using Rademacher complexity with its application to SVMs. A new algorithm SVM-2K will be introduced. SVM-2K is a multi-view learning technique that leverages the power of data-fusion to improve generalization error. Its performance can be analysed using the Rademacher approach, but the resulting bound is non-trivial to evaluate. Experimental results will illustrate the ideas. A link will be drawn with a theoretical framework for assessing semi-supervised learning proposed by Balcan and Blum, resulting in a semi-supervised version of the algorithm. The approach illustrates a general strategy for incorporating additional sources of domain knowledge in a frequentist approach to analyzing and designing learning algorithms.