Ensembles of classifiers by Evolutionary Algorithms
The effectiveness of a classifier ensemble depends on the selection of accurate and diverse classification models, that is, the committee should be composed by precise models which make different prediction mistakes.
Finding the optimal global solution is a difficult problem since the number of possible combinations of classifiers is very large, making the search space very broad.
Evolutionary Algorithms are metaheuristics that have been shown to be effective in NP-hard problems and one of their advantages over traditional search methods is they are less likely to get stuck in a local optimum; the main reason for that is due to the fact that they process a set of solutions (population), instead of just one solution.
Furthermore, there are diversity-guided Evolutionary Algorithms in which the selection of the individuals to compose the next generation is based on the similarity between the members of the population.
Given these characteristics, the use of Evolutionary Algorithms with diversity-enforcing heuristics is an appropriate approach to optimize the construction of ensembles.
Therefore, an Evolutionary Algorithm, which encourages the selection of diverse classifiers to form the committee, was developed in this project, it is called Diversity-based Classifier Ensemble (DCE).
However, running this Evolutionary Algorithm to create ensembles consumes a great amount of processing power and memory.
For that reason, a parallel Evolutionary Algorithm called Parallel Diversity-based Classifier Ensemble (PDCE) was also developed in this project using the global parallelization model to distribute the computational cost among multiple CPUs.
The preliminary results obtained from the computational experiments indicate that the DCE and PDCE algorithms can be useful tools for solving the classifier ensembles optimization problem when compared with exhaustive search methods.
Still, based on these results, some points of improvement and some aspects of the adopted approach will be further investigated in the continuation of this project.