Optimizing Ensemble Performance with Condorcet Voting: A Study on Weak Learners for Image Classification

Introduction:

Convolutional neural networks (CNNs) are a primary tool for image classification. This study proposes a novel approach to enhance ensemble learning by modifying the voting rule for aggregating results from individual classifiers. Typically, simple and weighted majority rules are used, but recent literature suggests other rules may be more efficient for specific tasks, particularly in multi-classification. This project tests the Condorcet voting rule for image classification.

Methods:

Ensemble learning combines predictions from multiple classifiers into a single result based on voting. Traditional voting rules often limit the potential of these ensembles, especially with weak learners whose accuracy is below 50% in multi-classification tasks. By exploring the Condorcet voting rule, this study aims to improve accuracy without domain-specific knowledge, efficiently balancing the classifiers' achievements and weights. This approach may benefit weak learners, which consume less energy and achieve peak performance faster than traditional methods.

Classical networks such as VGG, ResNet, EfficientNet, and other CNNs were employed. Condorcet rule was compared against traditional simple and weighted majority rules. The CIFAR-100 dataset was used for a balanced and comprehensive evaluation of the models' performance. The models were limited to 35 layers, with average accuracy across individual models being no more than 28% (random guessing yields 1% accuracy in a 100-class setup).

Results:

The ensemble model using the Condorcet rule showed a minimum of 4% accuracy improvement compared to simple and weighted majority rules, and over 15% improvement relative to the average accuracy of individual models.

Conclusions:

This study suggests that alternative voting rules, such as the Condorcet, can improve the performance of ensemble in image classification without domain-specific knowledge, and without altering the energy spent or training time of individual classifiers. Further study on even weaker learners to optimize the balance between energy consumption and accuracy is promising for the field.