Factors Impacting Performance of Multiclass Image Classification: A Study Using Mars Images

In image classification problems, such as classifying martian images into classes with science and engineering interests, we observe that the performance of a pre-trained convolutional neural network (CNN) model such as ResNet50 deteriorates when moving from a binary classification problem to a multiclass classification problem. Through experimentation, we identify the factors affecting the performance of multiclass classification problems: number of classes, sample size, and the degree of class imbalance in the dataset measured using Shannon Equitability Index. We also derive conditions under which the balanced accuracy metric could exceed the accuracy metric in a binary classification problem.