A 3-Band Iteration Method to Transfer Knowledge Learned in RGB Pretrained Models to Hyperspectral Domain
Lei Wang
and
Sailing He
We propose a 3-band iteration method to transfer knowledge learned from RGB (red, green and blue) data pretrained models to the hyperspectral domain. We demonstrate classification of a Multi-spectral Choledoch database for cholangiocarcinoma diagnosis. The results show quicker and more stable training progress: 92%+ top-1 accuracy in the initial 3 epochs. Some advanced training techniques in the RGB computer vision field can be easily utilized and transferred to the hyperspectral domain without adding more parameters to the original architecture. The computational cost and hardware requirements remain the same. After voting, the highest top-1 accuracy on the validation set reached 95.4%, and the highest top-1 accuracy on the test set reached 94.3%. We can directly use our models trained on high-dimensional spectral images to test and infer on RGB color images. We visualized some results by Grad-CAM (Gradient-weighted Class Activation Mapping) on RGB test data, and it shows the transferability of knowledge. We trained the models solely on classification task on spectral data, and these models showed their ability to predict on RGB images with different fields of views. The results indicate good segmentation even when the model has never been trained on any segmentation task.