SwinFishNet: A Swin Transformer-based approach for automatic fish species classification using transfer learning

Abstract

The fish market is a crucial industry for both domestic economies and the global seafood trade. Accurate fish species classification (FSC) plays a significant role in ensuring sustainability, improving food safety, and optimizing market efficiency. This study introduces automatic FSC using Swin Transformer (ST) through transfer learning (SwinFishNet), which proposes an innovative approach to FSC by leveraging the ST model, a cutting-edge architecture known for its exceptional performance in computer vision tasks. The ST’s unique ability to capture both local and global features through its hierarchical structure enhances its effectiveness in complex image classification tasks. The model utilizes three distinct datasets: the 12-class BD-Freshwater-Fish dataset, the 10-class SmallFishBD dataset, and the 20-class FishSpecies dataset, focusing on image processing-based classification. Images were preprocessed by resizing to 224 224 pixels, normalizing, and converting to tensor format for compatibility with deep learning models. Transfer learning was applied using the ST, which was fine-tuned on these datasets and optimized with the AdamW algorithm. The model’s performance was evaluated using classification accuracy (CA), F1-score, recall, precision, Matthews correlation coefficient, Cohen’s kappa and confusion matrix metrics. The results yielded promising CAs: 0.9847 for BD-Freshwater-Fish, 0.9964 for SmallFishBD, and 0.9932 for the FishSpecies dataset. These results underscore the potential of the SwinFishNet in automating FSC and demonstrate its significant contributions to improving sustainability, market efficiency, and food safety in the seafood industry. This work offers a novel methodology with broad applications in both commercial and research settings, advancing the role of artificial intelligence in the fish market.