Reaching New Heights in EuroSAT Land Cover Classification with an Optimized Swin Transformer

Remote sensing plays a vital role in understanding and managing our planet, and accurate, timely land cover classification from satellite imagery is key to many applications. However, the complexity of satellite data, with its high intra-class variability and subtle interclass differences, makes this a challenging task. While Convolutional Neural Networks (CNNs) have traditionally been used, their ability to capture long-range dependencies is limited. In this work, we explore the potential of Vision Transformers (ViTs) for this task, ultimately focusing on an optimized Swin Transformer model. We began by systematically comparing several ViT architectures and found the Swin Transformer to be the most promising for the EuroSAT dataset. Our approach combines the Swin Transformer’s strengths, its hierarchical design and shifted-window self-attention, with a suite of carefully chosen optimization methods. These included techniques like Stochastic Weight Averaging, Cosine Annealing Warm Restarts, and various data augmentation strategies. Our model achieved state-of-the-art results with a classification accuracy of 99.19% and a weighted F1-score of 99.18% on the EuroSAT benchmark. These results demonstrate a significant improvement over the base Swin Transformer model, highlighting the effectiveness of our targeted optimization strategy. We believe this work provides a strong foundation for further development of ViT-based models in remote sensing.