Deep Hybrid Attention Framework Combining CNN and Vision Transformers for Food Category Prediction

With the rapid growth of digital food applications and image-based dietary monitoring systems, automatic food classification has become an essential task in computer vision. This paper presents FoodVision-Fusion, a novel hybrid framework for intelligent food classification that combines Convolutional Neural Networks (CNN) and Vision Transformers (ViT) to achieve robust and fine-grained image recognition. Initially, a baseline CNN model is implemented to establish foundational performance on the food image dataset. To enhance feature representation, an EfficientNet-B0 model is employed as a transfer learning reference, leveraging pre-trained weights for extracting deep spatial features. Building upon this, the Vision Transformer (ViT) captures global contextual dependencies within the images, enabling superior discriminative learning. Finally, the proposed CNN–ViT hybrid model integrates convolutional local features and transformer-based global attention through an attention-fusion layer for optimal prediction accuracy. Experimental results demonstrate that the hybrid framework significantly outperforms individual models, achieving improved precision, recall, and robustness against image variations. The findings highlight the framework’s potential for real-world applications in dietary monitoring, nutritional analysis, and automated food recognition systems.