Evaluating the Gemma 3 (1B) Lightweight Language Model on Thai Elementary Science Exams

Google’s recent release of the Gemma 3 (1B) lightweight language model-promoted as one of the most capable open LLMs for resource-limited environments—raises important questions about its practical use in education. This study evaluates the model’s performance on Thai Grade 6 O-NET science questions using a structured content analysis framework. Thirty multiple-choice questions, randomly sampled from national exams across five academic years, were input into the model and its responses were evaluated across five scientific domains. Using a directed content analysis approach, five experienced primary school science teachers independently rated each AIgenerated response based on three dimensions: content accuracy, scientific reasoning, and clarity of explanation. While the model performed well in descriptive domains such as geology, it struggled in abstract areas like physics and biology. The findings highlight key limitations of lightweight LLMs in science education and underscore the importance of critical oversight when deploying them in instructional settings.