Correcting Recognition Errors in Japanese Handwritten Answers for Zero-Shot Automated Scoring with Large Language Models

This paper proposes a method for correcting character-recognition errors in Japanese handwritten answers to writing-type questions from exercise books for automated scoring. We created a model to correct character-recognition errors by fine-tuning the Text-to-Text Transfer Transformer using pairs of automatically recognized data from handwritten answers and their manual corrections. The data comprised handwritten Japanese answers from 185 junior high school students to writing-type questions in a Japanese language task. In addition, we augmented the training data using the five best results of the character-recognition model with confidence scores to learn additional patterns of recognition errors. The experimental results revealed that the answers corrected by the proposed method were closer to the actual answers than those before the correction and data augmentation was effective for the correction model. We further conducted an empirical study on how error correction in Japanese handwritten answer recognition affects automated grading accuracy of a zero-shot automated grading system using GPT-4o and GPT-4o-mini. Experimental results showed that error correction improved grading accuracy by up to 13 percentage points. Moreover, a detailed analysis of grading cases elucidated how error correction influences grading outcomes. These findings underscore the potential of error correction methods to enhance the accuracy of automated grading for descriptive answers.