Enhancing IELTS writing automated scoring with M-LoRA fine-tuned LLAMA-3 and human feedback-driven PPO reinforcement learning

This paper proposes an innovative automated essay scoring(AES) and feedback generation method based on the LLaMA-3 model and Multi-task LoRA (M-LoRA) fine-tune technology, aimed at improving the accuracy of IELTS essay scoring and the quality of personalized feedback generation. Our approach consists of three key stages: multi-task supervised fine-tuning of LLaMA-3 model, designing of the reward model, and reinforcement learning model training based on fine-grained human feedback. Before the experiment, we collected a private dataset of 5,088 IELTS essays with expert-annotated feedback and used this dataset to train and fine-tune the entire model. Firstly, through multi-task supervised fine-tuning, we successfully captured features efficiently across the four key dimensions of IELTS essay scoring: Task Response, Coherence and Cohesion, Lexical Resource, and Grammatical Range and Accuracy, effectively addressing the issue of catastrophic forgetting in scoring tasks. Secondly, we designed and trained a reward model to optimize the ability to generate feedback by scoring the quality of the feedback. Finally, we further fine-tuned the generated feedback using a reinforcement learning strategy model based on fine-grained human feedback, making the feedback more refined and personalized. Our findings demonstrate significant improvements in both essay scoring and feedback generation, showcasing practical applications in real-world educational settings. This research highlights the limitations of current large language models in grasping the complexities of essay scoring, emphasizing the need for more effective methods like ours to advance this field.