Can ChatGPT Help Me Grade Reflective Diaries?

Students are often asked to write reflective diaries to connect their perspectives and their personal experiences to concepts and knowledge learnt in class. However, grading reflective diaries in large classes is a time-consuming chore for teachers. A teacher in a Humanities and Digital Technologies course demonstrates the integration of LLM to grade reflective diaries according to the users’ prompt approaches. An LLM prompt was built to assess and assign a “number grade” to students’ reflective diaries using the CRAFT method (Context-Role-Action-Format-Target Audience) as a rubric. A 0-shot prompt is used on OpenAI’s o3-mini to create consistent analyses, trained with lecture materials transcribed with WhisperAI. The result shows a grade discrepancy between the teachers and the LLM’s grades to be <0.5, which is below the teachers margin of subjectivity. The grading experience shows potential benefits of using LLM to measure students' performance, the necessity of being explicit in prompts and criteria, and the need of taking ethical considerations into account when integrating LLM assistance to assessment.