Abstract
Ultrasound (US) skill acquisition requires repeated targeted practice and timely formative feedback, conditions often constrained in traditional teaching. This initiative harnesses artificial intelligence (AI) to transform pedagogy by providing scalable, automated feedback mechanisms that enhance student engagement and skill acquisition in renal US. To achieve this, we developed a fully automated framework for grading renal US images by fine-tuning deep learning (DL) models into a unified workflow delivering on-demand feedback on students’ US acquisition skill.
The system integrated two-stage DL based pipeline, each stage addressed distinct task in the grading process. In the first stage, we classified images into three categories: optimal, sub-optimal, and wrong organ. This was achieved by fine-tuning EfficientNet B3. The second stage focused on further sub-classification of sub-optimal cases. To achieve this, we adapted ResNet-50 as multi-label classifier that identified sub-optimal images as incorrect gain, incorrect position and/or artifact. Both models were trained with rigorous hyperparameter optimization using 10-fold grid searched cross-validation, which improved generalization and minimized overfitting.
A total of 2807 US images, featuring 562 optimal, 1288 sub-optimal, and 957 wrong images were utilized for developing the framework. The proposed model achieved the following classification performance metrics: area under the curve (0.973) and accuracy (0.931), sensitivity (0.911) and specificity (0.892) on the test set. For evaluation, 231 students across six groups submitted 786 renal US images. Student’s feedback was collected through 5-point scale based questionnaire (42.4% response rate): 32–48% (Score 4 or 5) found the AI-led feedback beneficial for confidence, acquisition and presentation, while over 49% (Score 4 or 5) endorsed its usability and engagement. Two representative groups were further invited for focussed group interviews. These discussions emphasized challenges such as device access and limited bedside tutor guidance, yet highlighted the value of structured AI-based feedback and strongly advocated for real-time integration. Excellent performance was observed in the objective structured clinical examination (OSCE) (mean 9.7/10) after piloting the initiative.
This platform operationalizes AI not merely as a feedback tool but as an educational catalyst; empowering students with on-demand automated feedback that supports self-directed iterative improvement and reduces dependency on faculty interaction that is intermittent. The system has potential in enhancing learning engagement, reinforcing optimal scanning skills, and fostering diagnostic confidence. There could be an opportunity for this AI application to be adopted by ultrasound education and potential implementation in OSCE.
Presenter(s) / Author(s)
