Integrating and Evaluating GenAI in Scientific Writing and Coursework

The increased use of Generative AI (GenAI) in scientific publications and within the classroom raises questions about its impact on the quality of the science and written outputs. Some journals now require declarations regarding the use of GenAI in publications. In educational settings, if students also declare their use or lack of use of Gen AI, will lab reports/assessments score better or worse? Does the extent to which they use GenAI affect the quality of their reports as judged by teachers? Specifically, we aim to: 1) Quantify the proportion of students using GenAI: We hypothesize that approximately 50% of students report using GenAI in their lab reports, based on perceived usage in previous classes. 2) Assess the relationship between GenAI use and lab report scores: We will evaluate whether students who declare the use of GenAI perform better or worse than students who do not. We hypothesize that scores will average out due to students’ varying abilities in their effectiveness of using GenAI. 3) Determine the extent of GenAI use and report scores: We will examine how the extent of GenAI use is associated with perceived quality of assignments. We think students will declare variable levels of GenAI usage, and we will quantify the intensity of GenAI usage, from minimal to intensive, in a five-point scale, which 1 represents writing assistance such as grammar and spelling checks, and 5 represents direct use of information generated by AI, as reported. 4) Examine the relationship between GenAI usage and written exam performance: Overall, we hypothesize that students who declare the use of GenAI will score similarly in their exams. At a finer scale, we hypothesize that students who used GenAI intensively will score worse in written exams. We will collect data from student papers (Lab reports and essays) and exams from three courses, ENVS2002 Environmental Data Analysis; BIOL3303 Conservation Biology; and ENVS3401 Understanding Tropical Ecosystems in a Changing World, which the PI (Prof. Timothy Bonebrake) is an instructor.. By comparing GenAI declarations (whether GenAI was used, and if yes, to what extent) and grades in student papers and exams, we will fill a key knowledge gap pertaining to student learning, GenAI usage, and GenAI declaration. All procedures will be carried out with HREC approval and participant informed consent. The application for HREC approval is currently under review.