Exploring generative AI application in general chemistry: A lab-based study of prompt engineering, AI assisted analysis, and critical evaluation of AI generated outputs

This study investigates the integration of generative artificial intelligence (Gen AI) into a General Chemistry II laboratory at St. Bonaventure University to assess its impact on student learning, inquiry, and chemical reasoning. In a redesigned of a common thermodynamics/equilibrium experiment involving cobalt(II) chloride, students used ChatGPT and a retrieval-augmented Gen AI (BonnieChemBot) to assist in calculating equilibrium constants, thermodynamic values, and answering open-ended chemical questions. The lab targeted four key educational goals: (A) mastery of chemical concepts, (B) development of prompt engineering strategies, (C) facilitation of inquiry-driven dialogue with instructors, and (D) critical evaluation of Gen AI outputs by students using chemical intuition. Students attempted all analyses twice: first independently, then with the assistance of Gen AI. Finally, they were asked to reflect on their trust in the Gen AI solutions and generate final answers. Instructor intervention in post-lab analyses was limited to cases where students had clearly attempted the first two steps in this process. While students were encouraged to practice prompt engineering, no grades were assigned to its execution. Instead, grading focused solely on students’ final answers.