Pre/post-testing is a common paradigm in learning studies, but it measures only what students know before and after engaging with a learning resource. Suppose instead we could monitor students moving from initial engagement with a learning objective (LO) through a series of supported practice opportunities. This type of data is gathered by the Open Learning Initiative (OLI) General Chemistry courseware.
OLI combines short didactic instruction with formative (learn-by-doing fading to did-I-get-this) and medium-stakes summative assessments (online quizzes). The detailed logs of student interactions with the online materials are combined with coarser data from in-person engagements to study learning in hybrid courses. Here we present preliminary findings from the analysis of this rich dataset (five million interactions from hundreds of students at institutions ranging from community colleges to R1 institutions). In particular, we analyze learning curves (error rate versus opportunities to engage with a LO) to extract both the initial error rate (level of preparation for the course) and the learning rate (the rate constant in a 1st-order, exponential decay, model of error rate versus opportunity). Our preliminary results:
○ Support the use of flipped instruction. The data indicate use of OLI courseware before class can bring students, with different levels of preparation, to a nearly equivalent state before class.
○ Suggest the learning rate is not a function of initial performance. The data indicate that although less well-prepared students have higher initial error rates, this does not impact their learning rates. Although more engagements with a learning objective are required to catch up, remedial work may not be necessary to close the gap.
○ Suggest that LOs with more quantitative reasoning have lower initial performance, but the learning rate for such objectives are is at least as fast as for less quantitative LOs. This argues that the hints and feedback OLI provides for quantitative reasoning are effective at supporting student learning and that such just-in-time support for math is a viable alternative to remedial math instruction.
○ Suggest that the patterns of engagement with the courseware in the first four weeks of the course are a stronger predictor of course success than the initial error rate. This may imply that demands on student time outside of the course play at least as important a role as initial preparation on student completion rates.

Sandra Raysor

Appearances