Lecture 1 Model Answer
Karpicke and Roediger (2008) suggest that current teaching practice assumes that tests are only useful to assess knowledge and study guides advise students to drop material once it has been learnt, despite evidence that testing may also be instrumental in learning. They therefore set out to assess the role of repeated testing compared to repeated encoding in the retention of new information . The theory on which this is based appears to be the proposal that “a standard assumption in nearly all research” (p.966) is that strength of memory is improved via encoding. However, no references were provided for this theory.
University students were trained to associate 40 Swahili-English word pairs (e.g. mashua-boat) , using cycles of repeated study and testing. Each study period was followed by a 30 second distraction task involving multiplication problems. The test required participants to supply the English translation for the Swahili word (e.g., mashua - ?)and word recall accuracy was measured . Four different conditions compared (1) retaining all items in both the study and test phases (2) dropping learnt items from both study and test (i.e. those which had successfully been responded to on a previous test phase) (3) dropping successfully learnt items from further study only (4) dropping successfully learnt items from further test only.
Different groups of participants took part in each of the conditions and all participants were given a final test a week after learning .
After the word learning session participants were asked to predict how many items they would recall a week later.
If learning new vocabulary is improved by repeated exposure (rather than repeated testing) participants in condition 4 will show larger improvements compared to those in condition 3. If repeated testing improves learning the reverse will be found. Conditions 1 and 2 provide control conditions against which conditions 3 and 4 can be compared.
All four groups learnt the material at the same rate (Figure 1, although obviously those who dropped items took less time and were exposed to less trials).
All four groups predicted they would recall around 50% of the items after a week's delay. However, when memory for the English translation was tested the groups that dropped items from testing remembered around 35% of items, compared to around 80% for the groups which didn't drop items from testing . Keeping or dropping items from study had no effect (Figure 2).
CONCLUSIONS, IMPLICATIONS AND CRITICISMS
The authors conclude that "testing (and not studying) is the critical factor for promoting long-term recall". Further to this, they connected their results to a debate on how the rate of acquisition affects the rate of forgetting. Because participants in all groups had similar learning rates, but dramatically different forgetting rates, they conclude that other factors than rate of acquisition affect recall. Finally, they suggest that study guides which advocate dropping correctly recalled items from test as well as study periods are at odds with their findings. As were the participants own predictions of their success rate.
To critically assess these conclusions, we can conveniently break down the claims of the paper into four component claims.
(1). Testing is important for recall
The results are unambiguous - there was a large difference between participants who continued to test items, and those who dropped items from testing . The question remains about what, psychologically, was it about testing that produced this benefit. Obviously retesting involves recall from memory (retrieval), which is the explanation the authors favour, but it is also plausible that testing was more effortful and/or invokes more depth of processing for study items . Note that the difference cannot be explained by a difference in effort/fatigue between groups, since keeping or dropping items in study (which would also change the amount of time/effort) did not have any effect.
(2). Additional study is ineffective
This result contradicts common knowledge and associative/behaviourist theories of memory, which place repetition as central to learning . It is possible that with higher statistical power the difference between the group that dropped items from study as well as test and the group that dropped items from test but retained them for study would become significant . Furthermore, since there are so many reasons to believe in the importance of study for memory we might require replication of this effect with different stimuli, different learning-recall delays and/or different study methods .
It might be, for example, that the result would not generalise to situations where participants studied items with a greater degree of depth of processing (e.g. as with Hyde & Jenkins, 1973).
(3). No relation between learning and forgetting rate
The study shows that other factors can affect forgetting beyond rate of learning. However, because there were no significant differences between learning rate across groups it is not possible to assess the claim of whether changes in rate of learning effect rate of forgetting. The authors stated that they would address the question of whether speed of learning correlated with long term retention of information. It is not clear that they have addressed this question. In their discussion they claim that the forgetting rate is not necessarily determined by the speed of learning. Although they argue that speed of learning would be roughly equivalent for all conditions whereas rate of forgetting was clearly not equivalent, to make claims about speed of learning an experiment that kept learning conditions consistent but varied speed of learning would be required.
(4). Failure of metacognition
Participants were not able to predict how successfully they would recall the items they learnt. Additionally, there were not even any differences between the groups, showing that participants did not have strong insight into the value of the different learning methods.
One possibility is that the participants' predictions did not contain any information about their beliefs about the study methods . So, for example, they may have wanted to leave the experiment as quickly as possible and so tended to guess they would successfully recall 50% of the items using a simple "pick the middle value" heuristic . A stronger test of their (lack of) insight into the relative merit of the learning methods would be to ask them to rank the four methods .
Beyond these specific reservations, some general criticisms of the experiment may be made. The result is not established for non-verbal materials. A better test might be a more ecologically valid one involving real course-material and tested with a final exam.
The number of participants in the study is not given, nor are details of the statistical tests used (although this is excusable given the size of the group differences) . There is no reason to think that the effect would be particularly strong in students with respect to the rest of the population.
Hyde, T. S., & Jenkins, J. J. (1973). Recall for words as a function of semantic, graphic, and syntactic orienting tasks. Journal of Verbal Learning and Verbal Behavior , 12(5), 471-480
Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968.