This spring I taught my MSc module ‘PSY6316 Current Issues in Cognitive Neuroscience’ on the topic “How reliable is cognitive neuroscience?”. Here’s the module outline:
What has been called The Replication Crisis has sparked widespread introspection about the standards and protocols of science, particularly within the behavioural sciences. This course, though reading a series of landmark papers and class discussion, will consider to extent to which doubts about the reliability of empirical work affect cognitive neuroscience. Can we trust the published papers in this field? Are the effects which we investigate reliable? If not, how can work in cognitive neuroscience be made more trustworthy?
The basic idea was to read material on robust science and scandals of unreliablity in psychology, and ask the students to consider the extent these applied to cognitive neuroscience.
If asked students before they took the course, and after, a set of questions by anonymous questionnaire. The responses indicate that the course did at least induce some skepticism in the students:
Here are their before-vs-after responses to the statement ‘If you read about a finding that has been demonstrated across multiple papers in multiple journals by multiple authors, how likely do you think that finding is to be reliable?’
Here the responses for ”If PSYCHOLOGY continues as it has, significant progress will be made in understanding in the next 50 years’ and ‘If COGNITIVE NEUROSCIENCE continues as it has, significant progress will be made in understanding in the next 50 years’.
The full list of questions I asked, the responses and the plots are available here. Most importantly maybe, the reading list is also available, which contains landmark papers on replicability/reproducibility in psychology, as well as relevant readings concerning reliability in neuroimaging.
I have always run this course as a discussion class rather than lecture class, and I have always based it around controversies in cognitive neuroscience. Last year it was ‘sex differences in the brain‘. You can read a bit more about the thinking behind the course in
Stafford, T. (2008). A fire to be lighted: a case-study in enquiry based learning. Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education, 3(1), 20-42.
Related: materials from the “Open Science and Robust Research Practices” symposium held in Sheffield on 7/6/18.