Category: Research

New paper: “The path to learning: Action acquisition is impaired when visual reinforcement signals must first access cortex”

Using cunning experimental design we provide evidence which supports a new theory of how the brain learns new actions. Back in 2006, our professors Redgrave and Gurney proposed a new theory of how the brain learns new actions, centered around the subcortical brain area the basal ganglia and the function of the neurotransmitter dopamine. This was exciting for two reasons: it proposed a theory of what these parts of the brain might do, based on our understanding of the pathways involved and the computations they might support and because it was a theory that was in flat contradiction to the most popular theory of dopamine function, the reward prediction error hypothesis.

We set out to test this theory. We used a novel task to assess action-outcome learning, in which human subjects moved a joystick around until they could identify a target movement. We didn’t record the dopamine directly – a tall order for human subjects – but instead used our knowledge of what triggers dopamine to compare two learning conditions: one where dopamine would be triggered as normal, and one where we reasoned the dopamine signal would be weakened.

We did this by using two different kinds of reinforcement signals, either a simple luminance change (i.e. a white flash), or a specifically calibrated change in colour properties (visual psychophysics fans: a shift along the tritan line). The colour change signal is only visible to some of the cells in the eye, the s-cone photoreceptors. Importantly, for our purposes, this means that although the signal travels the cortical visual pathways it does not enter the subcortical visual pathway to the superior colliculus. And the colliculus is the main, if not only, route to trigger dopamine release in the basal ganglia.

So by manipulating the stimulus properties we can control the pathways the stimulus information travels. Either the reinforcement signal goes directly to the colliculus and so to the dopamine (luminance change condition), or the signal must travel through visual cortex first and then to the colliculus, ‘the long way round’, to get to the dopamine (s-cone condition).

The result is a validation for the action-learning hypothesis: when reinforcement signals are invisible to the colliculus learning new action-outcome associations is harder. We also did an important control experiment which showed that the impairment due to the s-cone signals couldn’t be matched by simple transport delay of the stimulus information; this suggests the s-cone signal is weaker, not just slower in terms of dopaminergic action. You can read the full thing here.

The results aren’t conclusive – no behavioural experiment which didn’t record dopamine directly could be – but we think it is a strong result. Popper said there are two kinds of results to be most interested in. One was the experiment which proved a theory wrong. The other – which we believe this is – is an experiment which confirms a bold hypothesis. There are no other theories which would suggest this experiment, and only the Redgrave and Gurney theory predicted the result we got before we got it. This makes it a startling validation for the theory and that is why we’re really proud of the paper.

This work was funded by our European project, im-clever, and all the difficult work was done by Martin Thirkettle, building on Tom Walton’s foundation.

Thirkettle, M., Walton, T., Shah, A., Gurney, K., Redgrave, P., & Stafford, T. (2013). The path to learning: Action acquisition is impaired when visual reinforcement signals must first access cortex. Behavioural Brain Research, 243, 267–272. doi:10.1016/j.bbr.2013.01.023

Written by Comments Off on New paper: “The path to learning: Action acquisition is impaired when visual reinforcement signals must first access cortex” Posted in Research

New paper: Memory Enhances the Mere Exposure Effect

This research used a novel testing strategy to overturn a long-standing claim in the literature. The mere exposure effect is the finding that simply experiencing something inclines you to like it. Obviously, back in the days of behaviourism this provided a marked contrast to reward-induced preferences. A landmark paper by Bob Zajonc showed that this effect could hold even if you weren’t aware of the original exposure. (Incidentally it was this paper, as far as I can tell, which reignited interest in subliminal perception after the topic had fallen into ‘hidden persuader’ ignominy).

For a long time, based partly on the influence of this seminal paper, it has been reported that explicit memory for stimuli will reduce the mere exposure effect. The logic is that explicit memory will allow people to use a deliberate discounting strategy (something along the lines of “I know I’ve seen that before, so maybe I just feel positive about it because I’ve seen it before”). This isn’t implausible, but does conflict with a large marketing literature which suggests that sustained engagement with marketing materials is more likely to lead to preference (and it is just such engagement with adverts which you would expect to be accompanied by explicit memory).

I put test stimuli in my PSY101 lectures, and then weeks later tested the students on their preferences for these stimuli and a matched group which they hadn’t seen. This allowed me to collect high number of participants for an experiment which had a high ecological validity (and still many elements of experimental control). Continue reading

Written by Comments Off on New paper: Memory Enhances the Mere Exposure Effect Posted in Research

Frontiers special issue on intrinstic motivation and open-ended development

Our special issue in Frontiers in Cognitive Science is now accepting submissions: Intrinsic motivations and open-ended development in animals, humans, and robots

This call stems from the EU FP7 project “IM-CLEVER”, programme of work that involved computer scientists, neuroscientists, psychologists and roboticist in developing robot controllers that can guide a robot to learn by exploring the world.

The special issue will gather together work related to this task. ‘Intrinsic motivations’ are those that guide exploration – things like curiosity, play or desire for mastery. The emphasis is on learning systems which are more than the simple stimulus-response or response-reward learning which has dominated learning theory for so long. ‘Open-ended development’ means learning that doesn’t have a goal or limit, but is instead designed to produce skills and abilities which can be build on to produce ever more complex skills and abilities. The call welcomes papers from experimental, theoretical and engineering perspectives. The full text of the call is here.

Written by Comments Off on Frontiers special issue on intrinstic motivation and open-ended development Posted in Research

Brain network: social media and the cognitive scientist

This just published in Trends in Cognitive Sciences. Abstract:

Cognitive scientists are increasingly using online social media, such as blogging and Twitter, to gather information and disseminate opinion, while linking to primary articles and data. Because of this, internet tools are driving a change in the scientific process, where communication is characterised by rapid scientific discussion, wider access to specialist debates, and increased cross-disciplinary interaction. This article serves as an introduction to and overview of this transformation.

Reference: Stafford, T., & Bell, V. (2012). Brain network: social media and the cognitive scientist. Trends in Cognitive Sciences, 16(10), 489–490. doi:10.1016/j.tics.2012.08.001

I’m on Twitter as @tomstafford, btw

Written by Comments Off on Brain network: social media and the cognitive scientist Posted in Research

Scholarly publications

(Google scholar profile, ORCID ID : 0000-0002-8089-9479)

Preprints / Under review

Stafford, T., Pirrone, A., Croucher, M., & Krystalli, A. (2019, May 22). Quantifying the benefits of using decision models with response time and accuracy data. https://doi.org/10.31234/osf.io/pn7g6. See also the interactive data explorer: sheffield-university.shinyapps.io/decision_power

Hampsey, E., Overton, P.G., Stafford, T. (under review). Microsaccade rate as a measure of drug response.. Open data/analysis scripts/etc: https://osf.io/je8hf/

Stafford, T., Holroyd, J., & Scaife, R. (under review). Confronting bias in judging: A framework for addressing psychological biases in decision making. https://doi.org/10.31234/osf.io/nzskm

Scaife, R., Stafford, T., Bunge, A., Holroyd, J. (under review). Backlash or Boon? The effects of moral interactions on implicit racial bias

2019 & in press

Bannard, C., Leriche, M., Bandmann, O., Brown, C., Ferracane, E., Sánchez-Ferro, A., Obeso, J., Redgrave, P. and Stafford, T. (in press). Reduced habit-driven errors in Parkinson’s Disease. Nature Scientific Reports. https://doi.org/10.31234/osf.io/b25tf

Thirkettle, M., Thyoka, M., Fernandes, N., Gopalan, P., Stafford, T. Offiah, A.C. (2019). Internet-based Measurement of Visual Assessment Skill of Trainee Radiologists: Developing a Sensitive Tool. British Journal of Radiology, 92(xxxx), 20180958.

Pirrone, A., Illin, J., Stafford, T., Milne, E. (in press). A diffusion model decomposition of orientation discrimination in children with Autism Spectrum Disorder. European Journal of Developmental Psychology.

Panagiotidi, M., Overton, P.G., Stafford, T. (in press). The relationship between ADHD traits and sensory sensitivity in the general population. Comprehensive Psychiatry

Holroyd, J., Scaife, R., Stafford, T. (in press). What is Implicit Bias? Philosophy Compass 12(10), e12437.

Kalfaoğlu, Ç., Stafford, T. and Milne, E. (in press). Frontal Theta Band Oscillations Predict Error Correction and Post Error Slowing in Typing. Journal of Experimental Psychology: Human Perception and Performance.

Holroyd, J., Scaife, R., Stafford, T. (in press). Responsibility for Implicit Bias. Philosophy Compass, 12(3), e12410.

2018

Silberzahn, R. et al (2018). Many analysts, one dataset: Making transparent how variations in analytical choices affect results. Advances in Methods and Practices in Psychological Science. See also : ‘Science isn’t broken

Stafford, T. (2018). Female chess players outperform expectations when playing men. Psychological Science, 29(3), 429-436. (Preprint).

Pirrone, A., Azab, H., Hayden, B. Y., Stafford, T., & Marshall, J. A. R. (2018). Evidence for the speed–value trade-off: Human and monkey decision making is magnitude sensitive. Decision, 5(2), 129-142.

2017

Stafford, T., Devlin, S., Sifa, R., & Drachen, A. (2017). Exploration and skill acquisition in a major online game. In Proceedings of The 39th Annual Meeting of the Cognitive Science Society.

Panagiotidi, M., Overton, P.G.,Stafford, T. (2017). Multisensory integration and ADHD-like traits: Evidence for an abnormal temporal integration window in ADHD. Acta Psychologica, 181, 10-17

Panagiotidi, M., Overton, P.G., Stafford, T. (2017). Co-occurrence of ASD and ADHD traits in the general population . Journal of Attentional Disorders

Panagiotidi, M., Overton, P.G., Stafford, T. (2017). Increased microsaccade rate in individuals with ADHD traits. Journal of Eye Movement Research. 10,1

Pirrone, A., Dickinson, A., Gomez, R., Stafford, T. and Milne, E. (2017). Understanding perceptual judgement in autism spectrum disorder using the drift diffusion model. Neuropsychology, 31 (2), 173-180

Pirrone, A., Marshall, J. A., & Stafford, T. (2017). A Drift Diffusion Model Account of the Semantic Congruity Effect in a Classification Paradigm. Journal of Numerical Cognition, 3(1), 77-96.

Stafford, T. & Haasnoot, E. (2017). Testing sleep consolidation in skill learning: a field study using an online game. Topics in Cognitive Science. 9(2), 485-496.(data + code)

Panagiotidi, M., Overton, P.G., Stafford, T. (2017). Attention Deficit Hyperactivity Disorder-like traits and distractibility in the visual periphery. Perception, 46 (6), 665-678

2016

Bednark J., Reynolds J., Stafford T., Redgrave P. and Franz E. (2016). Action experience and action discovery in medicated individuals with Parkinson’s disease. Frontiers in Human Neuroscience, 10, 427. DOI 10.3389/fnhum.2016.00427.

Bertram, C., & Stafford, T. (2016). Improving training for sensory augmentation using the science of expertise. Neuroscience & Biobehavioral Reviews, 68, 234-244.

Lu, Y., Stafford, T., & Fox, C. (2016). Maximum saliency bias in binocular fusion. Connection Science, 28(3),258-269.

2015

Thirkettle, M., Stafford, T., & Offiah, A. (2015). Internet Based Measurement of Visual Expertise in Radiological Skill. Perception, 44, 44-45.

2014

Stafford, T. & Dewar, M. (2014). Tracing the trajectory of skill learning with a very large sample of online game players. Psychological Science, 25(2) 511-518. See also: Data and analysis code.

Baldassarre, G., Stafford, T., Mirolli, M., Redgrave, P., Ryan, R. M., & Barto, A. (2014). Intrinsic motivations and open-ended development in animals, humans, and robots: an overview. Frontiers in Psychology, 5(985). doi: 10.3389/fpsyg.2014.00985 (introduction to Special Topic we edited).

Kalfaoğlu, Ç & Stafford, T. (2014). Performance breakdown effects dissociate from error detection effects in typing. The Quarterly Journal of Experimental Psychology, 67(3), 508-524.

Stafford, T. (2014). The perspectival shift: how experiments on unconscious processing don’t justify the claims made for them. Frontiers in Psychology, 5, 1067. doi:10.3389/fpsyg.2014.01067

Stafford, T., Elgueta, H., Cameron, H. (2014). Students’ engagement with a collaborative wiki tool predicts enhanced written exam performance. Research in Learning Technology, 22, 22797. doi:10.3402/rlt.v22.22797

Pirrone, A., Stafford, T., & Marshall, J. A. R. (2014). When natural selection should optimise speed-accuracy trade-offs. Frontiers in Neuroscience, 8(73). doi: 10.3389/fnins.2014.00073

Redgrave, P., Vautrelle, N., & Stafford, T. (2014). Interpretive conundrums when practice doesn’t always make perfect. Movement Disorders: Official Journal of the Movement Disorder Society 29(1), 7-10. doi:10.1002/mds.25726

 

2013

Bertram, C., Evans, M. H., Javaid, M., Stafford, T., & Prescott, T. (2013). Sensory augmentation with distal touch: the tactile helmet project. In Biomimetic and Biohybrid Systems (pp. 24-35). Springer Berlin Heidelberg.

Thirkettle, M., Walton, T., Redgrave, P., Gurney, K., & Stafford, T. (2013). No learning where to go without first knowing where you’re coming from: action discovery is trajectory, not endpoint based. Frontiers in Cognitive Science, 4:, 638. doi:10.3389/fpsyg.2013.00638

Walton, T., Thirkettle, M., Redgrave, P., Gurney, K. N., & Stafford, T. (2013). The Discovery of Novel Actions Is Affected by Very Brief Reinforcement Delays and Reinforcement Modality. Journal of Motor Behavior, 45(4), 351-360.

Bednark, J. G., Reynolds, J. N. J., Stafford, T., Redgrave, P., & Franz, E. A. (2013). Creating a movement heuristic for voluntary action: Electrophysiological correlates of movement-outcome learning. Cortex, 49(3), 771-780. doi:10.1016/j.cortex.2011.12.005

Thirkettle, M., Walton, T., Shah, A., Gurney, K., Redgrave, P., & Stafford, T. (2013). The path to learning: Action acquisition is impaired when visual reinforcement signals must first access cortex. Behavioural Brain Research, 243, 267–272. doi:10.1016/j.bbr.2013.01.023

2012

Stafford, T., Thirkettle, M., Walton, T., Vautrelle, N., Hetherington, L., Port, M., Gurney, K.N., Redgrave, P. (2012), A Novel Task for the Investigation of Action Acquisition, PLoS One, 7(6), e37749.

Stafford, T. & Grimes, A. (2012). Memory enhances the mere exposure effect. Psychology & Marketing, 29, 12, 995-1003.

Stafford, T., & Bell, V. (2012). Brain network: social media and the cognitive scientist. Trends in Cognitive Sciences, 16(10), 489–490. doi:10.1016/j.tics.2012.08.001

2011

Stafford, T., & Gurney, K. N. (2011). Additive Factors Do Not Imply Discrete Processing Stages: A Worked Example Using Models of the Stroop Task. Frontiers in Psychology, 2. doi:10.3389/fpsyg.2011.00287

Stafford, T., Ingram, L. and Gurney, K.N. (2011), Pieron’s Law holds during Stroop conflict: insights into the architecture of decision making, Cognitive Science 35, 1553–1566.

Yates, D.J. and Stafford, T. (2011), Insights into the function and mechanism of saccadic decision making from targets scaled by an estimate of the cortical magnification factor, Cognitive Computation, 3,89-93.

earlier key publications

Stafford, T. (2010). How do we use computational models of cognitive processes? In Connectionist Models Of Neurocognition And Emergent Behavior: From Theory to Applications. Proceedings of the 12th Neural Computation and Psychology Workshop, Birkbeck, University of London, 8-10 April 2010. World Scientific (pp. 326-342).

Stafford, T. (2009), What use are computational models of cognitive processes? In Mayor, J., Ruh, N.,  Plunkett, K. Connectionist Models of Behaviour and Cognition II: Proceedings of the 11th Neural Computation and Psychology Workshop. World Scientific

Eiser, J. R., Stafford, T., Henneberry, J., & Catney, P. (2009). “Trust me, I’m a Scientist (Not a Developer)”: Perceived Expertise and Motives as Predictors of Trust in Assessment of Risk from Contaminated Land. Risk Analysis, 29(2), 288-297.

Stafford, T. & Gurney, K.N. (2007), Biologically constrained action selection improves cognitive control in a model of the Stroop task, Philosophical Transactions of the Royal Society B: Biological Sciences, 362 (1485), 1671-1684.

Stafford, T. & Wilson, S. P. (2007), Self-organisation can generate the discontinuities in the somatosensory map, Neurocomputing, 70(10-12), 1932-1937.

Stafford, T. & Gurney, K. N. (2004), The role of response mechanisms in determining reaction time performance: Pieron’s Law revisited, Psychonomic Bulletin & Review, 11:975-987.

Eiser, J. R., Fazio, R. H., Stafford, T. & Prescott, T. J. (2003), Connectionist simulation of attitude learning: Asymmetries in the acquisition of positive and negative evaluations, Personality and Social Psychology Bulletin, 29:1221-1235.https://psyarxiv.com/nzskm

 

Link to: full list of journal publications

Invited Talks

Beyond Reinforcement Learning In Action Acquisition‘, 9 November 2011, Department of Psychological Sciences, Birkbeck University of London.

‘Infering cognitive architectures from high-resolution behavioural data’, 13 May 2011, York Centre for Complex Systems Analysis, University of York.

‘A novel task for the investigation of action learning’, 7 July 2010, Experimental Psychology Society, Manchester, 7-9 July 2010

‘An empirical test of some philosophies of science’, 21 May 2010, Heng Seng Centre for Cognitive Studies, University of Sheffield

‘How do we use computational models of cognitive processes?’, Neural Computation and Psychology Workshop, 8-10 April 2010, Birkbeck, London

– ‘What use are computational models of cognitive processes?‘, 19th of March 2010, Redwood Centre for Theoretical Neuroscience, UC Berkeley

– ‘Using the psychophysics of choice behaviours to infer mental structure from reaction times‘, 15th of January 2010, Department of Psychology, National University of Ireland, Galway

– ‘Pieron’s Law holds in conditions of response conflict’, 1st August, presented at the 31th Annual Conference of the Cognitive Science Society. Amsterdam, The Netherlands. Powerpoint here

– ‘The Nonconscious Mere Exposure Effect with Brand Logos: Real but Elusive’, Department of Psychology, City College, Thessaloniki, 1/6/09

– ‘Email: the technology and psychology of continuous partial attention’, UFI, Sheffield, 12/11/08

– “‘Things can be known’ : Teaching psychology through demonstrations” Keynote at 26th Annual Conference of the Association for the Teaching of Psychology, University of Lincoln, 11th of July, 2008

– ‘How to make students talk in seminars’, HEA Psychology ‘Postgraduates who Teach’ Network, University of Birmingham, 27 May 2008

– ‘The psychological foundations of privacy’, Privacy in Law, Ethics and Genetic Data, 1st international workshop PRIVILEGED Project (EC FP6), 10 Jan 2008

– ‘Debates in Cognitive Neuroscience’, Center for Inquiry Based Learning in the Arts and Social Sciences, University of Sheffield, 30 May 2007

‘Is there a science of advertising?’, University of Hull, Management School, 27th of April 2007

– ‘Residents’ Perception of Risk on Contaminated Sites’, Environment Agency training day, Leeds, 20th of June, 2007

 

Conference Papers & Posters

Grimes A, Stafford T & Roper S (2016), Ambient Rubbish: Examining the attitudinal impact of incidental exposure to brand litter, Academy of Marketing’s 11th Global Brand Conference, Bradford University, UK

“Linking total movement history to action learning”. Stafford & Thirkettle. Poster presented at Reinforcement Learning and Decision Making 2013, 24th-26th October, Princeton, USA

Testing theories of skill learning using a very large sample of online game players. Stafford & Dewar. Talk at the 35th Annual Meeting of the Cognitive Science Society, Berlin, Germany, July 31 -August 3, 2013

“The effect of acquisition of an internal forward model on an exploration task”. M. Dagioglou, J. M.Bugella, T. Walton, T. Stafford, P. Redgrave, R.C. Miall. Poster presented at the 22nd Annual Meeting of the Society for the Neural Control of Movement, April 23-29th 2012, Venice, Italy

Typing errors lead to increase in power and synchronization of theta oscillations“. Cigir Kalfaoglu, Tom Stafford and Elizabeth Milne. Poster presentation (by Kalfaoglu) in the British Association of Cognitive Neuroscience in Newcastle, UK on Wednesday the 11th of April, 2012.

“Instruction to relax enhances visual search performance by altering eye movements” David Yates and Tom Stafford. Oral Presentation (by Yates) at 34th European Conference on Visual Perception which will be held in Toulouse, France from Sunday 28th August to Thursday 1st September, 2011.

Visual search performance can be enhanced by instructions that alter eye movements” David Yates and Tom Stafford. Poster Presentation (by Yates) at 16th European Conference on Eye Movements which will be held in Marseille, France from Sunday 21st to Thursday 25th August, 2011.

“Learning the long way round: Action learning based on visual signals unavailable to the superior colliculus is impaired.” Martin Thirkettle, Tom Walton, Kevin Gurney, Peter Redgrave and Tom Stafford. Oral presentation (by Thirkettle) at 34th European Conference on Visual Perception which will be held in Toulouse, France from Sunday 28th August to Thursday 1st September 2011.

“What mistakes reveal about skilled performance: A study of touch-typing”. Cigir Kalfaoglu, Tom Stafford and Elizabeth Milne. Poster presentation (by Kalfaoglu) in the Experimental Psychologists Society Meeting in Oxford, UK from Wednesday the 13th to Friday the 15th of April, 2011.

Stafford, T., Javaid, M, Mitchinson, B., Galloway, A.M.J., Prescott, T.J. (2011). Integrating Augmented Senses into Active Perception: a framework. Poster presented at Royal Society meeting on Active Touch Sensing at the Kavli Royal Society International Centre, 31 January – 02 February, 2011

J.G. Bednark, E.A. Franz, T. Stafford, P. Redgrave, J.N.J. Reynolds. Tracking the learning of actions: An evaluation of the frontal P3a component. Society for Neuroscience Annual Meeting, 17-21 October 2009, Chicago.

Yates, D.J.; Stafford, T. Saccadic latency versus eccentricity for targets scaled by an estimate of the cortical magnification factor, 15th European Conference on Eye Movements, Southampton, 23-27th of August, 2009

Stafford, T; Grimes, A., Perkins, C. The Nonconscious Mere Exposure Effect with Brand Logos: Real but Elusive, The 21st Annual Convention of the Association for Psychological Science, 22nd-24th of May 2009, San Francisco, CA.

Stafford, T. What use are computational models of cognitive processes? 11th Neural Computation and Psychology Workshop, Oxford, 16-18 July, 2008.

Stafford, T. & Wilson, S.P. Self-organisation explains discontinuities in the somatosensory map. Fifteenth Annual Computational Neuroscience Meeting CNS*2006, July 16 – July 20, 2006.

Tom Stafford, Mark D. Humphries, Jonathan M. Chambers. The neural circuitry necessary for decision making by evidence accumulation. Poster presented at the Computational Cognitive Neuroscience conference in Washington, DC, 11/10/2005.

Stafford, T. & Gurney, K. (2005). The basal ganglia as the selection mechanism in a cognitive task. In J. J. Bryson, T. J. Prescott & A. Seth (Eds.) Modeling Natural Action Selection (pp. 77-83). AISB Press.

Stafford, T. & Gurney, K. (2005). The basal ganglia as the selection mechanism in a cognitive task. Poster presented at Modelling Natural Action Selection workshop in Edinburgh, July 2005.

Eiser, Richard, Stafford, Tom, Shook, Natalie & Fazio, Russell. Learning under uncertainty: Manipulating and simulating the role of expectations. Paper presented at the 14th General Meeting of the European Association fo Experimental Social Psychology, Wurzburg, 19-23 July 2005.

Catney, P., Lawson, N., Palaseanu-Lovejoy, M., Shaw, S., Smith, C., Stafford, T., Talbot, S., Hao, X. (2005, 1st of March 2005). Acid tar lagoons: risks and sustainable remediation in an urban context. Paper presented at the SUBR:IM conference, Natural History Museum, London.

Stafford, T. & Gurney, K.N. The Basal Ganglia provides an appropriate model for response selection in the Stroop task. Poster presented at the Annual Conference of the BPS Cognitive Psychology Section at Essex, September 6th-8th, 2000.

Books & Book chapters

Stafford, T. (2011). How do we use computational models of cognitive processes? In E. Davelaar (eds) Connectionist Models Of Neurocognition And Emergent Behavior: Proceedings of the 12th Neural Computation and Psychology Workshop(pp 326-342). World Scientific.

Stafford, T. (2010). The Narrative Escape, 40kbooks, Milan.

Moore, G & Stafford, T. (2010). The Rough Guide Book of Brain Training.

Stafford, T. (2009) ‘Hacking our tools for thought’ in Nold, C. (ed) Emotional Cartography: Technologies of the Self, pp 88-96. emotionalcartography.net

Stafford, T. (2009). What use are computational models of cognitive processes? In J. Mayor, N. Ruh & K. Plunkett (eds.) Connectionist Models of Behavior and Cognition II: Proceedings of the 11th Neural Computation and Psychology Workshop(pp 265-274). World Scientific.

Catney, P., Eiser, J.R., Henneberry, J. & Stafford, T. (2007) ‘Democracy, Trust and Risk Related to Contaminated Sites in the UK’, in: T. Dixon, M. Raco, P. Catney & D.N. Lerner (Eds.) Sustainable Brownfield Regeneration: Liveable Places from Problem Spaces. Oxford: Blackwells.

Stafford, T. & Gurney, K.N. (2006). Computational Models of Cognition. In An Introduction to Cognitive Psychology: Processes and Disorders. Second Edition. Ed. David Groome. Hove, UK: Psychology Press.

Stafford, T. & Webb, M. (2004). Mind Hacks: Tips and Tricks for using your brain. Sebastapol, CA: O’Reilly.

 

Other publications

Stafford, T. (2009). Lessons from the campaign against Elsevier: “We won, but how did we win?”. ACME: An International E-Journal for Critical Geographies, 8(3), 494-504.

Stafford, T. (2008). Teaching Questions Rather than Answers: inquiry-based learning on an MSc course. HEA Psychology Network Newsletter, Issue 46, January 2008. Available at: http://www.psychology.heacademy.ac.uk/html/newsletter.asp

Stafford, T. & Martin, C.J. (2007). How to do a neuroscience lab class with 120 students. HEA Psychology Network Newsletter, Issue 45, November 2007. Available at: http://www.psychology.heacademy.ac.uk/html/newsletter.asp

Stafford, T. (2007). Isn’t it all just obvious? The Psychologist, 20,2,94-95

The neural circuitry necessary for decision making by evidence accumulation
Mark D. Humphries, Tom Stafford, Jonathan M. Chambers & Kevin N. Gurney
ABRG Technical Report number 5, May 2006. Department of Psychology, University of Sheffield, UK

Stafford, T. & Webb, M. (2006) ‘What Is a Wiki (and How to Use One for Your Projects)’. O’Reilly Network, 7 July 2006. Available at
http://www.oreillynet.com/pub/a/network/2006/07/07/what-is-a-wiki.html

Stafford, T. & Gurney, K. (2005). The basal ganglia as the selection mechanism in a cognitive task. In J. J. Bryson, T. J. Prescott & A. Seth (Eds.) Modeling Natural Action Selection (pp. 77-83). AISB Press.

Webb, M. & Stafford, T.(2004) ‘Paying Attention (or Not) to the Flickr Daily Zeitgeist’. O’Reilly Network, 6 December 2004. Available at
http://www.oreillynet.com/pub/a/network/2004/12/06/mndhcks_1.html

Stafford, T. (2003). Integrating psychological and neuroscientific constraints in models of Stroop processing and action selection PhD Thesis, University of Sheffield. (abstract and contents only)

Stafford, T. (2003). Psychology in the coffee shop. The Psychologist, 16(7), 358-359

Stafford, T. (2000). Stroop Interference: Methodological Problems and Contrary Data, Psycoloquy, 11, #110.

Fundamentals of learning: the exploration-exploitation trade-off

The exploration-exploitation trade-off is a fundamental dilemma whenever you learn about the world by trying things out. The dilemma is between choosing what you know and getting something close to what you expect (‘exploitation’) and choosing something you aren’t sure about and possibly learning more (‘exploration’). For example, suppose you are in a restaurant and you look at the menu:

  • Fish and Chips
  • Chole Poori
  • Paneer Uttappam
  • Khara Dosa

Assuming for the sake of example that you’re not very good with Sri Lankan food, you’ve now got a choice. You can ‘exploit’ – go with the fish and chips, which will probably be alright – or you can ‘explore’ – try something you haven’t had before and see what you get. Obviously which you decide to do will depend on many things: how hungry you are, how good the restaurant reviews are, how adventurous you are, how often you reckon you’ll be coming back ..etc. What’s important is that the study of the best way to make these kinds of choices – called reinforcement learning – has shown that optimal learning requires that you to sometimes make some bad choices. This means that sometimes you have to choose to avoid the action you think will be most rewarding, and take an action which you think will be less rewarding. The rationale is that these ‘sub-optimal’ actions are necessary for your long term benefit – you need to go off track sometimes to learn more about the environment. The exploration-exploitation dilemma is really a trade-off : enjoy more now vs learn more now and enjoy later. You can’t avoid it, all you can do is position yourself somewhere along the spectrum.

Because the trade-off is fundamental we would expect to be able to see it in all learning domains, not just restaurant food choices. In work just published, we’ve been using a new task to look at how actions are learnt. Using a joystick we asked people to explore the space of all possible movements, giving them a signal when they made a particular target movement. This task – which we’re pretty keen on – gives us a lens to look at the relation between how people explore the possible movements they can make and which particular movements they learn to rely on to generate predictable outcomes (which we call ‘actions’).

Using data gathered from this task, it is possible to see the exploitation-exploration trade-off in action. With each target people get 10 attempts to try to identify the right movement to make. Obviously some successful movements will be more efficient than others, because it is possible to hit the target after going all “round the houses” first, adding lots of extraneous movements and taking longer than needed. If you had a success like this you could repeat it exactly (‘exploit’), or try and cut out some of the extraneous movement and risk missing the target (‘explore’). Obviously this refinement of action through trial and error is of critical interest to anyone who cares about how we learn skilled movements.

I calculated an average performance score for the first 50% and second 50% of attempts (basically a measure of distance travelled before hitting the target – so lower scores mean better performance). I also calculated how variable these performance scores were in the first 50% and second 50%. Normally we would expect people who perform best in the first half of a test to perform best in the second half (depressingly people who start out ahead usually stay there!). But this analysis showed up something interesting: a strong correlation between variability in the first half and performance in the second half. You can see this in the graph

This shows that people who are most inconsistent when they start to learn perform best towards the end of learning. Usually inconsistency is a bad sign, so it is somewhat surprising that it predicts better performance later on. The obvious interpretation is in terms of the exploration-exploitation trade-off. The inconsistent people are trying out more things at the beginning, learning more about what works and what doesn’t. This provides them with the foundation to perform well later on. This pattern holds when comparing across individuals, but it also holds for comparing across trials (so for the same individual, their later performance is better for targets on which they are most inconsistent on early in learning).

You can read about this, and more, in our new paper, which is open-access over at PLoS One A novel task for the investigation of action acquisition.

Written by Comments Off on Fundamentals of learning: the exploration-exploitation trade-off Posted in Research

New paper: A novel task for the investigation of action acquisition

Our new paper, A novel task for the investigation of action acquisition, has been published in PLoS One today. The paper describes a new paradigm we’ve been using to investigate how actions are learnt.

It’s a curious fact that although psychologists have thoroughly investigated how actions are valued (i.e. how you figure out how good or bad a thing is to do), and how actions are trained (i.e. shaped and refined over time), the same effort has not gone into investigating how a behaviour is first identified and stored as a part of our repertoire. We hope this task provides a useful tool for opening up this area for investigation.

As well as the basic description of the task, the paper also contains a section outlining how the form of learning the the task makes available for inspection is different from the forms of learning made available by other ‘action learning’ tasks (such as, for example, operant conditioning tasks). In addition to serving an under-investigated area of learning research, the task also has a number of practical benefits. It is scalable in difficulty, suitable for repeated measures designs (meaning you can do it again and again – it isn’t something you learn once and then can’t be tested on any more) as well being adaptable for different species (meaning you can test humans and non-human animals on the task).

The paper is based on work done as part of the EU robotics project I’m on (‘I’M-CLeVeR‘) and on Tom Walton’s PhD thesis, The Discovery of Novel Actions

Written by Comments Off on New paper: A novel task for the investigation of action acquisition Posted in Research