The hallmark of human behavior is flexibility.  We are able to generate goal-directed actions under diverse circumstances when the environment provides ambiguous or conflicting cues.  The main focus of my research has been to learn how people match external stimuli with internal states to choose responses and engage in flexible, goal-directed behaviors.  I have approached this problem using a range of methodologies and behavioral phenomena, including dual-task interference, bimanual coordination, and cognitive control.  Although this work represents a diverse set of findings, a central theme is that response selection processes are not generic but instead depend on the specifics of the task and the context in which it is embedded.  Thus, developing a viable theory of response selection requires understanding how the underlying processes are shaped by task demands, practice, and, most importantly, the way that individuals conceptualize their actions. 

Dual-task interference

Much of my research relates, in one way or another, to the limitations of dual-tasking – the decrements in performance observed when people try to do two things at the same time.  Dual-task situations are becoming increasingly prevalent in modern life, as, for example, people operate cell phones or ipods as they drive, talk, or write papers.  I believe that studying the interactions between concurrently performed tasks provides a powerful tool for examining the processes and representations that govern our behavior.  I view my general approach as analogous to a particle physicist who studies the results of collisions of particles too small to directly observe: by examining the interactions between simultaneously performed tasks, I attempt to probe the structure of response selection processes.  The patterns of dual-task costs I have observed in a number of studies are not accounted for by any of the existing theories of response selection and therefore provide an opportunity to break new theoretical ground.

The starting point of this line of research is a study examining whether practice allows individuals to perform two tasks at the same time.  We (Hazeltine, Teague, and Ivry, 2002) set out to determine whether the absence of dual-task costs could be accounted for by models that assumed that responses were selected in a one-at-a-time, serial fashion.  We added a short interval between the stimuli for the two tasks and found that, after practice, this interval affected the time between the two responses but not the response times (i.e., the interval between the stimulus and the response).  Thus, it appeared that individuals were able to select two responses at the same time.  However, a follow-up study (Hazeltine, Ruthruff, & Remington, 2006) showed that dual-task costs strongly depended on the specific combination of tasks.  When a task using auditory stimuli and manual responses was paired with a task using visual stimuli and vocal responses, dual-task costs were robust throughout training.  In contrast, the costs were essentially eliminated after training when a task using auditory stimuli and vocal responses was paired with a task using visual stimuli and manual responses.  The difference in dual-task costs was particularly striking because the stimuli and responses were highly similar for the two groups and because the single-task reaction times were nearly identical. 

To further illuminate how the relationship between the tasks interacts with practice to determine dual-task costs, we are currently performing a series of experiments in which we systematically manipulate whether the stimuli for the two tasks occur on the same modality and whether the responses occur on the same modality.  The results indicate that the complete elimination of dual-task costs reported by myself and others occurs only under a fairly narrow set of conditions.  Moreover, the findings suggest that early in practice, task difficulty plays the dominant role in determining dual-task costs.  However, after several sessions of practice, it appears that the overlap in the stimulus-response pairings across the two tasks drives the dual-task costs (Hazeltine, Ruthruff & Wifall, in prep). 

Given these findings, I have proposed a new model of response selection in which performance early and late in practice involve distinct processing strategies (Hazeltine, Ruthruff, & Remington, 2006).  Early performance is controlled in the sense that central operations are performed serially.  In contrast, after practice, central operations can take place in parallel.  However, this does not mean that the central operations for the two tasks do not interact.  When the two tasks involve similar central codes, interference between concurrently performed operations can occur, leading to persistent decrements in performance. 

The role of working memory in response selection

To better understand the interactions between concurrently performed tasks, my lab is investigating the role that working memory (WM) plays in response selection and whether it accounts for the fact that overlap in the stimulus-response mappings across the tasks rather than task difficulty appears to drive dual-task costs after practice.  A key assumption is that making the appropriate response to a stimulus engages the same processes as retrieving an item in a WM task.  This assumption has been suggested by other theorists, but the implications for skill acquisition in general and response selection in particular have not been explored.  Specifically, we propose that, after practice, dual-task performance primarily reflects the ability to simultaneously retrieve items from WM.   Whether it is possible to retrieve multiple items in WM at the same time has become an active topic in the memory literature.  Thus, this research has the potential of bridging two separate literatures.

To directly test this proposal, we are conducting a series of experiments combining a response selection task and a WM task.  The logic of these experiments is akin to earlier work examining whether WM consists of separate modality-specific subsystems.  However, the present experiments isolate the components of the response selection task that interfere with different types of information in WM.  We independently manipulate the stimuli and responses in the choice reaction time task, along with the type of information in the WM task.  The results indicate that response selection interferes with maintenance in WM, depending on the stimuli and responses of the choice reaction time task, and the type of information to be held in WM.  Thus, we have initial evidence that response selection processes do engage WM (Hazeltine & Wifall, 2011).

Bimanual coordination

Bimanual coordination encompasses a large class of situations in which the brain must simultaneously control multiple movements, such as when we use our two hands to manipulate an object or perform a task.  Bimanual coordination has been one of the most widely studied problems in motor control.  In my lab, we have re-evaluated this work, looking at bimanual coordination as a special case of dual-task performance.  Movements of the two hands may be highly similar in terms of their underlying representations, and thus, might be expected to produce interference even after extensive practice according to the WM account described above.  Indeed, dramatic interference can be observed with simple bimanual movements – consider the classic example of patting one’s head with one hand while rubbing one’s head with the other.  Even simple (and more easily studied) movements, such as depressing the index finger of the left hand and the middle finger with the right hand can produce robust interference.  Intriguingly, this combination of movements can be performed relatively effortlessly in real-world tasks, such as typing.  However, in most experimental tasks huge costs are observed when individuals are required to make different keypresses with the two hands. 

To examine this phenomenon, I investigated whether compatibility effects depended on the stimuli, the responses, or the way that individuals conceptualized their responses.  Spatially compatible stimuli were paired with number stimuli to indicate keypresses with the left and right hands.  Two groups of participants were used that differed only in terms of the mapping of the number stimuli; the mapping of the spatial stimuli was always the same and compatible.  Although spatially compatible stimuli are thought to directly activate their corresponding responses, the pattern of bimanual interference for the spatial stimuli depended on the mapping of the number stimuli (Hazeltine, 2005).  Even more striking, this effect was present even when no number stimuli were presented.  These results indicate that this prevalent form of bimanual interference is mediated by central representations that are based on the way subjects conceptualize their responses rather than conflict between stimuli or between responses as has been assumed in previous theories.

Other bimanual phenomena offer clues about the structure of response selection processes as well.  For example, making a pair of keypresses, one with a finger of one hand and one with a finger of the other, is usually performed more slowly than making two keypresses with the same hand (Hazeltine et al., 2007).  However, as with the compatibility effects, this pattern is not observed during everyday tasks such as typing.  The differences in bimanual costs may have to do with the extensive practice that typing movements receive, but they do not appear to stem from difficulty per se.  The spatially compatible stimuli produce very similar reaction times to typing stimuli, yet the spatial stimuli produce robust bimanual costs and the typing stimuli do not.  Bimanual costs were not related to the difficulty of the stimulus-response mapping but instead stemmed from overlap in the mappings for the responses of the two hands.  We are presently using skilled typist to examine how practice changes the representations of actions.  A theme of this research is that different stimuli used to signal the same responses can invoke very different response selection processes, and these processes are determined by the manner in which the task is conceptualized. 

Cognitive control

When individuals are confronted with multiple task demands, control processes are assumed to govern and coordinate the potentially conflicting cognitive operations.  Evidence for these control processes are found in a phenomenon called sequential effects: After a trial with high conflict, the effects of conflict are smaller than after a trial with little conflict.  That is, conflict causes individuals to pay less attention to the irrelevant information, thereby reducing its consequences.  This pattern is presumed to reflect the dynamic allocation of control, and we exploited it to probe the structure of task representations.  We used a conflict task called the Simon task, in which a task-relevant stimulus is presented in different locations that correspond with the locations of the appropriate responses.  However, its location must be ignored, because the identity of the stimulus, not its location, indicates the appropriate response.  First, we established that sequential effects in the Simon task occur even when neither the stimulus identity nor the stimulus location are the same as on the previous trial (Akcay & Hazeltine, 2007).  This finding indicates that sequential effects do not stem from the inhibition of specific-stimulus features but instead relate to high level representations.

We next examined the boundary conditions of sequential effects to examine task representations.  When we encouraged participants to think of a version of the Simon task as two separate tasks, sequential effects were not observed after a task switch.  Moreover, when the participants switched back to a task, the congruency of the last trial of that task affected the magnitude of the congruency effect on the current trial (Akcay & Hazeltine, 2008).  These findings suggest that sequential effects relate to representations of the task and not simply attentional states.  As with the findings from the dual-task studies, the results indicate that tasks are represented as more than just a collection of stimulus-response associations.  The brain appears to encode rich representations of the task and these representations have important effects on performance in terms of compatibility effects, learning, and control processes (Hazeltine et al., 2011). 

Learning to read

Can factors that affect skill learning be harnessed to improve reading?  Beginning readers must decode letters on the page into the sounds of language.  These decoded sounds can then be linked meaningful ideas and concepts, allowing children to leverage what they know (spoken language) to learn a new skill (reading).  Traditionally, decoding was taught as a process of memorizing a large set of rules that translate letters to sounds (e.g., “A with a silent E makes the ‘long A’, as in LATE”).  However, while rules are clearly an important educational tool, many psychologists now view learning to read as a process of encoding probabilistic relationships between the written letters and the sounds rather than explicit rules. As a result, learning to read may be more like acquiring a skill, like shooting a basketball, than like memorizing a list of rules.  It is therefore potentially significant for the instruction of reading that basic research has uncovered several principles that improve skill acquisition.  We test these principles in first-graders learning to read by partnering with a private-sector reading-technology company to develop short-term studies in which students learn a handful of decoding skills.  To examine the correspondence between reading and skill acquisition, we have devised a motor task that captures the same kind of probabilistic input-output relationships that are required for decoding in word reading. 

Translating this science to education is difficult and principles from cognitive science are rarely applied in the classroom. For reading, instructional curricula emphasizing decoding and phonics are more successful than others. Such approaches give children mastery of the letter->sound mappings or Grapheme Phoneme Correspondence (GPC) regularities. This can lead to fluent word recognition and eventually comprehension. These letter->sound mappings are not so different than the SR mappings studied in learning theory, making reading and cognitive science an apt comparison. Yet, the GPC mappings used in decoding are full of exceptions and quasi-regularities and are used in a complex array of tasks. Thus, applying the science of learning to the classroom is not straightforward.

Numerous laboratory studies show that variability in irrelevant stimulus components improves statistical learning; we applied this to reading in a short term learning study examining GPC regularities involving vowels. The results were clear. Across a wide range of factors, including initial performance level and learner gender, practice with variable items led to significantly greater improvements in the decoding skills we taught than practice with similar items, and to greater generalization to novel words and tasks (Apfelbaum, Hazeltine, & McMurray, 2012).

Summary

This body of research includes a range of topics, but each topic addresses the content of the central operations by which actions are selected.  Across the various domains, some common features of response selection processes emerge.  First, the data indicate that central operations vary as task demands change.  This point, while perhaps unsurprising to laypeople, runs counter to a large class of theories proposing a generic central processor or resource pool that is drawn upon to translate stimuli into responses.  Second, the composition of central operations is shaped by a range of factors, including the stimuli, the responses, practice, and the conceptualization of task goals.  Thus, the challenge is to develop models of response selection that can accommodate these factors while making specific predictions about performance under a range of conditions.  To this end, we are designing computational models of response selection processes that capture some basic principles.  For example, a feature of our modeling work is that the representations of stimuli and responses overlap.  Moreover, these representations are dynamic in the sense that they narrow with practice to emphasize only the goal-relevant features.  With these various approaches, I aim to learn how goal-directed behavior emerges from the welter of sensations and drives buzzing through our minds.