Frame-shifting: Event-Related Brain Response to Jokes

Seana Coulson, University of California San Diego

Marta Kutas, University of California, San Diego

Correspondence to:

Seana Coulson

Cognitive Science 0515

9500 Gilman Drive

La Jolla, CA 92093-0515

Voice: (858) 822-4037

Fax: (858) 534-1228

Acknowledgments

SC received support from National Research Service Award F32 DC00355. Research supported by the following grants to MK: HD22614; MH52893; AG08313.

ABSTRACT

Joke comprehension has been decomposed into two components: the registration of surprise, and a coherence stage, hypothesized to involve frame-shifting, in which the listener activates a new frame or schema from long-term memory to reinterpret information active in working memory. To find electrophysiological manifestations of frame-shifting, event-related brain potentials (ERPs) were recorded from the scalp as healthy adults read sentences that ended either as jokes that required frame-shifting, or with equally surprising nonjoke endings that were consistent with the contextually evoked frame. Because performance on comprehension questions indicated not all participants understood the jokes, participants were divided into good (83% correct) and poor (63% correct) joke comprehenders. In all participants, jokes elicited more negative ERPs 300-500 ms post-stimulus (N400), though in poor joke comprehenders the effect had a right anterior focus. Among good joke comprehenders only, the registration of surprise was indexed by late positivities 500-900 ms, while frame-shifting was indexed by a sustained negativity 500-900 ms focused over left lateral electrode sites. Temporal overlap of the ERP indices of surprise- and coherence- related joke processing suggests these two stages may not be as distinct from one another as traditionally assumed.

KEYWORDS: ERPs comprehension jokes language

INTRODUCTION

The ability to appreciate humor is one of the most intriguing aspects of human behavior, considered by many to be a defining human attribute (Nahemow, 1986). Though it recruits a number of cognitive processes, analysts have decomposed joke comprehension into two major components: the registration of surprise and the reestablishment of coherence (Suls, 1972). For example, the word "years" is surprising when it occurs in "I let my accountant do my taxes because it saves time: last spring it saved me ten years." However, to really "get" the joke, the listener must go beyond her surprise and formulate a new, coherent, interpretation in which the speaker is worried about going to jail, and pays an accountant to conceal illegal business practices. The coherence stage involves a process known as frame-shifting, in which the listener activates a new frame, or schema, from long-term memory to reinterpret information already active in working memory (Coulson, 2000). The present study uses event-related brain potentials (ERPs) recorded from the scalps of healthy adults to assess the psychological reality of frame-shifting, and differentiate it from the surprise component of joke comprehension.

Dissociation between the surprise and coherence components of joke comprehension is supported principally by data from patients with damage to either the left (LHD) or the right (RHD) cerebral hemisphere. Difficulty understanding jokes has long been associated with RHD, especially to the anterior portion of the right frontal lobe (Bihrle, Brownell, & Gardner, 1986; Brownell, Michel, Powelson, & Gardner, 1983; Gardner, Ling, Flamm, & Silverman, 1975; Shammi & Stuss, 1999). Moreover, subsequent research has suggested that the joke comprehension deficits associated with RHD involve the conceptual demands of frame-shifting (Bihrle et al., 1986; Brownell et al., 1983). For example, Bihrle and colleagues (1986) presented patients with jokes either verbally, or in a four-frame cartoon, and asked them to pick the punchline or "punch-frame" from an array of four choices: a straightforward ending, a neutral non sequitur, a humorous non sequitur, or the correct punchline. While both patient groups were impaired on this task, their errors were qualitatively different. The RHD patients erred by picking non sequitur endings, while LHD patients erred by picking the straightforward endings.

Drawing on Suls' (1983) two-stage model of humor, Brownell and colleagues (1983) argued that RHD patients retained the surprise component of joke processing, but were disproportionately impaired on the second stage in which coherence is reestablished between the narrative and the punchline. Bihrle et al. (1986) have also demonstrated that RHD patients have difficulty interpreting nonjoke materials that require frame-shifting. Given examples like the following,

Sally brought a pen and paper with her to meet the famous movie star. The article would include comments on nuclear power by well-known people.

RHD patients persisted in the original AUTOGRAPH interpretation, and seemed unable to incorporate information from the first sentence into the INTERVIEW interpretation suggested by the second. RHD patients' comprehension of these materials improved considerably, however, when misleading information was presented in the second sentence. These data seem to suggest that the deficits RHD patients experience in the comprehension and production of humor is cognitive rather than emotional, and involves inferential reanalysis.

While differences in RHD and LHD patients' behavior suggest that the surprise component of joke comprehension can be dissociated from the frame-shifting necessary to reestablish coherence, the implication for processing in normals is less obvious. First, brain damage often engenders functional reorganization and compensatory strategies that complicate inferences about the division of labor in the healthy brain (Farah & Feinberg, 2000). Further, the patient data do not address the time course of the two stages of joke comprehension, or the relationship between processes that subserve frame-shifting, and processes that subserve the comprehension of other sorts of surprising events. We addressed these questions by recording ERPs from normal adults as they read jokes and nonjoke control stimuli.

ERPs are the scalp reflections of sychronous synaptic activity, and afford an on-line measure of brain function with millisecond resolution. ERPs are also known to be sensitive to various aspects of language processing that are likely to be important for joke comprehension. For example, surprising events, both linguistic and nonlinguistic, are often associated with a late positive response in the ERP which is a member of the P3 family of components. In ERP language research, late positivities have been elicited by linguistic violations at the level of syntax, semantics, and even orthography (Coulson, King, & Kutas, 1998a, 1998b; Kutas & Hillyard, 1983; Muente, Heinze, Matzke, Wieringa, & Johannes, 1998). Although the P3 is not a unitary phenomenon, arising from the activity of multiple neural generators active in a variety of tasks, this well-studied ERP component can serve, nonetheless, as a useful measure of cognitive brain activity. Given that P3 amplitude varies with the subjective probability of a stimulus, stimulus meaning, and the amount of information the stimulus transmits (Johnson, 1986), late positivities in the ERP might be expected to reflect the extent that people find jokes surprising.

Although frame-shifting is not unique to jokes, this genre is unique in the extent to which the cue to shift is clearly demarcated by a disjunctor, or "punch-word," such as years in the joke about the accountant (Attardo et al., 1994). Moreover, from one perspective the disjunctor in a joke is merely an unexpected word whose occurrence initiates contextual integration. Previous research has linked general processes of contextual integration to the elicitation of a posterior negativity in the ERPs known as the N400. This component is evident between 200 and 600 ms after the onset of words presented in lists, sentences, and paragraphs (see Kutas, Federmeier, Coulson, King, & Muente, 2000 for review). Because the N400 was initially discovered in an experiment comparing congruous sentences with incongruous ones (Kutas & Hillyard, 1980), many view it as a marker of semantic anomaly.

However, subsequent research indicates that N400 is elicited by all words, congruous and incongruous, and is sensitive to the predictability of words in context. Approximately 90% of N400 amplitude is predicted by cloze probability, the percentage of people who produce a given word on a sentence completion ("cloze") task in which the target word has been omitted from the sentences. N400 amplitude and cloze probability are inversely correlated, being larger for unexpected words. In congruous sentences, N400 amplitude declines across the course of the sentence, as it becomes smaller with each open class word (Van Petten & Kutas, 1991). Because a similar decline does not occur in syntactic prose (grammatically well-formed sentences that make no sense), N400 amplitude has been interpreted as an index of the integrative demands posed by a particular word in a particular context (Kutas & Van Petten, 1994; Van Petten & Kutas, 1991).

To differentiate the brain processes involved in frame-shifting from the surprise component of joke comprehension, we recorded ERPs as neurologically intact participants read sentences that ended either as jokes or with an equally surprising nonjoke ending. Surprise was operationalized as cloze probability, and nonjoke endings were words that were unexpected in the context, but still consistent with the contextually evoked frame (see table 1). Both sorts of endings were unexpected, but only the jokes required frame-shifting. Further, because contextual constraint is thought to affect the specificity of people's semantic expectations (Schwanenflugel, 1991; Titone, 1998), stimuli were divided into high and low constraint sentences. High constraint stimuli were those sentences in which the best completion on the cloze task had a cloze probability of greater than 40%. Low constraint stimuli were sentences for which there really was no "best" completion, as even the most popular response for these items was offered by less than 40% of the respondents.

Because contextual constraint increases the specificity of readers' semantic expectations, the demands of joke comprehension might be more pronounced for high constraint jokes. Frame-shifting is important for the comprehension of both high and low constraint jokes, though, and only an ERP effect of ending (joke versus nonjoke) evident in both sorts of sentence contexts could be said to be a valid index of frame-shifting.

METHODS

Participants were 28 right-handed, monolingual English speakers. 10 were male, 9 had left-handed relatives, and all were healthy, college-aged adults with normal, or corrected to normal, vision. Participants' task was to read sentences and answer true/false questions while their on-going electroencephalagram (EEG) was recorded. EEG, sampled at 250 Hertz, was collected from 26 tin electrodes arranged in a geodesic dome-like pattern (Tucker, 1987), referenced to the left mastoid. Blinks and eye movements were monitored via electrodes beneath each eye and at the outer canthi. Informed consent was obtained, and all procedures conformed to the ethical requirements of the University of California, San Diego.

Materials included 60 experimental sentences that ended either as jokes (30), or as nonjokes (30), and 100 filler sentences. Jokes were assembled from various anthologies of one-line jokes, chosen so that understanding the joke required frame-shifting. In all cases, the disjunctor (the point at which the reader could, in principle understand the joke) was a sentence-final noun. Jokes that fit these criteria were normed on an off-line cloze task administered to 45 people from the same population as participants in the ERP experiment. Results of the cloze task enabled us to choose nonjoke endings by replacing the last word of jokes with contextually congruent completions that matched the joke endings for length (6.5 characters, sd=2), frequency (87 per million, sd=146), and cloze probability (3%, sd=2.5).

Experimental sentences in joke and nonjoke conditions were thus identical until the sentence-final word. However, joke endings required frame-shifting, while nonjoke controls were low-cloze items consistent with the contextually evoked frame. Experimental sentences were further divided into two classes of sentence constraint, determined by the probability of the most popular response for each sentence on the cloze task. This value was 40% or less for low constraint sentences, and greater than 40% for high constraint sentences. Examples of each of the sentence types can be found in table 1.

-----------------------------------------------------------

Insert Table 1 Here

-----------------------------------------------------------

Sentences were presented one word at a time at a rate of 2 per second. Duration for sentence-final words was 500 ms, while duration for all other words was 200 ms. ERPs were timelocked to the onset of sentence-final words. Each sentence was followed by a true/false question assessing comprehension, especially whether or not participants were getting the jokes. Response was signalled via a button press and response hands were counterbalanced across participants.

RESULTS

As can be seen in figures 1a and 1b, ERPs elicited by sentence final words displayed the P1/N1/P2 complex typical of visually presented words, a negative-going wave evident from 300-700 ms post-onset (N400), and a late positivity. ERPs were quantified by measuring the mean amplitude 300-500 ms, 500-700 ms, and 700-900 ms post-word onset relative to 100 ms pre-stimulus baseline. Measurements were subjected to repeated measures ANOVA with ending (Joke/Nonjoke), constraint (Low/High), and three factors that index scalp topography: hemisphere (Left/Right), laterality (Dorsal/Lateral), and anterior/posterior (4 levels). All p-values were corrected using the Huhyn-Feldt correction. However, for clarity we report original degrees of freedom.

Mean Amplitude 300-500 ms. Overall analysis of ERPs elicited 300-500 ms post-onset, the latency range in which N400 effects are typically most prominent, revealed a main effect of ending, and an interaction between ending and constraint (Ending F(1,27)=7.98; p<0.01; Ending x Constraint F(1,27)=4.34;p<0.05; Ending x Constraint x Anteriority F(3,78)=4.68, p<0.05, e=.42; Ending x Constraint x Laterality F(1,26)=4.26, p<0.05; Ending x Constraint x Laterality x Hemisphere F(1,26)=5.16, p<0.05). The main effect of ending results because ERPs to jokes were more negative than ERPs to nonjokes. This negative-going joke effect was characteristic of the N400 component: broadly distributed, but most evident over centro-parietal scalp regions, and slightly larger over the right side of the head (see figure 1). The interactions between ending, constraint, and various scalp topography factors result because N400 joke effects were reliable only in the high constraint sentences. Post hoc analysis revealed that among the high constraint stimuli, joke endings elicited greater amplitude N400 (Ending F(1,27)=14.55, p<0.001) whereas among low constraint stimuli, ERPs to jokes and nonjokes did not differ (F<1).

Mean amplitude 500-700 ms. A restricted effect of sentence constraint was observed over anterior regions of scalp, as indicated by a reliable interaction between constraint and anteriority (F(3,81)=6.01, p<0.05, e=.42) in the latency range 500-700 ms post-onset. This followed a similar marginal trend in the earlier 300-500 ms window (F(3,81)=3.44, p=0.07). As can be seen in figure 2, sentence final words in high constraint sentences were less positive than their low constraint counterparts over anterior scalp and slightly more positive over posterior (occipital and parietal) sites. This effect was qualified by a three-way interaction between ending, constraint, and anteriority (F(3,81)=7.07, p<0.05, e=.46), as constraint effects are driven more by participants' brain response to jokes rather than to nonjoke endings. Post hoc analyses of high and low constraint sentences indicate joke effects were reliable only in high constraint sentences, where ERPs to jokes were less positive over anterior scalp sites (Ending x Laterality x Anteriority F(3,81)= 4.9, p<0.05, e=.81).

Mean amplitude 700-900 ms. ERPs elicited by jokes 700-900 ms post-stimulus onset were more positive than those to nonjokes over posterior sites, and more negative over a few left anterior lateral sites, suggested by reliable interactions between ending, hemisphere, and anteriority (F(3,81)=3.64, p<0.05, e=.80) and between ending, laterality, and anteriority (F(3,81)=3.38, p<0.05, e=.95).

To summarize, joke effects were more pronounced for high than low constraint sentences. Relative to nonjoke controls, high constraint jokes elicited (1) greater amplitude N400s and (2) less positive ERPs over frontal scalp between 500 and 700 ms. Between 700 and 900 ms, in both high and low constraint sentences, relative to nonjokes, jokes elicited (3) greater negativity over left anterior sites and greater positivity over posterior scalp sites.

Spatial Analysis

Given our prior hypothesis that jokes might elicit qualitatively different ERPs than nonjokes, we conducted additional analyses in each of the three latency ranges using measurements normalized by the vector scaling method (McCarthy & Wood, 1985). Analysis of ERPs elicited by high constraint stimuli in the 300-500 ms window revealed no interactions among ending and any of the factors indexing scalp topography (all F's <= 1.05). This suggests the same neural generators were active in the processing of jokes and nonjoke controls between 300 and 500 ms.

In contrast, between 500-700 ms there was a reliable constraint by anteriority interaction (F(3,81)=6.21, p<0.05, e=.43), as well as three-way interactions between ending, hemisphere, and anteriority (F(3,81)=2.91, p<0.05, e=.94), and ending, constraint, and anteriority (F(3,81)=3.52, p<0.05, e=.56). The three-way interaction between ending, hemisphere, and anteriority reflects the left anterior focus of the joke effects between 500-700 ms, and suggests that a different configuration of neural sources may have been active during the processing of jokes and nonjoke controls. The interaction between ending, constraint, and anteriority results because apparent joke effects differed in high and low constraint sentences. In high constraint sentences, jokes were more negative over anterior scalp sites.

In low constraint sentences, jokes were more positive than nonjokes over anterior dorsal sites, and more negative over posterior sites, especially on the right side of the head. Post hoc analysis suggested that the topographic differences between joke and nonjoke endings were real in low constraint stimuli (Low Constraint: Ending x Hemisphere x Laterality x Anteriority F(3,81)=2.58, p=.059, e=1), but were likely due to amplitude differences in the ERPs to high constraint stimuli (all F's<=1.93).

Analysis of ERPs elicited 700-900 ms post-onset revealed a reliable interaction between ending, laterality, and anteriority (F(3,81)=5.15, p<0.01), reflecting the fact that, relative to nonjoke control stimuli, jokes were more negative over anterior lateral sites and slightly more positive over more dorsal, occipital and parietal electrodes. Topographic differences in this latency range are indicative of differences in the neural sources of ERPs elicited by joke and nonjoke endings.

In sum, between 300-500 ms (N400), the same set of neural sources were active for jokes and nonjokes. However, between 500 and 700 ms, the analyses indicate differences in the neural generators active both in the processing of high versus low constraint stimuli, as well as between joke and nonjoke endings for low constraint stimuli. Finally, analyses implicated the involvement of different neural sources in joke vs nonjoke processing between 700-900 ms.

Good and Poor Joke Comprehenders

Because we observed a wide range of performance on the comprehension questions, participants were separated into two comprehension groups based on a median split of their comprehension scores. The good joke comprehenders averaged 83% correct on the true/false questions that followed the joke stimuli, while the poor joke comprehenders averaged only 64% correct on the same questions. On questions that followed nonjoke stimuli, however, all but one participant scored 83% correct or better. The distinction between good and poor comprehenders thus reflects difficulty understanding the joke stimuli, and not with reading per se. To evaluate ERP differences related to participants' comprehension of the materials, we conducted additional analyses of mean ERP amplitudes measured 300-500 ms, 500-700 ms, and 700-900 ms post-word onset with between-participants factor comprehension group, and within-participants factors ending, constraint, and various scalp topography factors as above.

Between 300-500 ms post-word onset there was a null effect of comprehension group (F<1), but indications of differences in the distribution of the two groups' ERPs (Comprehenders x Hemisphere x Laterality x Anteriority F(3,78)=2.93;p<0.05, e=.86; Comprehenders x Ending x Hemisphere x Laterality F(1,26)=2.58;p<0.05). The latter reflects a more anterior distribution of the joke-related N400 among poor joke comprehenders, slightly larger over right hemisphere scalp sites (see figure 3.

Good joke comprehenders had generally more positive ERPs than poor joke comprenders throughout the rest of the recording epoch (500-700ms: F(1,26)=5.82, p<0.05; 700-900ms: F(1,26)=9.11, p<0.01). Between 500-700 ms, this effect was qualified by an interaction with ending, constraint, and various factors indexing scalp topography (Comprehenders x Ending x Constraint x Hemisphere x Laterality x Anterior/Posterior F(3,78)=3.43, p<0.05, e=.71).

Good Joke Comprehenders. Post hoc testing of the good joke comprehenders' ERPs between 500-700 ms post-stimulus onset revealed spatially restricted effects of ending (Ending x Laterality x Anterior/Posterior F(3,39)=3.26, p<0.05, e=.83), and of constraint (Constraint x Anterior/Posterior F(3,39)=5.99, p<0.05, e=.41). ERPs to jokes were less positive than those to nonjokes over left lateral anterior sites. And, collapsed across joke and nonjoke endings, low constraint stimuli elicited a fronto-central positivity, while high constraint stimuli elicited a positivity largest over occipito-parietal scalp.

Visual inspection of the data suggests constraint effects are due primarily to differences in ERPs to joke endings. Analysis reveals a marginal interaction between constraint, ending, and anteriority (Constraint x Ending x Anterior/Posterior F(3,39)=3.42, p=0.08, e=.48). The interaction also may reflect a slightly larger joke effect over anterior left lateral sites in the high than the low constraint sentences.

Post hoc analysis of good joke comprehenders' ERPs 700-900 ms post-onset revealed a three-way interaction between ending, laterality, and anteriority reflecting the negative-going joke effect over a few anterior lateral electrode sites and the small positivity over more medial, posterior sites (Ending x Laterality x Anterior/Posterior F(3,39)=3.22, p<0.05, e=.83).

Poor Joke Comprehenders. Post hoc analyses of the poor comprehenders' ERPs 500-700 ms post-onset revealed an interaction between ending, constraint, and various topographic factors (Constraint x Ending x Hemisphere x Laterality x Anterior/Posterior F(3,39)=3.4, p<0.05, e=.97). Jokes elicited less positive ERPs than nonjokes regardless of constraint; however, for high constraint stimuli, joke effects were bilateral and most evident over anterior scalp, while for low constraint stimuli, joke effects were evident only over posterior scalp sites, and were larger on the right side. Post hoc analysis of poor joke comprehenders' ERPs between 700-900 ms revealed only an interaction between constraint, hemisphere, and laterality reflecting the bilateral distribution of the ERPs to high constraint stimuli, and more right lateralized distribution of the ERPs to low constraint stimuli (Constraint x Hemisphere x Laterality F(1,13)=9.22, p<0.01).

Summary. A median split on comprehension scores for questions following the jokes resulted in a good and a poor joke comprehender group. Though the amplitude of ERPs in the N400 latency range did not vary with group membership, the scalp distribution of the joke effect did: the negative-going joke effect between 300-500 ms post-onset was more anteriorly distributed among poor comprehenders, especially over the right side of the head (see figure 3).

Between 500-700 ms, good comprehenders' ERPs were generally more positive than poor comprehenders', and displayed a different pattern of experimental effects. In the grand average, high constraint jokes apparently elicited less positive ERPs than nonjokes over frontal scalp sites. However, this particular pattern was not observed in the ERPs of either of the comprehension groups. Among good comprehenders, ERPs to high constraint jokes were less positive than nonjokes over anterior sites, and more positive over a few (occipital) posterior sites. Among poor comprehenders, ERPs to high constraint jokes were less positive than nonjokes over almost all the electrode sites. The ERPs of good and poor joke comprehenders thus cancel each other out over posterior sites (see figure 4a).

Similarly, the overall pattern of results for low constraint stimuli is not representative of either group's ERPs between 500 and 700 ms. Relative to nonjoke controls, low constraint jokes exhibited a nonsignificant trend for a frontally distributed positive-going joke effect, and a negative-going one over posterior sites: this is a composite of the frontally distributed positive-going joke effect in the good joke comprehenders' ERPs and the posterior negativity of poor joke comprehenders' ERPs (see figure 4b).

The overall ERP effects observed between 700-900 ms were representative of the good but not poor joke comprehenders' ERPs. As in the overall analysis, among good comprehenders jokes elicited less positive (more negative) ERPs than nonjokes over anterior left lateral sites, and more positive ERPs over posterior (occipital) sites. Among poor comprehenders, ERPs were modulated by sentence constraint, but not by ending (joke versus nonjoke).

DISCUSSION

Analysts have argued that joke comprehension involves two major components: the registration of surprise and the reestablishment of coherence (Suls, 1972). For example, in the following joke about a bartender, the last word is surprising because we expect the bartender to recommend a drink: "When I asked the bartender for something cold and full of rum, he recommended his wife." To fully comprehend this joke we need to establish a coherent interpretation of the bartender's "recommendation" as a complaint about his alcoholic wife. Because this reinterpretation process seems to require background knowledge, it has been hypothesized to involve frame-shifting, or, semantic reanalysis that results when the listener activates a new frame, or schema, from long-term memory to reinterpret information already active in working memory (Coulson, 2000). The present study was intended to assess the psychological reality of frame-shifting, and differentiate it from the surprise component of joke comprehension.

Accordingly, we recorded ERPs as participants read sentences that ended either as jokes, or with an equally surprising nonjoke ending. Stimuli were further divided into high and low constraint sentences as determined by the results of a separately administered cloze task. Sentences in which the best completion on the cloze task had a cloze probability of greater than 40% were deemed "high" constraint; sentences in which the most popular response on the cloze task was offered by less than 40% of respondents were deemed "low" constraint. Jokes and nonjoke controls were identical until the last word of the sentence, and care was taken to equate sentence final words for variables known to affect ERP indices of word processing: word length, word frequency, and cloze probability. The main difference between jokes and nonjoke controls was that while nonjoke endings were consistent with the contextually evoked frame, joke endings were intended to trigger frame-shifting.

Because the endings were designed to be equally surprising, ERP effects of ending should index the frame-shifting needed to establish a coherent interpretation of a joke. However, our results are complicated by the finding that the joke effects between 300-700 ms post-onset were reliable only in the high constraint sentences, and by the fact that some participants failed to get the jokes. Naturally, we would expect valid indices of frame-shifting to be present in both high and low constraint contexts, but absent from ERPs of poor joke comprehenders as they apparently failed to establish a coherent interpretation of the information presented.

N400 Effects

As reported in the previous section, ERP patterns differed as a function of ending type, sentence constraint, and comprehension group. Both joke and nonjoke endings elicited a negative-going response between 300-500 ms (N400). In low constraint stimuli, ending type did not modulate N400 amplitude; however, in high constraint stimuli, jokes elicited greater amplitude N400 than did the nonjoke controls. A negative-going ERP effect was observed in both good and poor comprehenders, though its scalp distribution differed as a function of comprehension group.

The presence of an N400 effect in poor comprehenders' ERPs and its absence in ERPs to low constraint sentences suggest this effect indexes the surprise stage of joke comprehension rather than the coherence stage. One might object that all of the stimuli were equally surprising -- given that joke and nonjoke endings were both unexpected words. However, because nonjoke endings were designed to be congruent with the contextually evoked frame, the ending type manipulation affected consistency with discourse-level expectancies. The observed differences in ERPs elicited by jokes and nonjoke controls thus demonstrate the brain's sensitivity to high level expectations such as those based on frames, scripts, or schemas retrieved from long-term memory (Barsalou, 1992; Rumelhart & Ortony, 1977; Schank & Abelson, 1977). These data structures contain causal and relational information, are organized hierarchically, and can be used to represent knowledge about a wide variety of objects, actions, and events (Fillmore, 1982; Minsky, 1975; Sanford & Garrod, 1981).

Though not unique to frame-shifting, our finding of joke effects on the N400 component indicates that joke comprehension engages neural processes generally associated with the processing of meaningful events. Our findings with joke processing mirror those of other researchers whose results also suggest N400 is sensitive to more global aspects of context. For example, St. George, Mannes, & Hoffman (1994) recorded participants' ERPs as they read ambiguous paragraphs that either were or were not preceded by a disambiguating title. Although the local contextual clues provided by the paragraphs were identical in the titled and untitled conditions, words in the untitled paragraphs elicited greater amplitude N400s. Similarly, Van Berkum, Hagoort, & Brown (1999) found that words which elicit N400s of approximately equal amplitude in an isolated sentence, do not elicit equivalent N400s when they occur in a 2-sentence context that makes one version more plausible than the other. For instance "quick" and "slow" elicit similar N400s in "Jane told her brother that he was exceptionally quick/slow." However, "slow" elicits a much larger N400 when this same sentence is preceded by "By five in the morning, Jane's brother had already showered and had even gotten dressed."

These demonstrations of the brain's relatively early sensitivity to discourse-level manipulations is consistent with the dynamic inferencing mechanisms assumed in many frame-based models of comprehension. In such models, comprehension is achieved by binding elements of the discourse representation to frames from long-term memory (Coulson, 2000; Lange & Dyer, 1989). Such models help explain how speakers are able to rapidly and routinely compute predictions, explanations, and speaker intentions (Shastri & Ajjangadde, 1993). Computational considerations led Shastri & Mani (1998) to propose that frame-based inferences necessary for language comprehension occur in a time frame on the order of hundreds of milliseconds. The onset of joke effects in the present study at 300 ms post-onset is consistent with their prediction.

The topography of the N400 effect differed across the two comprehension groups: the poor comprehenders' ERPs were more broadly distributed and had a right anterior focus (see figure 3). The right anterior distribution of this effect in the poor comprehenders' ERPs may reflect the recruitment of additional brain areas engendered by processing difficulty. In a systematic review of frontal activations revealed by PET and fMRI, Duncan and Owens (2000) report that particular regions in mid-dorsolateral and mid-ventrolateral areas of frontal cortex are activated by a broad range of cognitive demands, including perception, response selection, executive control and working memory. In all these cases, tasks that impose more taxing cognitive demands were associated with bilateral increases in blood flow to restricted areas of frontal cortex, and right hemisphere increases in blood flow to mid-ventrolateral areas (Duncan & Owens, 2000). Further, in fMRI studies of high level language processing, increasing the difficulty of discourse comprehension often results in increased right hemisphere activation (Robertson & Gernsbacher, 2000; St. George, Kutas, & Sereno, 1999). If the right anterior focus of the joke effect in our poor joke comprehenders can be taken at face value, it may reflect the importance of activation in right frontal regions for discourse-level processing. Clearly, however, such activation does not ensure successful comprehension, however, as the poor comprehenders did not always get the joke.

Sentence Constraint

We observed no effects of constraint on the N400, consistent with Kutas and Hillyard's (1984) finding that N400 amplitude does not index the violation of expectancy per se, but the congruence between the word and its preceding context. Rather, we observed a restricted effect of sentence constraint, in which sentence final words in high constraint sentences were less positive than their low constraint counterparts over anterior scalp, and slightly more positive over posterior scalp (see figure 2).

Differences in ERPs elicited by high and low constraint stimuli could reflect a variety of differences in the processing demands involved in the comprehension of each. For one thing, high constraint sentences are more likely to engender particular expectations about the features evoked by the last word of the sentence than are the relatively open-ended low constraint sentences. High constraint sentences are also more likely to describe a scenario that admits of a particular interpretation, thus allowing readers to commit to a particular frame to structure their interpretation of incoming words. Overall constraint effects may reflect the demands of the development of more or less concrete expectations about the overall scenario, the development of more or less definite expectations about the object named by the sentence-final word, as well as the violation of those expectations.

Although sentence constraint per se did not modulate N400 amplitude, N400 joke effects were overall larger and more robust in high constraint stimuli. Apparently, these stimuli more readily allow the reader to commit to a frame to structure her interpretation. It is this high-level commitment that we propose facilitates the processing of the nonjoke endings, on the one hand, and makes it more difficult to process the joke endings, on the other. Not only must the information active in working memory be reinterpreted in light of the new frame, but aspects of the initial interpretation will need to be suppressed. In low constraint sentences, by contrast, the general nature of the scenario may be somewhat unclear until the last word. As a result, the nonjoke endings to these low constraint sentences are less likely to be facilitated, either because readers do not completely commit to a particular frame, or because they commit to a frame that differs somewhat from the nonjoke frame designed by the experimenters.

The surprise component of joke processing was also indexed by a late positivity between 500 and 900 ms post-onset. In good comprehenders, joke endings elicited a posterior positivity in high constraint sentences (see figure 4a) and a fronto-central positivity in low constraint sentences (see figure 4b). Although we have operationalized sentence constraint so that it can be construed as a continuous phenomenon, differences in the scalp distribution of the positivities elicited by high versus low constraint jokes argue against this conventient construal. A number of factors presumably contribute to sentence constraint, and the current study suggests that different neural generators may mediate the processing, at least, of low versus high constraint jokes.

The scalp distribution of the posterior positivity elicited by high constraint jokes is characteristic of the P3b. One possible interpretation of this effect then is that the P3b elicited in the current study merely signifies the good comprehenders' classification of the sentence as a joke. However, such an interpretation is unlikely in view of the fact that low constraint jokes did not elicit this component even in good comprehenders. Instead, we suggest the posterior positivity elicited by high constraint joke endings reflects the violation of frame-level expectations, and perhaps the need to suppress the reader's initial interpretation, both of which may contribute to the updating of working memory representations. The careful reader is more likely to be surprised by the joke ending of a high constraint sentence that promotes frame commitment, and more likely to need to suppress the spurious inferences she made in her initial interpretation.

Among good (although not poor) joke comprehenders, low constraint jokes elicited a positivity over fronto-central sites. Anterior positivities have previously been associated with completely novel items in an oddball paradigm (e.g., Courchesne, Hillyard, & Galambos, 1975), as well as with attempts to retrieve source information about various events from episodic memory (Ranganath & Paller, 1999; Senkfor & Van Petten, 1998; Wilding & Rugg, 1996). Of the two, the latter seems less plausible, since participants are not likely to have heard these jokes before. Moreover, there is no reason they should attempt to retrieve an episodic trace for low but not high constraint jokes. The fronto-central positivity observed in the present study, however, may be related to the novel P3 or P3a, an anterior positivity often seen in in response to stimuli that evoke an orienting reaction, regarded as a sign of attention switching, and thought to originate in superior temporal cortex (Escera, Alho, Schroeger, & Winkler, 2000; Knight, 1997). Perhaps the joke endings to these relatively vague sentence contexts were perceived as novel and attention grabbing. Indeed, many low constraint contexts involved an appeal to an "ad hoc" category, as in "One good way to invest in the country is to buy a," or "In the game of golf, nothing counts as much as your," ("congressman" and "opponent," respectively, were the joke endings for these stimuli). These are less predictable, and thus more novel as jokes than the high constraint ones.

Frame-shifting

The ERP effect most closely associated with the frame-shifting process was the sustained negativity between 500-900 ms post-onset over left lateral anterior electrode sites. This effect was observed only in the ERPs generated by participants who consistently got the jokes. Further, among these good joke comprehenders, the sustained negativity was evident for both high and low constraint stimuli, suggesting that it reflects additional processing required for joke comprehension. Although spatial localization of the neural sources of ERP effects is inherently difficult, the extremely focal nature of the sustained effect is at least consistent with a generator in dorsolateral prefrontal cortex, and may index prefrontal activation implicated in the operation of verbal working memory (for review, see D'Esposito, Postle, Jonides, & Smith, 1998). Event-related fMRI data suggest dorsolateral pre-frontal cortex is part of a brain circuit that is important for the manipulation of information in working memory (D'Esposito, Postle, Ballard, & Lease, 1999), processing more relevant to the coherence phase of joke comprehension than to the registration of surprise.

In an experiment in which participants read simple sentences in a paradigm similar to that employed here, Kutas and King (1996) and Kutas (1997) report a slow-rising cumulative positive drift that was largest at left anterior recording sites, and which was more marked in ERPs generated by good comprehenders. Kutas (1997) found that the slow positive drift was also modulated by the demands of sentence comprehension, as ERPs elicited by more difficult sentences were more negative (less positive). Kutas and King suggest the time course of the slow positive drift may reflect dopaminergic neuromodulatory activity associated with with working memory processes in monkeys (e.g., Luciana, Collins, & Depue, 1998), and interpreted the positivity they observed as reflecting integration between items in long-term and working memory to form a mental model. Our finding that successful comprehension of jokes was associated with a sustained negativity over the same left lateral anterior electrode sites may reflect negative modulation of this slow positive drift in response to the demands of joke comprehension. Consistent with Kutas and King's (1996) interpretation, we suggest that the sustained negativity associated with joke comprehension indexes the rebinding of discourse elements in working memory. While not unique to joke interpretation, the presence of this ERP effect is consistent with the claim that successful frame-shifting invoked additional processing which made greater demands on working memory resources.

The goal of the present study was to assess the psychological reality of frame-shifting, and differentiate it from the surprise component of joke comprehension. We found that the surprise component was indexed to some extent by activity between 300-500 ms (during N400), as high constraint joke endings elicited larger amplitude negativities than nonjoke controls in both good and poor comprehenders' ERPs.[1] Among good joke comprehenders only, the registration of surprise was also indexed by later positivities between 500-900 ms. The response to high constraint jokes was characterized by a posterior positivity reminiscent of the P3b, while that to low constraint jokes was characterized by a fronto-central positivity reminiscent of the P3a. We therefore suggested that the posterior positivity may reflect the violation of frame-level expectations set up by high constraint sentence contexts, as well as the need to suppress the reader's initial interpretation. The fronto-central positivity, on the other hand, may reflect the perception of a stimulus from a completely novel category set up by low constraint sentence contexts. The absence of late positive joke effects in poor comprehenders' ERPs may be explained by the assumption that, like RHD patients, they do not suppress their initial interpretation (Gernsbacher & Robertson, 1995).

Moreover, the continued negative-going joke effects between 500-700 ms post-onset in poor joke comprehenders' ERPs (see figures 4a and 4b may reflect vain attempts to search semantic memory for information that might help them to make sense of joke endings. Because the sustained negativity 500-900 ms post-stimulus onset was elicited by both high and low constraint jokes in good, but not poor joke comprehenders, it was argued to index frame-shifting needed to establish a coherent interpretation of the joke. It is important to note that although the morphology (waveshape) and scalp topography of this sustained negativity clearly differentiate it from the positivities in the ERP to high and low constraint jokes, the three sorts of effects occur within the same time window. Temporal overlap of the ERP indices of surprise- and coherence- related processing suggest that these two stages may not be as distinct from one another as has been assumed in traditional accounts of joke processing.

As for the role of the intact right hemisphere in joke comprehension, our findings provide little support the hypothesis that right hemisphere processes are somehow crucial for joke comprehension. First, participants whose "N400" had a right anterior focus displayed difficulty understanding the jokes. Second, the brain potential most closely associated with frame-shifting needed for successful joke comprehension was a focal, sustained negativity 500-900 ms post-onset recorded at left anterior scalp sites. Without actually localizing the generators of these effects, it is impossible to know what they indicate about the involvement of the two hemispheres, other than that they appear to be different. However, taken at face value, these findings indicate that while discourse-level processing difficulty can engender RH activation, RH activation does not ensure comprehension. Perhaps the best way to reconcile the findings in the present study with evidence that right hemisphere stroke victims have joke comprehension deficits would be to record ERPs as RH patients and age-matched controls read or listen to stimuli like those employed here.

REFERENCES

Attardo, S., D.H. Attardo, P. Baltes, and M.J. Petray. (1994). The linear organization of jokes: Analysis of two thousand texts. Humor: International Journal of Humor Research 7(1): 27-54.

Barsalou, L. W. (1992). Frames, concepts, and conceptual fields. In Adrienne Lehrer and Eva Feder Kittay (eds.), Frames, Fields, and Contrasts: New Essays in Lexical and Semantic Organization. Hillsdale, NJ: Lawrence Erlbaum Associates.

Bihrle, A., H. Brownell, & H. Gardner. 1986. Comprehension of humorous and nonhumorous materials by left- and right- brain damaged patients. Brain & Cognition 5(4): 399-411.

Brownell, H., D. Michel, J. Powelson, & H. Gardner. 1983. Surprise but not coherence: Sensitivity to verbal human in right-hemisphere patients. Brain Language 18: 20-27.

Coulson, S. (2000). Semantic Leaps: Frame-shifting and Conceptual Blending in Meaning Construction. Cambridge and New York: CUP.

Coulson, S. & Federmeier, K.D. in press. Words and context: ERPs and the lexical/post-lexical distinction. Journal of Psycholinguistic Research.

Coulson, S., King, J.W., Kutas, M. 1998a. Expect the unexpected: Event-related brain response to morphosyntactic violations. Language and Cognitive Processes 13: 21-58.

Coulson, S., King, J.W., Kutas, M. 1998b. ERPs and domain specificity: Beating a straw horse. Language and Cognitive Processes 13: 653-672.

Courschene, E., Hillyard, S. & Galambos, R. (1975). Stimulus novelty, task relevance, and the visual evoked potential in man. Electroencephalography and Clinical Neurophysiology 39: 131-143.

D'Esposito, M., Aguirre, G.K., Zarahn, E.K., Ballard, D., Shin, R.K., & Lease, J. 1998. Functional MRI studies of spatial and non-spatial working memory. Cognitive Brain Research 7, 1-13.

D'Esposito, M., Postle, B.R., Ballard, D., & Lease, J. (1999). Maintenance versus Manipulation of information held in working memory: An event-related fMRI study. Brain and Cognition 41: 66-86.

Duncan, J. & Owen, A.M. (2000). Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends in Neurosciences 23: 475-483.

Escera, C., Alho, K., Schroeger, E., & Winkler, I. (2000). Involuntary attention and distractibility as evaluated with event-related brain potentials. Audiology & Neuro-Ontology 5 (3-4): 151-166.

Farah, M.J. & Feinberg, T.E. (Eds.) (2000). Patient-based approaches to cognitive neuroscience. Cambridge, MA: The MIT Press.

Fillmore, C. J. (1982). Frame semantics. In Linguistic Society of Korea (ed.), Linguistics in the Morning Calm. Seoul: Hanshin. 111-137.

Gardner, H., P. Ling, L. Flamm, & J. Silverman. (1975). Comprehension and appreciation of humour in brain-damaged patients. Brain 98: 399-412.

Gernsbacher, M.A. & Robertson, R. (1995). Reading skill and suppression revisited. Psychological Science 6: 165-169.

Johnson, R. (1986). A triarchic model of P300 amplitude. Psychophysiology 23:367-384.

Knight, R.T. (1997). Distributed cortical network for visual attention. Journal of Cognitive Neuroscience 9: 75-91.

Kutas, M. (1997). Views on how the electrical activity that the brain generates reflects the functions of different language structures. Psychophysiology 34: 383-398.

Kutas, M., Federmeier, K.D., Coulson, S., King,J.W. & Muente, T. (2000). Language. In (Eds. J. Cacioppo, L.G. Tassinary, G.G. Bernston), Handbook of Psychophysiology, 2nd edition. Cambridge and New York: Cambridge University Press.

Kutas, M. & Hillyard, S. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science 207: 203-205.

Kutas, M. & Hillyard, S. (1983). Event-related brain potentials to grammatical errors and semantic anomalies. Memory & Cognition 11: 539-550.

Kutas, M. & Hillyard, S. (1984). Brain potentials reflect word and semantic expectancy during reading. Nature 307, 161-163.

Kutas, M. and J.W. King. (1996). The potentials for basic sentence processing: Differentiating integrative processes. In T. Innue and J. McClelland (eds.), {\it Attention and Performance XVI: Information Integration in Perception and Communication}. Cambridge: MIT Press.

Kutas, M. & Van Petten, C. (1994). Psycholinguistics electrified: Event-related brain potential investigations. In Morton Ann Gernsbacher (Ed.), Handbook of Psycholinguistics. San Diego: Academic Press, 83-143.

Lange, T. E. \& Dyer, M. G. (1989). High-level inferencing in a connectionist network. Connection Science 1: 181-217.

Luciana, M., Collins, P., & Depue, R. 1998. Opposing roles for dopamine and serotonin in the modulation of human spatial working memory functions. Cerebral Cortex 8: 218-226.

McCarthy, G. & Wood, C.C. (1985). Scalp distributions of event-related potentials: An ambiguity associated with analysis of variance models. Electroencephalography and Clinical Neurophysiology 62: 203-208.

Minsky, M. (1975). Frame system theory. In P. N. Johnson-Laird, P. C. Wason (eds.), Thinking: Readings in Cognitive Science. Cambridge, UK: Cambridge University Press. 355-376.

Muente, T.F., Heinze, H.J., Matzke, M., Wieringa, B. & Johannes, S. 1998. Brain potentials and syntactic violations revisited: No evidence for specificity of the syntactic positive shift. Neuropsychologia 36, 217-226.

Nahemow, L. 1986. Humor as a data base for the study of aging. In: Nahemow, L, McCluskey-Fawcett, KA, McGhee, PE, editors. Humor and aging. Orlando, FL: Academic Press, pp. 3-26.

Ranganath, C. & Paller, K.A. (1999). Frontal brain potentials during recognition are modulated by requirements to retrieve perceptual detail. Neuron 22:605-13.

Robertson, D.A., Gernsbacher, M.A., Guidotti, S.J., Robertson, R.R.W., Irwin, W., Mock, B.J., Campana, M.E. (2000). Functional neuroanatomy of the cognitive process of mapping during discourse comprehension. Psychological Science 11: 1-8.

Rumelhart, D. E. & Ortony, A. (1977). The representation of knowledge in memory. In R. C. Anderson, R. J. Spiro, W. E. Montague (eds.), Schooling and the Acquisition of - 27 -Knowledge. Hillsdale, NJ: Lawrence Erlbaum Associates.

Sanford, A. J. & Garrod, S. C. (1981). Understanding Written Language: Explorations Beyond the Sentence. Chichester: John Wiley \& Sons.

Schank, R. C. & R. P. Abelson. (1977). Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures. Hillsdale, NJ: Lawrence Erlbaum Associates.

Schwanenflugel, P. (1991). Contextual constraint and lexical processing. In Greg B. Simpson (Ed.) Understanding word and sentence. Amsterdam, The Netherlands: North Holland, pp. 23-45.

Senkfor, A.J. & Van Petten, C. (1998). Who said what? An event-related potential investigation of source and item memory. Journal of Experimental Psychology: Learning, Memory, & Cognition 24: 1005-1025.

Shammi, P., & Stuss, D.T. (1999). Humour appreciation: a role of the right frontal lobe. Brain 122: 657-666.

Shastri, L. and Ajjanagadde, V. (1993). From simple associations to systematic reasoning: A connectionist representation of rules, variables, and dynamic bindings using temporal synchrony. Behavioral & Brain Sciences 16(3):417-494.

Shastri, L. & Mani, D.R. (1998). Massively parallel knowledge representation and reasoning: Taking a cue from the brain. In (Eds.) J. Geller, H. Kitano, and C. Suttner Parallel Processing for Artificial Intelligence 3. Amsterdam and New York: Elsevier Science.

St. George, M., Kutas, M., Martinez, A., Sereno, M.I. (1999). Semantic integration in reading: engagement of the right hemisphere during discourse processing. Brain 122: 1317-25.

St. George, M., Mannes, S. & Hoffman, J. E. (1994). Global semantic expectancy and language comprehension. Journal of Cognitive Neuroscience 6: 70-83.

Suls, JM. (1972). A two-stage model for the appreciation of jokes and cartoons: an information processing analysis. In: Goldstein JH, McGhee PE, editors. The psychology of humor: theoretical perspectives and empirical issues. New York: Academic Press, 81-100.

Titone, D. (1998). Hemispheric differences in context sensitivity during lexical ambiguity resolution. Brain & Language 65, 361-394.

Tucker, D.M. (1993). Spatial sampling of head electrical fields: the geodesic sensor net. Electroencephalography and Clinical Neurophysiology 87: 154-163.

Van Berkum, J.J.A., Hagoort, P., Brown, C.M. (1999). Semantic integration in sentences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience 11: 657-671.

Van Petten, C. & Kutas, M. (1991). Influences of semantic and syntactic context in the processing of open- and closed- class words. Memory & Cognition 19: 95-112.

Wilding, E.L. & Rugg, M.D. (1996). An event-related potential study of recognition memory with and without retrieval source. Brain 119: 889-905.

TABLES

Table 1. Sample Stimuli

Low Constraint: Statistics indicate that Americans spend eighty million a year on games of chance, mostly

Joke Ending: weddings.

Nonjoke Ending: dice.

High Constraint: She read so much about the bad effects of smoking she decided she'd have to give up the

Joke Ending: reading.

Nonjoke Ending: habit.

Table 1. Sample Stimuli

FIGURES

Figure 1a. Grand average (n=28) ERPs to joke (dotted) and nonjoke (solid) endings from all electrode (n=26) sites. Negative voltage is plotted up in this and all subsequent figures.

Figure 1b. Grand average (n=28) ERPs to joke (dotted) and nonjoke (solid) endings over a right posterior electrode site.

Figure 2. Grand average ERPs (n=28) for the last word of low (solid) and high (dotted) constraint sentences.

Figure 3. Voltage maps of mean amplitude of the joke effect (jokes minus nonjokes) measured 300-500 ms in good (left) and poor joke comprehenders.

Figure 4a. Grand average ERPs to joke (dotted) and nonjoke (solid) endings to high constraint sentences in good joke comprehenders (n=14) and poor joke comprehenders (n=14).

Figure 4b. Grand average ERPs to joke (dotted) and nonjoke (solid) endings to low constraint sentences in good (left) and poor (right) joke comprehenders.

---

[1] Inspection of figure 4b suggests that among good comprehenders, joke endings for low constraint sentences also elicited greater amplitude N400. Null results among this group may reflect the presence of an overlapping positivity.