Summary of responses to request for information on Judgment Fatigue (Also
called Syntactic Satiation)
 
Original post:  LINGUIST List:  Vol-6-974. Mon Jul 17 1995. ISSN: 1068-4875.
 
Special thanks to the following respondents:
 
Kimberly Barskaitiki, Tim Beasley, Elizabeth Bergman, Michael Bernstein,
Vivian Cook, Joseph Davis, Rianne Doeleman, Karen Emmorey, Suzette Haden
Elgin, Ted Harding, Stephen Helmreich, Erika L. Konrad, John E. Limber,
Bruce Nevin, John Robert Ross, Carson T Schutze, Linda Shockey, William
Snyder, Karin Stromswold, Joyce Tang Boyland, anyone I may have missed, and
anyone who thought about responding but hasn't yet.
 
Karin Stromswold and William Snyder have actually investigated this
phenomenon.  I have not read either of their papers yet, but I look forward
to doing so.  William Snyder (snyder@psyche.mit.edu) presented a poster on
this topic at LSA 1994, which was mentioned in Rick Hudson's summary post
of July 29, 1994.  Individual's interested in Dr. Snyder's poster should
contact him directly.
 
Hudson's post (LINGUIST List:  Vol-5-855. Fri 29 Jul 1994.  ISSN:
1068-4875.) summarized the literature available which addresses the
difference between linguists' and non-linguists' intuitions.  Hudson also
mentions (but does not cite) four references provided by Joyce Tang Boyland
regarding satiation effects (both syntactic and semantic).  Hudson's
original post is available from Linguist List archives.  Joyce Tang
Boyland's response is included below (in Part II of post, following).
 
Karin Stromswold's response (karin@muddle.rutgers.edu) was my connection to
Snyder, Hudson, and Boyland.  Her post is most relevant to my original
query and is included in its entirety below (in Part II of post,
following).
 
Most responses mentioned the influence of continual or repeated exposure
over a period of time, either mentioning examples similar to perceptual or
semantic satiation such as a word-repetition effect. (Bergman, Cook, Davis,
Doeleman, Haden Elgin, Harding, Helmreich, Limber, Schutze, Snyder,
Stromswold, Tang Boyland).  There are many differences, however, between
the effects of semantic satiation and "syntactic satiation" or judgment
fatigue:
 
First, semantic satiation is the detachment of form and meaning which
occurs as you continuously repeat some lexical item (see also Stromswold's
post below).  Judgment fatigue, in contrast, cannot be induced by the
repeated presentation of any sentence.  In fact, published experiments by
Nagata show that raters' intuitions only become more stringent on repeated
presentation.
 
Second, the loss of the meaning of a sentence is not an effect of judgment
fatigue.  The meaning of the sentence may be quite lucid to the individual
reading it, but that individual's attention is focused upon some aspect of
the grammatical acceptability of the sentence, and it is the ability to
judge grammatical acceptability which is lost, not the ability to make
sense of the utterance.
 
Third, while the particular characteristics of the stimuli which induce
judgment fatigue are not yet fully established, it appears that different
types of sentences induce the effect to different degrees (Stromswold,
Snyder as cited in Hudson's post).  This is not the case for semantic
satiation, where the repetition of any lexical item can induce the effect.
 
Fourth, the "scanting out" effect as noted by Haj Ross  points out that the
loss of intuition can be triggered by one single stimuli, not necessarily
the repeated presentation of numerous stimuli (relevant quote posted
below).  Conversely, subjects can rate hundreds of stimuli on the basis of
grammaticality, with no fatigue effect (beyond boredom), even if the
stimuli contain permutation of the same lexical items, be they open or
closed class items.
 
Note that point four is definitely true in the case that the stimuli are
all dichotomously "great--grammatical" or "horrible--ungrammatical".  There
is a definite influence of marginality, an effect which, to date, I believe
no one has explained.
 
Fifth, semantic satiation (as well as perceptual fatigue effects) is often
attributed to some type of neurological fatigue at the cortical level.
This could not be the (only) reason for judgment fatigue, however, because
there may not be any single word or syntactic structure which is repeated
in all of the stimuli (such as could be the case for judgment fatigue
induced by the scope of negation or negative polarity items).  For this
reason, it would be quite implausible to claim that judgment fatigue is the
result of some type of physiological / chemical / neurological saturation
which has occurred at the cortical level.
 
At this point, I do not believe judgment fatigue and semantic satiation
have much in common as behavioral disturbances.  (Incidently, while I am
not an expert on semantic satiation, I have not yet been forced by the
evidence to believe that semantic satiation is the result of a low-level
neurological fatigue.  Perceptual fatigue is a neurological phenomenon, but
I attribute semantic satiation to other conceptual strategies.)
 
Many linguists (Beasley, Nevin, Konrad, and one anonymous) also observed
'that the longer you stay in linguistics, the less you speak English.'  To
quote from Bruce Nevin's response:  << A student was once asked, in my
hearing, if a certain famous linguist was a native speaker of English.  The
reply: 'I think he used to be.' >>  Hudson's previous summation further
confirms that linguists and non-linguists do demonstrate significantly
different abilities in discriminating grammatical and ungrammatical
sentences.  This type of long-term "drift" of linguists' judgments may be
of interest in its own regard, but this type of "judgment drift" is not
identical to the temporary loss of linguistic intuition which I intended to
address.  As noted by Beasley and Konrad, it may indeed be the case that
exposure to second languages, exposure to dialects different from ones
native dialect, or exposure to a large number of perhaps marginal sentences
which occur rarely in natural spoken or written contexts (such as sentences
demonstrating violations of theoretical principals such as "weak crossover"
or "island constraints", for example) may actually alter an individual's
criteria for grammatical acceptability.  If one understands judgments of
grammatical acceptability to be metalinguistic type-categorizations, then
one may expect that an individual's discriminative ability will be altered
based upon that individual's personal experiences with exemplars of the
categories.
 
Others commented that judgment fatigue does not happen if the stimuli are
contextualized (Davis, Nevin) or if the referential situation is more
concrete rather than abstract (Davis, Doeleman, Nevin).  This is not the
case.  A loss of linguistic intuition of grammatical acceptability
definitely can happen spontaneously, such as while one is listening to a
conversation or reading texts, regardless of the concreteness of the topic,
with the following qualifications:  judgment fatigue is inspired only by
particular types of stimuli (which have not yet been clearly defined, but
see Part II of post, following) and by the listener's / reader's focus upon
the grammar of the utterance or sentence.
 
Other individuals related hard-to-comprehend sentences as examples inducing
judgment fatigue.  Some such sentences, notably sentences involving scope
ambiguity, may induce confusion, but confusion is not synonymous with the
loss of ones own linguistic intuitions.  For ambiguous or confusing
stimuli, we say, "Someone should rewrite this sentence.  It is ambiguous
(or anomalous)."  When we have judgment fatigue, we say, "I'm a monoglot,
but damned if I can decide whether this sentence is grammatical.  I sure
don't FEEL like a native English speaker!"
 
Lastly, some responses or general expressions of empathy or encouragement,
and actual examples (Barskaitiki, Emmorey, Shockey).  Thank you again to
all who responded.  If you are interested in the current proposal for my
experiments, or if you would like to hear more about the results and
conclusions from the experiments when I am finished running them, please
contact me.
 
The following are what I found to be especially informative quotes from
four individual respondents who gave specific comments regarding the types
of stimuli which tend to induce judgment fatigue.  Kimberly Barskaitiki
also sent a long, specific, detailed response with many interesting
examples.  (Thank you!)  Her response is not included here because it
presented fewer strong, testable, theoretical claims.  (I must note here
that Ross and Beasley are quoted without their expressed permission, for
which I hereby beseech the authors' belated dispensations.  I post all
comments here in only the most charitable light, with the hope of sharing
the author's experience and information.)
 
**************
From: Karin Stromswold  (karin@ruccs.rutgers.edu)
 
>As a graduate student at MIT in 1986, I wrote a paper on the
>phenomenon you described for a course I took with Merrill Garrett. I
>dubbed the phenomenon "syntactic satiation" as a nod to work on
>semantic satiation (the widely studied phenomenon that words loose
>their 'meaning' on repeated presentation.  e.g.., repeat the word
>"snow" 20 times and it starts sounding funny. and you can't come up
>with as many semantically associated words like "ice" and "cold").
>
>The study wasn't rigorously done and I never attempted to publish the
>paper (which was called "Syntactic Satiation), but what I discovered
>was
>
>1) with repeated judgments, subjects' judgments got less certain and
>reliable (some subjects explicitly said they weren't sure anymore).
>But even before subjects' lost confidence in their judgments/made
>mistakes, they got much slower in making judgments.
>
>2) syntactic satiation seems to be relatively specific.  For example,
>I had my subjects make repeated judgments of wh-questions (e.g.,
>that-t, long distance, island etc.) and I found that their ability to
>judge wh-questiosn was impaired, but not their ability to judge other
>stimuli (e.g., double object datives)
>
>3) Satiation did *not* occur when subjects were asked to make
>judgments about semantic plausibility (where implausible
>sentences involved violations of animacy restrictions, e.g.,
>The juice spilled the child that stained the rug)
>
>4)  I also didn't get any clear evidence of satiation at the
>morphological level (e.g., asking subjects to judge sentences
>with case/tense violations, e.g., I gave it to he; she give him a book)
>
>5) Rate of presentation seemed to be an important factor in inducing
>satiation. If subjects are urged to go quickly, they satiate faster.
>This goes along with findings in the semantic satiation where whether
>semantic priming or satiation occurs seems to depend on rate of
>presentation (at least in part).
>
>6) Being forced to make ungraded judgments (i.e., good vs. bad with no
>intermediate judgments allowed) also seemed to induce faster satiation
>
>7)  Satiation seems to be temporary, although I can't say what the
>refractory period is.  It is probably less than 1 month (the interval
>I retested at)
>
>8) I didn't detect satiation when I gave subjects mixtures of sentences
>to judge (in my case, datives, passives, wh-questions, and tense/case).
>
>When I did the lit review for the paper, I found a few other studies
>that looked at things related to syntactic satiation/syntactic priming.
>I will see if I can dig up the paper and find the references for you.
>Also, William Snyder at MIT presented a poster (?) at the LSA conference
>in 1994 (?) where he looked at this phenomenon.
 
From: Karin Stromswold  (karin@ruccs.rutgers.edu)
 
**************
>From John Robert (Haj) Ross (fj44@jove.acs.unt.edu):
 
>...The term I remember
>being in on the birth of was actually "to scant out", the term being
>proposed by Arnold Zwicky and me, and doubltless others, in the summer of
>1963, while we were working at MITRE Corporation in Bedford Mass, and
>scanting out was a common hazard.
 
From: fj44@jove.acs.unt.edu (Ross John Robert)
 
**************
 
>From Joyce Tang Boyland:
 
>Much of the information I have comes from responses I received to a
>query I posted to Linguist List about 4 years ago.
>...One of the refs was for Carson Schutze's MA thesis.  The others
>I think were only old cognitive psych papers on semantic not syntactic
>satiation.  I can dig them up if you are still interested.  Basically
>I don't think there has been much actual research on the subject;
>Schutze's lit review was very thorough but he only mentioned one or two
>not very conclusive pieces of research on satiation.  I'm very glad you
>are planning to research this experimentally.  I used to have stronger
>opinions on this than I do now, but I do think that syntactic satiation
>does happen.  I think that knowing about it ought to have an effect on the
>practice of linguistics, and I would also say that it has some implications
>for theories of acquisition and of historical language change, which I can
>expound on further if you are interested.
>
>Another thought is that there are at least two different things going on in
>syntactic and semantic satiation.  From the Linguist query I posted several
>years ago, I gathered that Haj Ross some time ago coined the phrase "scanting
>out" to describe the experience of losing one's intuition on what the word
>"scant" means and can be used for after thinking about it overtly for too long
;
>I think this is a case of excessive meta-linguistic attention interfering with
>what one would normally know, like the millipede in the fable being asked how
>it could possibly walk and then it couldn't walk anymore.  When I was little I
>used to play with saying a word over and over to myself until it became only a
>sequence of sounds without meaning; this might have been a similar phenomenon.
>I see this as a sort of evaporating of subconscious associations or constraint
s
>(for lack of better terminology) when under the spotlight of consciousness.
>
>I think a second process, in syntactic satiation in particular, is similar
>to something that's being studied in social psychology, which is that the
>frequency of your witnessing something may affect how acceptable you think
>it is, if you didn't have an opinion on it in the first place (which people
>often don't have on linguistic constructions).  (Having an opinion in the
>first place, if I remember correctly, is something that the researcher whom
>Schutze cites did not control for.  My copy of S is in another building at the
>moment.) (I am reconstructing vague memories very freely here, so I may not
>be reflecting the literature faithfully, but these are the lines along which
>I am thinking.)  So in this case people won't be *losing*all* the
>associations or constraints that a word or construction would normally have,
>but rather, a *particular* type of use they hear which violates a particular
>constraint may become *more*acceptable*.  The gross effect of certain uses
>becoming more acceptable may appear to be the same as the millipede effect,
>since one's intuitions are changing, but more careful inspection should show
>a different fine structure.
>I suppose I might as well add here that the degree to which a construction
>is susceptible to satiation effects (this latter one esp.) might well be a
>measure of its susceptibility to (or even progress towards) grammaticalization
.
 
From: jtang@cogsci.Berkeley.EDU (Joyce Tang Boyland)
 
**************
 
>From Tim Beasley (tbeasley@ucla.edu)
 
>A few things cause my intuition to crash and burn.
>
>The most frustrating and reliable is judging individual items in a list of
>sentences, and ranking them from ok to ? to ?* to *.  If I start from the ok
>side, I will over-accept sentences.  If I start from the * side, I will
>over-reject sentences.  It's worst when the sentences proceed stepwise,
>changing one lexical or syntactic feature (especially small ones).  And when
>the person asking me to rank them interacts, asking me if I'm sure.
>
>I.e., I become muddled if, in comparing sentences, I accept one, and then
>realize that the second is some sort of extension of or analogical to the
>first. If the first is right, then the second _must_ be right.  And then the
>third must be ok.  Er, well, no, but that doesn't stop me.
>
>Similarly, comparing two quasi-homonymous structures or lexemes in a battery
>of sentences designed to elicit the differences between them usually results
>in really, really bad results.  One such test posted to Linguist caused me
>to alter my own usage for several weeks.
>
>Lexical items that cause me to stumble regularly:  any, all.  Often there
>will be one very blatant reading of a sentence; if somebody suggests a
>second interpretation, my intuition curls up to hibernate for the duration.
>In a word: s-c-o-p-e.  Other problems with scope crop up, too (negation,
>adjectives, etc.).
 
[...reference to interference from experience with different languages...]
 
>I've come to assume that the linguistic muddle (apart from inter-language
>problems) results from attempting to find a coherent, rational meaning for
>the sentences.  Agreement and other such grammatical errors are trivial to
>spot, usually.  Otherwise, if the task succeeds, I assume the sentence must
>be ok, unless there is some clear, overwhelming error in word choice, focus,
>government, etc.  Lists of marginal sentences weaken my sense of what a
>clear error is as I move my ok/not-ok boundary marker (my perception of the
>norm) to allow for marginalia and enable me to interpret the next in a
>series of increasingly aberrant sentences.
>
[...another reference to experience with different dialects of English...]
 
>In any case, it takes a leap of humility to realize when I can no longer
>judge sentences in my native language....
 
From:  tbeasley@ucla.edu (Tim Beasley)