Moving from left to right on the page, we combine two elements
creating a new object, then we combine2 this new structure with the
next word to the right, and so on.
(4) (a) Nemo È ate
(b) (Nemo È ate) È Dory’s
(c) ((Nemo È ate) È Dory’s) È seaweed
(d) (((Nemo È ate) È Dory’s) È seaweed)
Let us call this the structured-concatenation hypothesis. This ap-
proach does not suVer from the problem of (2), in that any subpart
of (4d) is not identical to a subpart of any of the strings in (3). For
example, the Wrst concatenation in (4) is not identical to the Wrst
concatenation of (3b):
(5) (Nemo È ate) 6¼ (Dory È ate)
This is what we want, since we do not want sentence (2) to mean the
same thing as (3b). Nevertheless, the structured concatenation hypoth-
esis suVers in a diVerent, important, way. If you look closely at the
brackets in (4d), you will note that Dory’s is structurally closer to ate
than it is to seaweed. We can capture this more precisely by counting
the number of parentheses enclosing each item. If we were to number
matching opening and closing parens, we get the annotated structure
in (6):
(6)(
1
(
2
(
3
Nemo È ate)
3
È Dor y’s)
2

the Structured-Concatenation hypothesis cannot be right, and that the
sentence in (8) is probably structured more like (9):
The fish ate Dor
y
’s seaweed.
Note that the structure represented in (9) cannot be created by a
procedure that relies strictly and solely on linear order (such as the
concatenation-as-addition and the structured-concatenation proced-
ures). Instead, we need a richer hierarchical structure (such as that in
(9)) to represent which words go together. This hierarchical structure
must represent the intuition that the bears a closer relationship to Wsh
than it does to eat or seaweed, and that Wsh bears a closer relationship
to the than it does to eat, etc.
On the other side of the scale, human syntax seems to be Wlled with
relationships that are not obviously ‘‘close.’’ Take, for example, the
contrast between (10a and b):3
(10) (a) The men that John saw are tall.
(b) *The men that John saw is tall.
Either of the two addition hypotheses fail immediately with such
sentences. The noun that the verb are agrees with in (10a) (i.e. the
men) is nowhere near the verb itself. Worse, there is a closer noun for
the verb to agree with (i.e. John). Accounting for these facts is easy if
one takes a hierarchical approach to combinatorics.
The focus of this book is on capturing the nature, mechanics, and
forms of both ‘‘closeness’’ relationships and relationships that are more
distant like the agreement facts in (10).
3 Thanks to Massimo Piatelli Palmarini for pointing out these examples to me.
constituent structure 11
2.2 Regular grammars
It is worth brieXy discussing the long tradition in linguistics and

that descriptions of sentences are ‘‘built up’’ by grammars and ‘‘knocked down’’ by
automata. This derivational metaphor is now widely rejected by practitioners of most
varieties of syntax, except followers of the Principles-and-Parameters framework (a.k.a
‘‘GB theory’’ and ‘‘Minimalism’’) and even then it is largely viewed as a metaphor (cf.
Epstein et al. 1998). As such, the distinction between automata and grammars is blurred.
Although they are not technically the same thing, for the purposes of this book we will
treat them as if they are notational variants.
12 preliminaries
Each dot represents a state, the words are functions from state to state.
This machine generates sentences such as those in (12) (but not limited
to them):
(12) (a) The man comes.
(b) The old man comes.
(c) The old old man comes.
(d) The old old old man comes.
(e) The old old old old man comes.
etc.
(f) The old men come.
(g) The old old men come.
(h) The old old old men come.
(i) The old old old old men come.
etc.
More sophisticated versions of regular grammars are found today in
StratiWcational Grammar (Lamb 1966) and in connectionist/neural
net/Parallel Distributed Processing models (see e.g. Rumelhart and
McClelland 1987).5
Chomsky (1957) argued that simple regular grammars are insuY-
cient descriptions of human language. (For a transparent retelling of
Chomsky 1957’s results, see Lasnik 2000).6 Chomsky enumerates four
problems with regular grammars:

is poss ible to ‘‘circumWx’’ the expression anti . . . missile.Soananti
missile missile is a missile that attacks missiles. This circumWxation
can be applied an inWnite number of times: an anti anti missile missile
missile, is a missile that attacks missiles that attack other missiles.
(14) (i) missile
(ii) anti missile missile
(iii) anti anti missile missile missile
In this language, then, we have a pattern where the number of times the
word missile is found is exactly one more than the number of times
the word anti is found (anti
n
missile
nþ1
). Another example might be
the correspondence between subjects and verbs in Subject-Object-Verb
(SOV) order languages like Japanese. If we have three nouns marked as
subjects [ NP
1
,NP
2
,NP
3
]wemust also have three verbs which agree
with those subjects [ . . . V
3
,V
2
,V
1
].

2
(ii) Either S1 or S2 *Either S1 then S2
(iii) The boy, who said S
1
, is arriving today *The boy, who said
S1, is arriving today (Chomsky 1957: 22)
Examples which Chomsky does not mention are cases such as that in
(16), where the verb insist requires that the embedded verb be be a bare
inWnitive.
(16) I insist that John be honest.
Another example of non-local dependencies is the structure of
embedded clauses in the Zu
¨
ritu
¨
u
¨
tsch variety of Swiss German dis-
cussed by Shieber ( 1985) (and the related facts from Dutch discussed
in Bresnan et al. (1982) Huybregts (1984)8). In Zu
¨
ritu
¨
u
¨
tsch, as in many
Germanic languages, some verbs, such as ha
¨
lfen ‘help’, require that
their objects take the dative case (marked, among other mechanisms by

ranging over strings of words, a and b are nouns and c and d are verbs,
and n,m $ 0) The relationships here are non-local and thus require
memory of the kind that a simple regular grammar or Wnite-state
8 See also the survey of the literature on the non-context-freeness of human language in
Pullum (1986).
constituent structure 15
automaton cannot provide. These constructions also provide examples
of the kind of counting dependency described in (13) in that the
number of subjects and the number of verbs have to correspond, but
are added to the structure non-locally.
While it is true that in traditional regular grammars and Wnite-state
automata we have no access to anything other than the immediately
preceding state, we should note that in modern connectionist model-
ing it is possible to encode such information. The networks are con-
siderably more complex (with parallel rather than serial connections)
and each state is weighted to reXect statistical frequency (determined
by exposure to a training regimen); these two factors combined can
mimic the eVects we are describing here (Rumelhart and McClelland
1987).
Chomsky also shows that grammars of human languages are struc-
ture dependent, a fact that cannot be expressed in a Wnite-state gram-
mar which merely reXects concatenation.9 This is Problem 3.He
illustrates this with the phenomenon of subject-aux inversion (SAI):
(18) (a) Mary has gone.
(b) Has Mary gone?
As a Wrst hypothesis, we might argue that the general procedure here is
to invert the Wrst two words in the sentence (such a formalization
would be consistent with concatenation view of syntax). But this
hypothesis is easily disproved:
(19) (a) The man has gone.

we saw that one typical approach to a purely linearly organized sen-
tence structure (regular grammars) seems to fail on conceptual and
empirical grounds. Instead, a richer, hierarchical, structure is needed.
The fact that such hierarchical structures can be referred to by other
grammatical processes provides not only evidence for their existence,
but also drives the Wnal nail into the coYn of a concatenation/Wnite
state11 account of word combinatorics.
The hierarchical organization of sentences represents constituents.12
The idea of constituent analysis of sentences dates back at least to
10 DeWned, perhaps, as the auxiliary with the fewest brackets around it.
11 However, it does not disprove the approaches of connectionism/neural networks or
stratiWcational grammar, which all involve enriched networks which have been claimed to
be able to mimic the empirical eVects of constituency. For example, in StratiWcational
Grammar, the network connections themselves using a special node type (downward and )
represent constituency (see Lamb 1966 or Lockwood 1972 for details). One way of captur-
ing constituency eVects in Connectionist modeling is by enriching the system with a
semantic role network as proposed in Hinton (1981). This kind of approach imports the
insights of various versions of Dependency Grammar (see Chapter 9 for discussion of these
approaches).
12 It is worth clarifying a bit of terminology at this point. People frequently use the
terms ‘‘constituent’’ and ‘‘phrase’’ interchangeably. The reason for this is quite simple: all
phrases are constituents and most constituents are phrases. However, as we will see later in
constituent structure 17
Thomas de Erfurt’s Grammatica Speculativa (c. ad 1300), and perhaps
earlier, although it appears in a more modern form in the ‘‘immediate
constituent’’ analyses of the American Structu ralists in the early part of
the twentieth century (e.g. BloomWeld 1933; for a history of the idea, see
Seuren 1998 and Newmeyer 1986).
We can tentatively deWne constituents as in (20):
(21) Constituents are groups of words that function as units with

13 For more on this test, see Barton (1991).
18 preliminaries
(24) What does Bruce love to do?
(25) (a) Eat at really fancy restaurants.
(b) *Eat at really fancy.
The opposite of the fragment test is checking to see if the string of
words can be omitted or deleted in some way. Starting again with (22),
compare the strings in (26):
(26) (a) really fancy (constituent)
(b) at really (not a constituent).
If we delete (26a) from (22), we get a meaningful sentence, but if we
delete (26b) we get something very odd-sounding indeed:
(27) (a) Bruce loves to eat at restaurants.
(b) *Bruce loves to eat fancy restaurants.
Not all constituents can be deleted, for example in this sentence, verb-
phrase constituents (such as the string [eat at fancy restaurants],
proven to be constituent by the fragment test) are not omissible:
(28) *Bruce loves to.
This is presumably because there are additional requirements at work
here (such as the fact that loves requires a verbal predicate, or the
structure is meaningless).
However, it is frequently the case that constituents can be substituted
for by a single word instead (the replacement test) (Harris 1946).
Usually, a pro-form14 (pronoun, proverb, or proadjective, propreposi-
tion) is used (29):
(29) (a) eating at really fancy restaurants (constituent)
(b) eating at really fancy (not a constituent)
Using the proverb too, the fragment test yields:
(30) (a) Bruce loves [eating at really fancy restaurants] and Dory
loves to [too].