Probabilistic Methods in Computational Psychlinguistics

Instructor: Roger Levy
Dates: 7-11 August 2006
Classroom: 3.06

This is the course website for the ESSLLI 2006 course Probabilistic Methods in Computational Psycholinguistics running from 7-11 August 2006 at the University of Malaga. At its core, the processing of natural language involves dealing with uncertainty all the time, and in psycholinguistic research probability theory is playing a larger and larger role in modeling how people deal with this uncertainty. This course will cover a variety of probabilistic approaches; the empirical data covered will be tightly focused on sentence processing and specifically syntactic comprehension, but the probabilistic models presented here have broad applicability to many problems in psycholinguistics.

The course is now finished. Thank you to everyone who attended!

Class schedule

Date Topic Required reading Related reading
7 Aug 2006
Introductory Material & Pruning Approaches Jurafsky 1996 (focus on Section 4)
Narayanan and Jurafsky 1998
Probability theory refresher:
Manning & Schuetze, chapter 2, or Goldsmith tutorial
8 Aug 2006
Competition Models McRae et al. 1998
Green and Mitchell 2006

MacDonald et al. 1994
Spivey and Tanenhaus 1998
Ferretti and McRae 1999
9 Aug 2006
Reranking and attention-shift approaches. Crocker and Brants 2000
Narayanan and Jurafsky 2002

Narayanan and Jurafsky 2004
10 Aug 2006
Information-theoretic approaches Hale 2001
Levy 2006 (focus on sections 5 and 6)
Hale 2003

Den and Inoue 1997
Keller 2004
Hale 2006
11 Aug 2006
Connectionist approaches Christiansen and Chater 1999 Elman 1991
Tabor, Juliano and Tanenhaus 1997
Tabor and Tanenhaus 1999
Steedman 1999
Rohde 2002

Lecture Slides

Here are the final slides from the course lectures. Feel free to contact me if you have any follow-up questions about them or about other aspects of the course!

[Day 1|Day 2|Day 3|Day 4|Day 5]

(Note that there isn't a strict one-to-one correspondence between the original class schedule and which day a particular topic ultimately got covered.)

Readings and other references

The class schedule lists required readings and also related/background readings for each topic. Each lecture will focus on the required readings, and it will be assumed that you have done these readings before class and are ready to discuss them. The related readings are provided in case you are interested in further pursuing one or more of the topics covered in the class.

The readings listed above, together with some other useful references, can be found here. Many of these readings are available online. Please email me if you are unable to obtain any of the required readings.


Here is some software we'll be using in the class:

An implementation of the Spivey-Knowlton (1996) normalized recurrence algorithm, to be covered in the section on competition models (8 August). It will run out of the box on Linux; for other operating systems, you will need to recompile.

A prefix probability parser, to be covered in the section on information-theoretic models (10 August). To use this parser you will need to install Java (version 1.4 or later) on your computer.

Background and other recommended courses

This is an advanced course, and it will be taught assuming some familiarity with several topics including probability theory, natural language syntax, and cognitive psychology. At the beginning of the course there will be a (very!) brief review of this foundational material.

If you're planning on taking this course, there are several other courses at ESSLLI 2006 that might be of interest to you. These include:

The course descriptions can be found here.