PSYCH 227 / LINGUIST 247 Seminar in Psycholinguistics: Information-Theoretic Models of Language and Cognition Autumn 2009

COURSE INFORMATION
Instructors	Dan Jurafsky, jurafsky@stanford.edu   Office: Margaret Jacks (bld 460) 113 Office Hours: TBA Michael Ramscar, michael at psych dot stanford dot edu Office Hours: TBA
Place/Time	126 Margaret Jacks Hall, Tuesday/Thursday 3:15-4:30pm
Description	Information theory and its relation to learning and to reference, meaning, and information encoding. Information-theoretic models of linguistic structure at the phonological, morphological, and syntactic levels, and the links between information theory and information structure/clause packaging. The role of uncertainty and uniform information density in sentence processing
Required Work	• 50%: Final project Final project due date: Wednesday, December 9, 5pm, by email Final project details here • 25%: Presentation of two papers • 25%: Participation in discussion (including posting notes about the reading)
More on the Final Project

SCHEDULE
Wk	Date	Who	Topic and Readings
1	Sep 22	Ramscar/ Jurafsky	Planning meeting, course topics, overview of big questions
1	Sep 24	Richard (for Goldsmith)	Probability and Information Theory For those with no probability theory or information theory, start with: John A. Goldsmith. 2007. Probability for linguists. For those with no information theory, the above plus: Sheldon Ross. 2010. A First Course in Probability. Eigth Edition. Section 9.3 "Surprise, Uncertainty, and Entropy", pages 425-429. For those with no (or rusty) probability theory, the above two plus: The first few chapters of any good probability book, like Ross, or this online pdf text or this online html text For those who already know information theory and probability: Robert A. Rescorla. 1988. Pavlovian Conditioning: It's Not What You Think It Is. American Psychologist, 43(3), 151-160
2	Sep 29	Ben, Carl	Symbols and Prediction Fodor, Jerry A. and Zenon W. Pylyshyn. 1988. Connectionism and Cognitive Architecture: A Critical Analysis. In S. Pinker and J. Mehler, eds., Connections and Symbols, Cambridge, MA: MIT Press. The final pdf is available from Elsevier here. Ramscar & Dye. manuscript
2	Oct 1	Nat	Symbols and Prediction Michael Ramscar, Daniel Yarlett, Melody Dye, Katie Denny, and Kirsten Thorpe. 2009 manuscript. The Effects of Feature-Label-Order And Their Implications For Symbolic Learning
3	Oct 6	DaveA, Carl	Shannon, the Shannon Game, and the Entropy of English Shannon, C. Prediction and entropy of printed English. Bell System Technical Journal, 30, 50-64. Thomas M. Cover and Roger C. King. 1978. A Convergent Gambling Estimate of the Entropy of English. IEEE Transactions on Information Theory 24:4, 413-421.
3	Oct 8	Tyler, Chigusa	Zipf and Least Effort Least Effort and why frequent words and morphemes are short: Pages 1-8 and 19-22 (DONT READ 22-31, that's for later) from Zipf, George K. (1949). Human behaviour and the principle of least effort: An introduction to human ecology. New York: Hafner. Pages 20-39 and 172-176 from Zipf, George K. (1935). The psycho-biology of language: An introduction to dynamic philology. Houghton Mifflin. Why frequent phonemes are more likely to change: Pages 73-81 and 109-121 from Zipf, George K. (1935). The psycho-biology of language: An introduction to dynamic philology. Houghton Mifflin. Pages 98-108 from Zipf, G. (1949). Human behaviour and the principle of least effort: An introduction to human ecology. New York: Hafner. Velocity of communication (included in PDF for #2 above): Pages 283-287 from Zipf, George K. (1935). The psycho-biology of language: An introduction to dynamic philology. Houghton Mifflin.
4	Oct 13	Chigusa, ???	Predictability and Phonetic Reduction Aylett, M. and Turk, A. (2004). The smooth signal redundancy hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1), 31-56. Van Son, R., & Pols, L. (2003). How efficient is speech? Proceedings of the Institute of Phonetic Sciences, 25, 171– 184.
4	Oct 15	Tanja, Richard	Predictability and Phonetic Reduction Pluymaekers, M., Ernestus, M., & Baayen, R. (2005). Lexical frequency and acoustic reduction in spoken Dutch. The Journal of the Acoustical Society of America, 118, 2561 Alan Bell, Jason Brenier, Michelle Gregory, Cynthia Girand, and Dan Jurafsky. 2009. Predictability Effects on Durations of Content and Function Words in Conversational English. Journal of Memory and Language 60:1, 92-111.
5	Oct 20	Tejaswi, Justine	Uniformation Information Density across a Discourse Genzel, D. and Charniak, E. (2002). Entropy rate constancy in text. In Proceedings of ACL-02. Keller, F. (2004). The entropy rate principle as a predictor of processing effort: An evaluation against eye-tracking data. In In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Barcelona, pp. 317-324.
5	Oct 22	Robin,DaveC	Uniform Information Density in Syntax Hale, John. 2003. The Information Conveyed by Words in Sentences. Journal of Psycholinguistic Research 32:2. 101-123 T. Florian Jaeger. 2009 manuscript. Redundancy and Reduction: Speakers Manage Syntactic Information Density.
6	Oct 28	Richa,Karthik,Richa	Optimal Lexicons Piantadosi, Tily, Gibson (2009). The communicative lexicon hypothesis. Proceedings of Cogsci 2009. Terry Regier, Paul Kay, and Naveen Khetarpala. 2007. Color naming reflects optimal partitions of color space. PNAS 4:4, 1436-1441. Roland Baddeley and David Attewell. 2009. The Relationship Between Language and the Environment: Information Theory Shows Why We Have Only Three Lightness Terms. Psychological Science 20:10, 1100-1107.
6	Oct 30	Justine, Carl	Expectation and Meaning Hagoort, P., and Van Berkum, J. J. A. (2007). Beyond the sentence given. Philosophical Transactions of the Royal Society. Series B: Biological Sciences, 362, 801-811 Ramscar, M., Matlock, T., & Dye, M. (in press) Running down the clock: the role of expectation in our understanding of time and motion. Language and Cognitive Processes.
7	Nov 3	Tyler,DaveC	Information Structure I Prince, Ellen F. 1981b. Toward a taxonomy of given/new information. In Peter Cole, ed., Radical Pragmatics. New York: Academic Press. 223-54. Ariel, M. 2001. Accessibility theory: An overview. In T. Sanders, J. Schilperoord, & W. Spooren (Eds.), Text representation: Linguistic and psycholinguistic aspects (p. 29- 87). Amsterdam: John Benjamins.
7	Nov 5	Tanya,Karthik	More on Referring Expressions Brennan, S., and Clark, H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology, 22, 1482-1493. Tily & Piantadosi (2009). Refer efficiently: Use less informative expressions for more predictable meanings. Proceedings of the Workshop on Production of Referring Expressions, Cogsci 2009.
8	Nov 10	Tejaswi,DaveA	Information Structure and Verb Predictability Prince, Ellen F. 1992. The ZPG letter: Subjects, definiteness, and information-status. In S. Thompson and W. Mann, eds., Discourse Description: Diverse Analyses of a Fundraising Text. Amsterdam/Philadelphia: John Benjamins. 295-325. Philip Resnik. 1996. Selectional Constraints: An Information-Theoretic Model and its Computational Realization. Cognition 61:127-159.
8	Nov 12	Richard,Robin	Markedness and Iconicity Martin Haspelmath. 2008. Frequency vs. iconicity in explaining grammatical asymmetries. Cognitive Linguistics 19.1:1-33. Martin Haspelmath. 2006. Against markedness. Journal of Linguistics 42.1: 25-70
9	Nov 17	Richard,Nat	Phonology and Morphology Jason Riggle and John Goldsmith. 2007 ms. Information theoretic approaches to phonological structure: the case of Finnish vowel harmony. MS under review Milin, P., Kuperman, V., Kostic, A. & Baayen, R.H. Paradigms bit by bit: an information- theoretic approach to the processing of inflection and derivation. In press in Blevins, James P. and Juliette Blevins (eds.), Analogy in Grammar: Form and Acquisition. Oxford: Oxford University Press.
9	Nov 19	Uriel	Phonology and Morphology Uriel Cohen Priva. 2008. Using Information Content to Predict Phone Deletion. Proceedings of the 27th West Coast Conference on Formal Linguistics, pp 90--98.
10	Dec 1	Karthik/Richa/Tejaswi,Justine,Nat	Final Project Presentations
10	Dec 3	DaveC,Chigusa,Richard,DaveA	Final Project Presentations

Students with Documented Disabilities: Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Student Disability Resource Center (SDRC) located within the Office of Accessible Education (OAE). SDRC staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is being made. Students should contact the SDRC as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066)