6.864: Natural Language Processing

[main] | [about] | [homeworks] | [links]

Instructors: Regina Barzilay, Michael Collins,
Time & Location: Tues & Thurs 1-2.30, 32-155
Office Hours: By appointment

Announcements:

[12/15] An example final (from 2005) is here
[12/12] Please complete the online class evaluation, from the Underground Guide, at this address! Thanks!
[11/28]Homework #5 is released
[11/13] A histogram of the midterm exam scores. The scores are listed in descending order. Midterm solutions.
[11/12]Homework #4 is released
[10/29] The policy for the midterm and the final exam is to allow you to bring lecture notes from class, as well as any additional notes you've written yourself to supplement this material. We will not allow you to bring other materials to the exam (for example, you can't bring an entire book such as Manning and Schuetze, nor can you bring individual chapters from a book).
[10/28] Homework #3 is released
[10/27] Note on the mid-term: the material covered by the mid-term can potentially include anything up to and including the lecture on 10/26. The practice mid-term has some questions which are on material that has not been covered yet this year; specifically, you should ignore Part 1 question 2, and the whole of part 3. In general, the questions you've seen on psets 1 through 3 should be a good indication of the kind of questions you'll see on the mid-term.
[10/26] Midterm location: 56-114, time: 7:30-9:30 on 11/2
[10/26] Practice midterm and its solution
[10/17] TA office hours are Monday 5-6 and Wednesday 4-5 in the fourth floor lounge of the Gates tower of Stata.
[10/12] The midterm for the class is scheduled on 11/2.
[10/6] Homework #2 is released
[9/22] We've now added the latex source for homework 1 to the homeworks page, in case it's useful.
[9/19] Homework #1 is released
[9/7] Access to Manning&Schutze textbook
[9/7] Link to the 2005 webpage of 6.854

Date	Topic	References
9/7	Introduction and Overview	Lillian Lee I'm sorry Dave, I'm afraid I can't do that: Linguistics, Statistics, and Natural Language Processing circa 2001 To appear in the National Academies' study on the Fundamentals of Computer Science Wentian Li Random Texts Exhibit Zipfs's-Law-Like Word Frequency Distribution IEEE Transactions on Information Theory
9/12	Parsing and Syntax I	pdf slides, pdf slides (4 per page)
9/14	Smoothed Estimation, and Language Modeling	pdf slides, pdf slides (4 per page) Background Reading: An Empirical Study of Smoothing Techniques for Language Modeling. Stanley Chen and Joshua Goodman. 1998. Harvard Computer Science Technical report TR-10-98. Background Reading: On the Convergence Rate of Good-Turing Estimators. David McAllester and Robert E. Schapire. In Proceedings of COLT 2000. Note: the two "background reading" papers are provided because we think they may be useful and/or interesting to you. We don't expect you to necessarily understand the material in the background readings -- the lecture slides contain all material that is required for the class. The first reading (Chen and Goodman) gives a very comprehensive overview of different smoothing techniques. The second reading (McAllester and Schapire) gives proofs regarding the Good-Turing estimators.
9/19	Parsing and Syntax II	pdf slides, pdf slides (4 per page) Background Reading: Statistical techniques for natural language parsing. Eugene Charniak. 1997. In AI Magazine. Background Reading: Head-Driven Statistical Models for Natural Language Parsing. Michael Collins. 2003. In Computational Linguistics.
9/21	Parsing and Syntax III	pdf slides, pdf slides (4 per page) (Note: we spent the first hour of lecture completing the slides from 9/19) Background Reading: Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency. Dan Klein and Chris Manning. 2004. In Proceedings of the 42nd Annual Meeting of the ACL, 2004. Some people in class expressed an interest in unsupervised learning of parsing models (i.e., learning without a treebank). This paper covers that topic. Again, as background reading you're not expected to know this material, it's posted in case you find it of interest/relevance.
9/26	Log-Linear Models	pdf slides, pdf slides (4 per page)
9/28	Tagging	pdf slides, pdf slides (4 per page) (Note: we didn't cover history-based models, i.e., we stopped at slide 46.) Background Reading: Daniel M. Bikel, Richard Schwartz and Ralph M. Weischedel. 1999. An Algorithm that Learns What's in a Name. In Machine Learning, Special Issue on Natural Language Learning. Background Reading: Andrew McCallum, Dayne Freitag and Fernando Pereira. Maximum Entropy Markov Models for Information Extraction and Segmentation. In proceedings of ICML 2000.
10/3	Distributional Similarity and Clustering	Ido Dagan, Lillian Lee and Fernando Pereira Similarity-Based Methods for Word Sense Disambiguation, in proceedings of ACL 1997. Scott Miller, Jethran Guinness and Alex Zamanian Name Tagging with Word Clusters and Discriminative Training, In proceedings of NAACL/HLT 2004.
10/5	Unsupervised Vocabulary Induction	Rie Kubota Ando and Lillian Lee Mostly-Unsupervised Statistical Segmentation of Japanese: Applications to Kanji., in proceedings of NAACL 2000. Alex Park and James Glass Towards Unsupervised Pattern Discovery in Speech, In proceedings of ASRU 2005.
10/12	Word Sense Disambiguation	David Yarowsky Decision Lists for Lexical Ambiguity Resolution: Application to Accent Restoration in Spanish and French. , in proceedings of ACL 1994. David Yarowsky Unsupervised Word Sense Disambiguation Rivaling Supervised Methods. , In proceedings of ACL 1995.
10/17	History-Based Models, and Global Linear Models	pdf slides, pdf slides (4 per page)
10/19	History-Based Models, and Global Linear Models	Continued the slides from last lecture
10/24	Global Linear Models Part II	pdf slides, pdf slides (4 per page)
10/26	Text Segmentation	Marti Hearst Multi-paragraph Segmentation of Expository Text , in proceedings of ACL 1994. Igor Malioutov and Regina Barzilay Minimum Cut Model for Spoken Lecture Segmentation. , in proceedings of ACL-COLING 2006. The MIT Lecture Browsing and Retrieval Tool.
10/31	Graph-Based Approaches	Jon Kleinberg Authoritative sources in a hyperlinked environment , in proceedings of SODA 1998. LexRank Demo
11/7	Summarization and Generation: Content Selection	J. Kupiec, J. Pedersen and F.R. Chen A trainable document summarizer , in proceedings of SIGIR 1995. K. Knight and D. Marcu Statistics-Based Summarization --- Step One: Sentence Compression, in proceedings of AAAI 2000.
11/9	Summarization and Generation: Aggregation and Ordering	Regina Barzilay and Mirella Lapata Collective Content Selection for Concept-To-Text Generation , in proceedings of EMNLP 2005. Regina Barzilay and Mirella LapataAggregation via Set Partitioning for Natural Language Generation, , in proceedings of HLT/NAACL 2006.
11/14	The EM Algorithm (Part I)	pdf slides, pdf slides (4 per page)
11/16	The EM Algorithm (Part II)	pdf slides, pdf slides (4 per page) Required reading for the class: notes on EM
11/21	Dialogue Systems	Scott Miller, David Stallard, Robert Bobrow and Richard Schwartz A Fully Statistical Approach To Natural Language Interfaces , in proceedings of ACL 1996 Diane J. Litman, Michael S. Kearns, Satinder Singh and Marilyn A. Walker Automatic optimization of dialogue management , in proceedings of ACL 2000.
11/23	Machine Translation (Part I)	pdf slides, pdf slides (4 per page)
11/28	Machine Translation (Part II)	pdf slides, pdf slides (4 per page)