6.864: Advanced Natural Language Processing

[main] | [general information] | [homeworks] | [exams] | [projects]

Announcements:

[11/27] Solution for homework 4 has been posted.
[11/22] Last year's final exam has been posted.
[11/22] The second midterm is scheduled on the 2nd of December during class. It will be an open book, open notes exam. This midterm covers the following topics: sequence classification (HMMs, log-linear sequence classification), EM for product of multinomials, forward-backward algorithm, topics in MT (sentence-level alignment, word-level alignment: IBM model1 and model2).
[11/21] Solutions for homework 3 have been posted
[11/9] Deadline for the Homework 4 has been extended to Tuesday, November 16th, 5 PM EST.
[11/3] Histogram of the midterm grades.
[11/3] Homework 4 has been posted.
[11/2] The second midterm is scheduled on Dec 2nd and project presentations are scheduled on Dec 7th and 9th.
[10/28] When NLP goes wrong
[10/17] Solutions for homework 1 and 2 have been posted
[10/17] The first midterm is scheduled on the 21st of October during class. It will be an open book, open notes exam. The midterm includes all the material covered in class until the 14th of October. Examples of the past midterms are available on-line.
[10/15] Homework 3 has been posted.
[10/15] Here is the solution to the log-linear question discussed in the class.
[10/7] NLP research is featured in NY Times.
[10/4] Project deadlines are available on projects page.
[9/24] Homework 2 has been posted.
[9/22] This year, MIT is hosting EMNLP, one of the main conferences in statistical natural language processsing. Please consider attending. The conference will be held at the Stata Center on October 9-11.
[9/21] Clarification on the homework: In question 3, you are not supposed to write any code. You only have to provide the shapes of the plots as you think they would look like.
[9/21] The homework is due September 24th, 5PM.
[9/20] The first midterm is scheduled on October 21st during class.

Date	Topic	References
9/8	Introduction and Overview	Lillian Lee I'm sorry Dave, I'm afraid I can't do that: Linguistics, Statistics, and Natural Language Processing circa 2001 The National Academies' study on the Fundamentals of Computer Science
9/14	Language Models	Additional Reading: Stanley F. Chen and Joshua T. Goodman "An Empirical Study of Smoothing Techniques for Language Modeling", Technical Report, Harvard University, 1998.
9/21	Language Models (Cont.)	Notes on ML vs. MAP estimation for the binomial distribution Additional Reading: Wentian Li Random texts exhibit Zipf's-law-like word frequency distribution. IEEE Transactions on Information Theory, 38(6), pp. 1842-1845.
9/16	Topic Classification	Jurafsky&Martin, chapter 12 and 13.
9/23	Syntax: PCFGs	Jurafsky&Martin, Section 20.2.
9/28	Lexicalized PCFGs	Michael Collins "Three Generative, Lexicalised Models for Statistical Parsing." , Proceedings of the ACL/EACL, 1997.
9/30	Lexicalized PCFGs	Michael Collins "Three Generative, Lexicalised Models for Statistical Parsing." , Proceedings of the ACL/EACL, 1997.
10/05	Global Linear Models
10/12	Log-Linear Models	Jurafsky&Martin, chapter 6.6 and 6.7.
10/14	Tagging	Adwait Ratnaparkhi. 1996. A Maximum Entropy Part-Of-Speech Tagger. In Proceedings of the EMNLP.
10/28	EM	Alignment example. Required Reading: notes on EM by Michael Collins.
11/2	EM for product of multinomials	Biological Sequence Analysis by Durbin, Eddy, Krogh, and Mitchison, pp. 323-324. Regina's notes
11/4	Hidden-Markov Models	Jurafsky&Martin, chapter 6.1, 6.2, 6.3 and 6.4.
11/16	Machine Translation	Jurafsky&Martin, chapter 25. P. Koehn's MT tutorial (We looked at the performance slides from this handout) Regina's notes
11/18	Machine Translation	Peter F. Brown, Vincent J.Della Pietra, Stephen A. Della Pietra and Robert L. Mercer "The Mathematics of Statistical Machine Translation: Parameter Estimation" , Computational Linguistics 1993. Regina's notes