6.864: Advanced Natural Language Processing

[main] | [general information] | [homeworks] | [exams]

Announcements:

[11/3] Problem set 4 has been released. Due date: November 13, 5pm.
[10/26] Sample project ideas have been posted.
[10/21] Important information about the project and other deadlines has been posted.
[10/21] Problem Set 3 has been released. Due date: November 2, 5pm.
[10/20] This year's exam and solutions have been posted.
[10/6] Previous year exams have been posted.
[10/2] Problem Set 2 has been released. Due date: October 19, 5pm.
[9/25] Project proposals will be due ~~October 27~~ November 5.
[9/18] Problem Set 1 has been released. Due date: October 2, 5pm.
[9/18] Kevin Knight(USC) will give a talk "The Voynich Manuscript" on September 23rd. It should be a great lecture -- the topic is fascinating and Kevin Knight is an amazing speaker. I strongly recommend attending it. This talk may also give you some ideas for your project.
[9/17] The first midterm is scheduled on the 15th of October during class. It will be an open book, open notes exam.

Date	Topic	References
9/4	Introduction and Overview	Lillian Lee I'm sorry Dave, I'm afraid I can't do that: Linguistics, Statistics, and Natural Language Processing circa 2001 The National Academies' study on the Fundamentals of Computer Science
9/15	Language Models	Additional Reading: Stanley F. Chen and Joshua T. Goodman "An Empirical Study of Smoothing Techniques for Language Modeling", Technical Report, Harvard University, 1998.
9/17	Language Models (Cont.)	Additional Reading: Wentian Li Random texts exhibit Zipf's-law-like word frequency distribution. IEEE Transactions on Information Theory, 38(6), pp. 1842-1845.
9/22	Sampling	Additional Reading: Chapter 11 of Bishop, Pattern Recognition and Machine Learning
9/24	Language Models (Cont.) and estimation	Notes on ML vs. MAP estimation for the binomial distribution
9/29	Morphology	Jurafsky&Martin, chapter 3.1 and 3.12. Rie Kubota Ando and Lillian Lee. "Mostly-Unsupervised Statistical Segmentation of Japanese: Applications to Kanji.", Proceedings of NAACL, 2000.
10/1	Morphology	M. Creutz "Unsupervised segmentation of words using prior distributions of morph length and frequency." , Proceedings of ACL, 2003. David Yarowsky, Grace Ngai, and Richard Wicentowski "Inducing Multilingual Text Analysis Tools via Robust Projection across Aligned Corpora." , Proceedings of HLT, 2001. (section 7 only) Some FAQs from this class
10/6	Syntax: PCFGs	Jurafsky&Martin, chapter 12 and 13.
10/8	Syntax: Parsing	Michael Collins "Three Generative, Lexicalised Models for Statistical Parsing." , Proceedings of the ACL/EACL, 1997.
10/15	Midterm with Solutions
10/20	Log-linear Models	Jurafsky&Martin, chapter 6.6 and 6.7. S.R.K. Branavan, Harr Chen, Luke Zettlemoyer. Regina Barzilay. Reinforcement Learning for Mapping Instructions to Actions. In Proceedings of ACL 2009.
10/22	Log-linear Models (cont.)	Jurafsky&Martin, chapter 6.6 and 6.7.
10/27	Tagging	Adwait Ratnaparkhi. 1996. A Maximum Entropy Part-Of-Speech Tagger. In Proceedings of the EMNLP.
10/29	No class, project meetings
11/3	EM	Required Reading: Additional notes on EM by Michael Collins.
11/5	EM part 2	Biological Sequence Analysis by Durbin, Eddy, Krogh, and Mitchison, pp. 323-324.
11/10	EM part 3
11/12	HMMs