CS224U: Natural Language Understanding

This is a short paper (≈6 single-spaced pages) summarizing and synthesizing several papers in the area of your final project.

Groups of one should review 5 papers, groups of two should review 7 papers, and groups of three should review 9.

The ideal is to have the same topic for your lit review and final project, but it's possible that you'll discover in the lit review that your topic isn't ideal for you, so you can switch topics (or groups) for the final project; your lit review will be graded on its own terms. Major things to include (the italicized phrases make good section headings):

1. General problem/task definition: What are these papers trying to solve, and why?
2. Concise summaries of the articles: Do not simply copy the article text in full. We can read them ourselves. Put in your own words the major contributions of each article.
3. Compare and contrast: Point out the similarities and differences of the papers. Do they agree with each other? Are results seemingly in conflict? If the papers address different subtasks, how are they related? (If they are not related, then you may have made poor choices for a lit review...). This section is probably the most valuable for the final project, as it can become the basis for a lit review section..
4. Future work: Make several suggestions for how the work can be extended. Are there open questions to answer? This would presumably include how the papers relate to your final project idea.
5. References section: The entries should appear alphabetically and give at least full author name(s), year of publication, title, and outlet if applicable (e.g., journal name or proceedings name). Beyond that, we are not picky about the format. Electronic references are fine but need to include the above information in addition to the link.

This is a short, structured report designed to help you establish your core experimental framework. Required pieces:

1. Hypotheses: A statement of the project's core hypothesis or hypotheses.
2. Data: A description of the dataset(s) that the project will use for evaluation.
3. Metrics: A description of the metrics that will form the basis for evaluation. We sort of expect these to be familiar quantitative metrics, but we're open-minded – non-standard evaluations and qualitative evaluations are welcome where appropriate, though your mentor is likely to push you toward hypotheses that can be assessed quantitatively.
4. Models: A description of the models that you'll be using as baselines, and a preliminary description of the model or models that will be the focus of your investigation. At this early stage, some aspects of these models might not yet be worked out, so preliminary descriptions are fine.
5. General reasoning: An explanation of how the data and models come together to inform your core hypothesis or hypotheses.
6. Summary of progress so far: what you have been done, what you still need to do, and any obstacles or concerns that might prevent your project from coming to fruition.
7. References section: In the same format as for lit review.

These will be 4-minute videos submitted via video link – for example, a public or private YouTube link.

Your video needs to report initial results – at least the results for your baseline models and some preliminary numbers for the models that you are focused on.

We'll provide additional guidance during the May 29 lecture on presenting research in NLP. A selection of exemplary video presentations from prior years of this course is available here (link restricted to enrolled students).

The paper should be 8 pages long, in ACL submission format. Here are the LaTeX and Word templates for the current ACL style. Please submit the paper via the course's Canvas site.

We'll provide additional guidance during the May 29 lecture on writing up research in NLP. The course readings include many exceptionally good examples of NLP papers in this format. A selection of exemplary papers from prior years of this course is available here (link restricted to enrolled students).

Update (May 27): At the end of your paper (after the 'Acknowledgments' section in the template), please include a brief authorship statement, explaining how the individual authors contributed to the project. You are free to include whatever information you deem important to convey. For guidance, see the second page, right column, of this guidance for PNAS authors. We are requiring this largely because we think it is a good policy in general. This statement is required even for singly-authored papers, because we want to know whether your project is a collaboration with people outside of the class. Only in extreme cases, and after discussion with the team, would we consider giving separate grades to team members based on this statement.