\documentclass[12pt,letterpaper]{article}
\usepackage{graphicx,textcomp}
\usepackage{natbib}
\usepackage{setspace}
\usepackage{fullpage}
\usepackage{color}
\usepackage[reqno]{amsmath}
\usepackage{amsthm}
\usepackage{amssymb,enumerate}
\usepackage[all]{xy}
\usepackage{endnotes}
\usepackage{lscape}
\newtheorem{com}{Comment}
\newtheorem{lem} {Lemma}
\newtheorem{prop}{Proposition}
\newtheorem{thm}{Theorem}
\newtheorem{defn}{Definition}
\newtheorem{cor}{Corollary}
\newtheorem{obs}{Observation}
\usepackage[compact]{titlesec}
\usepackage{dcolumn}
\usepackage{tikz}
\usetikzlibrary{arrows}
\usepackage{multirow}
\usepackage{xcolor}
\newcolumntype{.}{D{.}{.}{-1}}
\newcolumntype{d}[1]{D{.}{.}{#1}}
\definecolor{light-gray}{gray}{0.65}
\usepackage{url}
\newcommand{\Sref}[1]{Section~\ref{#1}}
\newtheorem{hyp}{Hypothesis}

\title{Text as Data: Homework 5}
\date{Due: 12/5 at 5pm}

\begin{document}
\maketitle

In this homework we will analyze a collection of news stories from the New York Times from the November 1-3, 2004 (the day before, of, and after the 2004 general election). This data come from the New York Times Annotated Corpus and is for academic use only. We have done some preprocessing in order to simplify the homework tasks.


\section{Preprocessing and Creating a Document-Term Matrix}

\begin{itemize}
\item[a)] From the course website, download {\tt nyt\_ac.json} 
\item[b)] Using the {\tt JSON} library in python, import the data. Use {\tt type} to explore the structure of this data. How are this data organized?
\item[c)] Extract the title and text from each story.  Create an individual document for each story and write each of the files to a new directory (we will use this later to run {\tt Mallet})
\item[d)] Using the loaded {\tt json} file, create a document term matrix of the 1000 most used terms.  Be sure to: 
\begin{itemize}
\item[-] Discard word order
\item[-] Remove stop words
\item[-] Apply the porter stemmer
\end{itemize}
\item[e)] Include in your document-term matrix the \emph{desk} from which the story originated, which we will include later
\end{itemize}


\section{Using {\tt MALLET} to Fit Topic Models}

\begin{itemize}
\item[a)] Go to {\tt http://mallet.cs.umass.edu/download.php} and install {\tt MALLET} on your computer
\item[b)] Following the syntax from the code posted on the course website (flake.mallet) and from the {\tt MALLET} website, apply LDA with 4 and 8 topics
\item[c)] Using the code from the course website {\tt Examp\_tc11.R} move the output of {\tt MALLET} into {\tt R} and create a table describe both versions of LDA
\item[d)] Compare the 4 and 8 topic models.  How do the 4 and 8 topic models differ?  What information is included in the 8 topic model that isn't in the 4 topic model
\item[e)] For each originating desk, calculate the average proportion of documents from a desk dedicate to each topic.  How does the topic attention differ across desks?
\end{itemize}


\section{Using the {\tt Structural Topic Model} in {\tt R}}

\begin{itemize}
\item[a)] Download the {\tt stm} package for {\tt R} from {\tt CRAN}
\item[b)] Convert the document-term matrix to the appropriate format.  To do this, create a list in {\tt R} where each component of the list corresponds to an individual document.  Store in each component of the list a two rom matrix.  The number of columns corresponds to the number of non-zero entries for the document in the document-term matrix.  The first row will describe the words used in the document (the columns with the non-zero entry).  The second row will correspond to a count of each of the words in the document (they should all be non-zero)
\item[c)] Following the help file in {\tt STM} fit a model with 8 topics that conditions on the {\tt desk} of origin for topic prevalence
\item[d)] Use {\tt labelTopics} to label each of the topics
\item[e)] Compare the 8 topic proportions for each document to the 8 topic proportions without conditioning on {\tt desk} (in vanilla LDA).  How do the results differ?
\end{itemize}


\section{Supervised Learning with Naive Bayes}

\begin{itemize}
\item[a)] Using the version of Naive Bayes outlined on slide 24 of lecture 14, write a function to estimate $p(C_{k})$ and $\boldsymbol{\theta}_{k}$ for an arbitrary collection of categories.  Hint: to compute the probability of a document from a category, note you can work with the log of the probability equivalently.  
\item[b)] Let's focus on documents that came from Business/Financial desk and National Desk.  Using leave-one out cross validation, calculate the accuracy of Naive Bayes to calculate the label.
\item[c)] Compare the performance of Naive Bayes to the performance of 2 of the following 3 algorithms using 10-fold cross validation:
\begin{itemize}
\item[-] LASSO 
\item[-] Ridge 
\item[-] KRLS 
\end{itemize}
How does Naive Bayes compare? 
\end{itemize}




\end{document}
