READ THIS ENTIRE PAGE IMMEDIATELY, AND BEFORE DOING ANYTHING.

In this homework you will be judging the quality of state-of-the-art "conversational agents"; systems which carry on spoken-language dialogues with humans.

1. Call the "TellMe" speech recognizer. The number is 1-800-555-TELL. Try at least two different kinds of questions (sports, stock quotes, entertainment, travel, etc). Try to "barge in" (i.e. talk while TellMe is talking to you). Write down any errors in the speech recognition ("misrecognitions") or speech synthesis (weird pronunciations) or any thing unnatural in the dialogue that occurs. Write a short description and summary of these errors.
2. Call the United Airlines automatic flight information, at 1.800.824.6200. Ask about a flight. Again, write down any errors, misrecognitions, or unnatural incidents that occur. Write a short description and summary of these errors.
3. The third part of the homework is to judge the quality of one particular component of a conversational agent: the "speech synthesis". To do this, you will be a participant in the Blizzard Challenge 2005. This is the annual "bake-off" at which different labs around the world who work on text-to-speech synthesis all compete to see how natural speech they can generate. The information on how to participate is below. There are two "stages". For the homework for this class, you are only required to participate in stage one. In fact, in addition to everything you will learn from this homework, and fulfilling the requirement, this homework has the advantage that they will pay you $5 for participating in stage one. Stage two is optional (but they will pay you an additional $10 for participating in stage two). The two things to turn in for this problem: print out the page just after completing the "feedback questionnaire" to show that you participated (make sure you answer all the feedback questions), and also write down 5 examples of errors or strange pronunciations or in general unnatural synthesis examples that you heard. Try to describe carefully what was unnatural about the examples.

WHAT TO TURN IN:
Turn in each problem on 3 separate pieces of paper stapled together. Put your name on each page. Problem 1 on the first page, Problem 2 on the second page, and then the rest of the pages should include the post-feedback-questionnaire page from the Blizzard challenge plus the 5 errors.

The following is the announcement from the Blizzard Challenge people which describes what they want you to do.

The Blizzard Challenge 2005:
Evaluating corpus-based speech synthesis on common databases
http://www.festvox.org/blizzard/

Speech Synthesis Listening Tests:

In order test synthetic speech generated on computers we would like a number of North American English native speakers in the age range of 18-24 to listen to a set of synthetic utterances built from different voice building techniques.

Each stage will take around 30-60 minutes, and we ask for participants to take part in two stages. We will send participants a $5 Amazon.com voucher for completing the first stage, and an additional $10 Amazon.com voucher for completing the second stage, which should be carried out one to seven days after the first.

Each stage consist of five tests which play various waveform files to the listener who then is asked to score them on a 1-5 scale, or in some tests type in what they hear.

To carry out the listening test please go to

http://www.speech.cs.cmu.edu/blizzard/register-U.html

You will be asked to register, then listen to and evaluate a set of sentences. We find it takes around 30-60 minutes to complete all the tests. It is good if you can carry this out in a single session, but you can take a break and return later, continuing from where you left off.

After completing the first stage you must wait at least one day before you continue with the second stage.