CS 124 / LING 180 From Languages to Information, Dan Jurafsky, Winter 2014



Week 5: Group Exercises on QA: Application to Mobile Devices Feb 11, 2014

Your goal today is to use the personal assistants on your smart phone. Siri, Google Now, etc. If no one in your group has a smart phone, we'll just rearrange the groups a bit. If you have more than one peronsal assistant in your group, compare and contrast!!

  1. Write a couple of texts or emails. What is the speech recognition word error rate (edit distance substitutions + deletions + insertions from the correct error string)? Can you characterize what's going on with the errors?






  2. Make and cancel some calendar appointments. Again, analyze any errors: did they fail because of speech recognition or natural language understanding? If the NLU, what went wrong?






  3. Try to find a business (a restaurant or etc.). Again, analyze any errors: did they fail because of speech recognition or natural language understanding? If the NLU, what went wrong?






  4. Does the speech recognition system allow barge-in?






  5. Make a list of the set of tasks that the dialog manager seems capable of. Anything that you can think of that they didn't accomplish?






  6. Do some error analysis on the TTS. Any problems in pronunciation? If so is it a wrong phone, incorrect stress, or a problem with the prosody (the rhythm/pitch)?






  7. What does the system do if it seems to be unsure what you said? Does it have a strategy for confirming? If so, what is it? Does it always confirm?






  8. Can the system make use of the context (by which I mean specficially the previous question)? If so what aspects of this can the system use?






  9. Think of somehing you might want to do that requires more than one interaction (i.e. you say something, the system says something, you respond). Assuming the system can't do this now, discuss how you might do it.