In P3, we brainstormed different application areas and selected cooking as our initial application area and we identified the real needs of busy home cooks as being able to multi-task and to simplify the cooking process (such as follow the recipe easily). However, after watching all the P3 skits, we found the application area of cooking was a common choice, and many teams have targeted at similar needs such as multi-tasking and getting better cooking instructions. So we decided to conduct another round of brainstorming/bodystorming, and see if we could come up with some more compelling use case for cooking, or even a brand-new application area. Leveraging the thought of “busy people want to be able to multi-task”, we explore a list of different application areas and finally decided to change our application area to Spa(an area all of us are very excited about!).

Originally we envisioned our app to help busy professionals stay productive while in hottub/spa, since they might be normally too busy to enjoy a spa/bathtub. However after going through the needfinding process by talking to people who have been to spa/hottub and conducting an online survey, we have the following findings:

- many people go to spa or use bathtub to relax themselves

- what they would do in the spa/bathtub...

• most of them listen to music
• quite many would like to make phone calls, text friends and check social media
• some of them check emails, do readings and work

- they carry devices with them to do all the above, such as smartphone, tablet or even laptop

- they touch their devices carefully because their hands and body are wet

- they tend to stay at one location and relax themselves without much movement

- they hold the device close to them while read from it, the screen of the device is too small to see from a distance

- where to put the devices? We collected a range of answers about where people put their device:

• hold it all the time
• put it on the table/shelf far away from the water(but not able to control it)
• put it in waterproof bag
• try to utilize a whole in the wall(?!)

From the findings, we realized that people need to entertain themselves and stay connected while in spa or bathtub in a relaxing way.

The challenge is: how might we create a hassle-free, entertaining, and personalized atmosphere for bathtub and spa users?

Design idea initial brainstorm - interactive spa experience

What is Spaatify?

Spaatify provides a way for users to enjoy a bath or a spa, while staying connected and being entertained through Kinect gesturing technology and their mobile device. When a user enter a spa, she can connect her device (such as smartphone, tablet or laptop) with Spaatify. Spaatify would allow the user to control her device with gestures and project the content on a large screen. (Maximizing enjoyable spa experience with no electric shock involved :P)

Features

• provide entertainment to user such as playing music/movie
• allow user to multi-task (provides access to VoIP calls, contacts, texts, emails, facebook, twitter, web browsing)
• mobile device is the source of data
• huge projection screen for easy viewing
• simple and intuitive gesture to control the device

Functionalities:

1. access to personal music library (music control)
2. make or answer phone calls
3. check emails
4. access to facebook
5. watch movies
6. read newspapers
7. listen to radio

Meeting our design coach - Doug

We met our coach Doug and discussed the ideas we have: we realized that although we have designed a couple of cool functionalities people can use, such as picking up phones, reading emails, listening to music, we are not crystal clear about what should be the core functionality of our app, what should be that one major thing we are going to give our users. Doug also shared with us his own experience and envision, which helps us the rethink what the real need is.

We rethought our core idea: instead of going to the multitasking idea (letting people reading emails and making phone calls) while in the tub, we ask ourselves: what is the part people love most about spa? It's the feeling of being in a relaxing and care-free space. Therefore, we are going to provide them the environment to relax. Also, we redefined our space of usage: instead of setting the device in a real spa, we try to set it in personal bathtub. The design idea is to create a relaxing and soothing environment so that you can have the same enjoyable “spa” experience in your own bathtub. The reason is that we realized people who can go to professional spa place, are already pretty relaxed there, whereas a lot of people do not go there because it’s expensive/far and people don’t have time to go. By creating the right environment and interaction, “spa” experience can be accessed by anyone who has a bathtub so they can relax themselves.

Design idea brainstorm TAKE 2 - customizing your personal relaxing space

What is Spaatify?

Spaatify provides a personalized space for users to enjoy a bath or a spa. When a user enter a spa, she can customize the space with Spaatify according to her mood. Spaatify would allow the user to interact with the relaxing space with gesture to enjoy the visuals on a large screen and control their favourite music.

Features

• allow users to customize the space according to their mood
• keep things simple - create a simple yet relaxing environment
• huge projection screen for easy viewing
• simple and intuitive gestures to navigate through the screen and control music volume

Interactive Space: Now the environment is personalized, the user is already to enjoy a relaxing moment in the bathtub. While the user is sitting in the tub, she can interact with the space with Spaatify. There are several interactions we have brainstormed to enhance user's experience in bathtub --

• Abtract art - provides a way to blend user's gestures to art and music

• Game - provides a simple yet relaxing game to keep the user entertained

• News feed - displays an array of news feeds of different categories

• Music control - allows users to dj their own music with gestures

• Pictures browsing - allows users to browse the selected albums of different themes

• Light - adjusts the hue and brightness of the environment with user's gestures

• 3D space exploration - allows the user to explore the world in 3D (with google earth possibly)

• Space exploration - allow the user to explore the universe with informative pop-ups

Design prototype of shortlisted ideas

After several rounds of brainstorm/bodystorm, we came up with 8 different design ideas for the spa/bathtub users to interact with the space. Based on the level of engagement and interactivity, we have shortlisted the following four ideas for initial prototyping. These ideas are

1. Space Exploration
2. News Feed Browsing - Tile Display
3. News Feed Browsing - Cover Flow
4. Abstract Art Creation

1. Space Exploration

- First, the user customizes the environment with her mood of the day with one facial expression

• 1.1 Customizing the space with an expression that represents your mood

• 1.2 Based on pictures, Spaatify adjusts lighting and music to reflect your mood

- Then, the user enter the interactive space, which shows the view of the universe. In this design, we would like to create a serene and simple environement for the user while the user is enjoying the bathtub and navigating the universe with the hand and arm gestures.

• 2.1 You will then see a large screen showing the universe

• 2.2 Move your hand to zoom in and rotate

• 2.3 Zoom-in view of the universe

• 2.4 Tab on object to view infomration

Video Prototyping

2. News Feed Browsing - tile display

- First, the user customizes the environment with her mood of the day with one song that best represents the mood

• 1.1 Customizing the space with a song that represents your mood

• 1.2 Based on your song, Spaatify adjusts lighting and music to reflect your mood

- Then, the user enter the interactive space, which shows an array of news feed content. We would like to display the news in a simple, easy-to-read manner such that the user need minimal mental power to digest and read the content. In this design, we would like to provide a simple and efficient interface to the user such that she can keep herself updated with the happenings aorund the world in a relaxing environemnt.

• 2.1 You can choose news feed from different categories

• 2.2 You can then browse the selected news within the category by tapping on the tile

• 2.3 Tab to read the content

• 2.4 Tab the specker icon to control volume by raising or lowering your hand

Video Prototyping

3. News Feed Browsing - Cover Flow

- First, the user customizes the environment with her mood of the day by selecting one picture from the tiles

• 1.1 Customizing the space with a picture that represents your mood

Interactive Space

• 2.1 You can choose news feed from different categories by waving your hand to flip the covers

• 2.2 You can then browse the selected news within the category

• 2.3 Tab to read the content

• 2.4 Tab the specker icon to control volume by raising or lowering your hand

4. Abstract Art Creation

- First, the user customizes the environment with her mood of the day by selecting one picture from the wheel

• 1.1 Customizing the space with a picture that represents your mood (wheel arrangement)

Interactive Space

• 2.1 You will some abtract visuals (such as firework, sound visualization) on the screen

• 2.2 You can move or tap to create randomness to the canvas

100,000 stars project

For our “abstract” interaction idea with user, we explored available visualization tools on the web and found this 100,000 Starts project particularly interesting. It is a WebGL experiment project developed by Google Art Team. The whole graphic is rendered using WebGL. To make sure we can use their code if we are going in this direction, we emailed the Google Art Team and got positive reply - they said feel free to use it.

Kinect facial recognition for customizing environment

We investigated the possibility of tracking facial expressions by Kinect(in response to our idea to get people’s mood by their facial expression), and found there is this WPF kinect app that tracks faces and represent facial expression with a comic facial expression. Though the facial expression can be tracked, there is still problem of categorizing them as “happy” or “sad” - in fact this can be really hard categorization problem, and we are not sure if there’s any tool that takes graphical data of facial expression, processed it and output the mood. One possible way is just to use certain parts of graphical data of the comic face, such as the shape of their mouths.

Kinect facial recognition for customizing environment

We did simple feasibility tests on how kinect works with skeleton recognition(with WPF implementation), a few observations:

1. identified the working range of kinect: this will affect where we are going to set the device. Based on the result, it might be set about 1~1.5 bathtub length away from the person’s head/hands.
2. tried a few gestures: The device recognized most of them well, though not entirely accurate
   • can detect - waving hands, moving hands up and down, putting hands close to head(can be common in spa/hottub/bathtub), stretching
   • not exactly accurate - when putting hands on top of the head, the skeleton cannot reflect the touch of two parts; when lean head left and right, the difference is very small
   • cannot really detect - nod head, shake head.
3. tried in different lighting conditions: since we are likely to utilize lighting to help people relax, we need to know if kinect can detect people under different lighting conditions, especially when it’s dimmed down. Fortunately with infrared camera the kinect detects well even under quite dark scenario.
4. others: the demo program(just displaying skeleton) tracks people’s skeleton pretty smooth in real-time; also the seated mode would be helpful in our use case since people will most likely be sitting in the tub.

We also tried out another type of demo: use hand movement to control the mouse on the screen. We did this one with both WPF(using C#) and KinectJS demo, it feels the control with native app(C#) is smoother and more responsive than using KinectJS. Our final implementation language will be decided based on which function set we are going for.