User Tests

Scenario 3

In the 3th user test scenario I asked 4 users. Most of the participant have worked and played with Kinect before but they didn’t know what they can do with the interactive shop window. The light to display the product was installed and a wave gesture is symbolized by screen. The users didn’t have a time limit or a goal to reach. Time spend in front of the prototype was between approximately 3 min up to 6min.

Because the user knows that a wave gesture can give him more control he was quickly able to discover the virtual theme.

Parts of the design need to be redesigned. Wave gesture displayed through a visual feedback was a great help to understand what gesture needs to be performed to get more control. The light for the active bag was a great help and made it easier to understand the concept of two scenes for two products.

Theme Design

When designing the application I tried to bring in an abstract layer. People should still easily discover that the theme is “New York” but it should not look like a photo. Therefore I designed it in a cartoon like way. I tried to use typical aspects from New York when working with the graphics. Using photos as a draft and reassemble them to a new art work helped to keep the New York touch but added a surreal visual value. Changes between the scenes are displayed through an animation. The animation has an important part because this is the point when most users get attracted by the shop window.

The interactive shop window prototype uses two scenes: Central Park and City.

Scene: City
The City scene displays the fast living and uprising business life. Everything is loud and fast.
Buildings in the city are tall and grey. Only a few famous buildings are added to build the connection to the trend city New York. Everybody knows the city and its very famous attraction like the state of liberty or the empire state building. Therefore I added this two famous constructions to the design to make it easy readable for the visitor to see what the theme is about. But New York is not only famous for this two constructions, New York is a major player in the movie industry like in the movie King Kong. Therefore I used King Kong as a fun attractor for the shop window visitor.
The bag displayed in the shop window is a bag specially designed for business lady‘s. To display more information about the bag I used the famous “ONE WAY” street signs and replaced its content with the new bag information. Using such a connection helps the user understand that the theme is “New York” but also brings in the connection between the bag and the theme.
As an addition to this visual aspects and features I added sound. Having sound as a background noise as well as having sound for playful features supports the theme even more.

Scene: Central Park
The Central Park scene is, in contrast to the City scene, more quite and nature related. The Central Park in New York is another very important part for this city. Many people use it to relax from their stressful and hectic life. They go out for a run or play sports or have a picnic with friends. In my design I tried to bring in this relaxing and playful aspect. I used butterflies to follow the active hand, a squirrel running around and added girls having a picnic. All of it supported by background sounds from a park. A man which walks into the scene displays that there is a wave gesture required to gain more possibilities to control the interactive shop window. With such a direct message it’s easy for users to understand how to get an additional interaction layer.


Sound in the prototype can help increase the fun factor while using the application. Certain hot spots in the different scenes play sound when the user activates them. Every scene has a theme background noise to increase the connection between the visual theme and the real world.

The City:

Audio MP3

Central Park

Audio MP3

King Kong

Audio MP3

All sound are from

Arduino and Electronics

For my prototype I displayed two products. Both of them are related to one scene which will be projected to the screen. To make the connection between the product more visual I used light. The light will higlight the product of the active screen.

To achieve this functionality I used Arduino and two relays. On the code side I used the serial listener which listens to the Tinker socketserver that controls switching the lights on and off.

Actionscript 3

The design of my shop window theme should be easy to change and quick in development time. Hence, I used Adobe Flash to achive this goal. The application is built in Actionscript 3 and is set up as an MVC based application. A socket server receives the values from the processing and passes them on to Flash where the values are handled by an appcontroller. The combination of pure Actionscript 3 coded in the Adobe Flash Builder Enviroment and the Flash IDE gave me the possibility to design quick prototypes and test them thouroughly before I had the final graphics.

Processing and SimpleOpenNI

For my final prototype I used SimpleOpenNI and Processing for the user- and hand tracking. using the Microsoft Kinect camera. Because I used Flash to display all Information on the screen I used the Server class from processing. The only values I send to the flash are the hand position and the center of mass of the closest user. These two values can easily be accessed through SimpleOpenNI.

User Tests

After I finished the first working prototype, I started with user tests. None of the test users were informed about the subject but all of them have recognized the subject of “New York“.
With the user tests I wanted to find out if people understand the functionality of the interactive shop window. Could they understand how to interact with the application? I asked them what they would change and what was difficult for them to do.

To test the prototype I used different scenarios.

Scenario 1:

For the first user tests I asked 3 people to try the interactive shop window at a very early stage. Each of the user was only informed that the installation is a shop window. Neither the possibilities of interaction nor functionality are known. The users had to discover everything by them self. The users didn’t have a time limit or a goal to reach. Time spend in front of the prototype was between 29 seconds up to 1min.

Read more »

Interactions with the shopping window

Using a Microsoft Kinect sensor, Ventana recognizes a person’s presence in front of the window. Immediately it greets it’s opponent with an audio-visual feedback. The user is now aware that a communication is established. Simply by raising and waving the hand, the user is now able to navigate and explore the contents inside Ventana. The interaction patterns are purposely kept simple and straight forward. Thus, the user learns to fully interact with Ventana in very little time.

Nobody in front of the shopping window
The screen is in indle mode virtually empty. This is emptyness changes dramatically when entering the camera sigh. Through this massive difference on the screen it will attract the pedestrian and direct their gaze in direction of the shopping window.

Walking past
If pedestrians walk past the shop window, the foremost person is identified. A parallax effect displays the movements of the active person.
In each case, at the height of a certain product, the scene of the background changes to a theme that fits the product.

If a person stopps in front of the shop window, a signal displays the information that through a wave gesture you can get more information and functions.

Performing a wave gesture activates a cursor which can be moved by small hand movements.

Hand movements
After the cursor has been activated, it can be moved around freely on the screen. As long as the cursor is active, the scene can not be changed until the cursor is released.

Active hot spots
Running the cursor over an hot spot area will display a playfull animation or a product information. These actions are also supported with audio events.

Size of the user
The concept is subject to size of the user to create different visual feedback. For example if a child is in front of the shop window special information and child related animations should be shown.

Singel User
The current prototype is build for a single user mode. The priority which user has the control happened over the distance which individual users have to the camera. The system always selects the closest person.

Multi User
The concept allows multi-user interaction to be installed for large display in shopping windows. Multiple users should be able to deal with the shop window at the same time.

3D Renderings for the exhibition setup

The idea would be to line the interior with dark fabric.
Thanks to Florian Wille for designing this 3D Renderings.

Read more »


The first working prototype focused on the technical solution. How can I track the user passing by, how can I track when a user gets control and how can this change the setup.
I ended up having two levels of interaction.
The first level is the passing by
When someone passes by at the shopping window a city pops up in the display. Having such a big change in the shopping window attracts automatically the users attention. In this mode I also added a parallax effect. Having such an effect in place made the test users already moving back and forth just to see if something happens.


Read more »
« Newer PostsOlder Posts »