Theme – New York

For my bachelor project I built an interactive shop window with the theme “New York”. I choose this theme because I think its a challenging topic and I do have a direct comparison to the shopping window from Jelmoli.
My shop window will not be related to the one from Jelmoli and I do not get support, requirements or guidelines from Jelmoli.

One thing I wanted to achieve with this shop window is to bring New York closer to the person in front of it. To achieve this I will use different technologies. It is important to me that I will attract the user. When shoppers pass by they should stop and start playing with it. The setup should be very playful and therefore target a younger audience.

Read more »

Shopping windows needs a theme

During the research part I talked to Sasha Wohlgemuth from Jelmoli about shop windows. One important aspect, he told me, was that a shop window only has the reason to bring the people into the shop. To accomplish this difficult task, a shop window needs to display a message and a lifestyle. This theme of the shop window will also be used in the shop it self to make the connection between the shopping window and the shop. Emotions are very important. If the shop window can reach a persons attention, he or she is more likely to enter the shop.


Here is an example of the current Jelmoli theme: MY STYLE IS…. New York
Its a good example to show less information and products but still reach the shoppers attention.

Read more »

Most common gestures currently used by digital systems

In this section I would like to show the most common gestures currently used by systems. There are a few gestures which are very common and already so established that we already use them intuitively.

Tap or point to open, select or activate
Most touch based applications use the tap to send click events. It is used to open a function, select an item or activate it. Most touch screen mobile phone use this as one of the most used gesture. But if you look at controller free applications like Microsoft Kinect this function is hardly used yet. „Pointing is the most natural gesture for selection.“ [Saffer, Dan (2008): Designing Gestural Interfaces. O‘Reilly Media, Inc. ISBN 978-0-596-51839-4 Page: 76]

Drag and Drop
From GUIs on personal computers we know drag and drop functions. In a natural user interface such a function can be a very clear gestures as it can be directly transformed in how we move objects in real life.

Pinch to shrink and spread to enlarge
Since Apples iPhone and iPod Touch got really popular nearly everybody knows that the pinch or spread of fingers or hands gestures can change the size of an object. It became one of the most popular gesture without a direct real life counterpart.

Wave to activate
Waving is a simple gesture and has already a wide usage area. Not only is it needed to activate the user tracking on Microsoft‘s Kinect , it‘s also very common in public restrooms for the water tap, paper spender or the toilet flush.

Exhibition Setup

Here are my exhibition setup plans. The idea is to build a box which should look like a shop window. The reason I build such a construction is that I do not have the possibility to display it during the exhibition in a real shop window.


Read more »

Projector and Kinect testing

Before I could create the exhibition design, I had to know how large the projected image can be and what the distance of the projector from the screen will be. I tested it with two different projectors. A short-distance projector and a normal projector.
It turned out that the use of a short distance projector is the better solution because this makes the construction easier and the size of the image is bigger. Unfortunately, the color quality of the short distance projector is not as good.

Next, I conducted tests to see how well Microsoft Kinect is playing with the projection. It was mainly about whether the tracking still works when set up directly in the projected area. After that I checked the angle of the camera to identify more possible positions of it.

The findings of this tests were that the Kinect and the projector cover the same area, which results in a much easier setup. Also tracking through a projected area didn’t seemed to be a problem.

Thanks to Jan Huggenberg for helping me with the test setup

Techniques in Computer Vision

When building an interactive system it’s important to look at the technologies available. What would be a perfect setup to build a prototype for an interactive system. What are the advantages and disadvantage of the different solutions? In this chapter I explain the technologies I used and why. I explain the basic functions of the prototype and the technology behind it.

Before building my prototype I decided to do some tests in different programming languages and techniques. I did tests three different environments:

  • Processing and OpenCV
  • Processing with SimpleOpenNI
  • Actionscript 3 with Kinect, OpenNi and NodeJS.

In the following videos I have some thoughts about each technology with its advantage and disadvantage

OpenCV in Processing
OpenCV is an open source computer vision library developed by Intel. I used the processing edition from
OpenCV for Processing is a good tool develop a working prototype in a short time. The problem with Processing edition from OpenCV is, that it needs a lot of performance. In the next two videos we can see the difference with different resolution.

OpenCV and Processing 320 x 240
This resolution works quite good and the performance of the program works fine.
Read more »

Context Inquiry (PDF)

Through the method of observing, interviewing and user centred gesture design I looked at gestures. What gestures could be used to interact with an application? What gesture would make sense when interacting with a digital system. What gesture are acceptable in public space? How would people use an interactive shop window and what gestures to interact with it? What would the user rate as acceptable in public space? How do people behave in front of shop window and what are the key attractor that pedestrians stop and look at shop windows? What is important for shop window designers?


Through observing and interviewing different professionals and pedestrians I tried to find out what is important for them and what they expect from a shop window.



Trough the methodology of observing shoppers and their behaviour in front of shopping windows as well as trying different gestures in public space I gained a deeper aknowledge of my desired user group.

Knowing that some gestures in public space are not acceptable is important when designing gestural interaction. Most gestures we can use for interactive shopping windows need to be easy to learn as the person who will use such a system doesn‘t want to learn it first.

It‘s important, that a shopping window uses a catching entry point. The more unique a shopping window is, the more people will look at it closer.

Having the possibility for users to look at the object in a more detailed way will increase the possibility that the user will buy something. Offering a order or buy option in the shopping window can make it possible to sell products even when the shop is closed.

What gesture can be performed in public

I created a three color rating system to analyse a small set of gestures. Answering questions like: what hand or finger position would people use or how much would they move their bodies in a public place, resulted in a rating from green to red for each gesture.

Green: Totally acceptable and fine to use and perform in public space

Orange: OK, but it depends on the situation. Mostly only OK if an interactive system gets controlled with it.

Red: Not OK. People do not like to perform it because its offensive or makes them look silly

One Foot
Performed wile walking, playing, dancing
Read more »

Possible In-Window Purchase

When we design the next generation of shop windows its important to think about the additional values. Not only can such a system create an entertaining buzz, but it can also drive sales. If we add the ability for the customers to purchase products directly from the window, even when the shop itself is closed. To establish an easy connection between the shopping window and an online store we have a few possibilities.

Order form
Having an order form directly integrated in the system can make it easy for people to order a product. There will be no additional device needed to order a product.

Video Order
Shoppers will record their adress on video using a camera and a microphone built into the shopping window. This way no additional device is needed for the shoppers.

Order using SMS
Nearly everybody owns a mobile phone. Being able to use text message from a mobile phone is a convenient way to order a product.

QR Code
The QR (Quick Response) Tag, was developed by a Japanese company in 1994. Its a 2 dimensional code to store information. Most smart phones have the possibility to read QR codes. Using a QR code in the description of the product makes it easy for the user to order it online. One advantage of such a system is that the order address or payment option not directly will be handled in the shopping windows which makes the order process more secure and private.

QR-Code which contains the URL:

Google Wallet (NFC)
With Google launching a mobile payment system based on NFC Chip payment we will soon get the possibility to pay all over the world with our mobile phones. This system will allow to pay products through the window directly from the street

Technologies of Interactive Shopping Windows

Interactive shop windows use different technologies. Currently the most common are touchscreens. Many examples us this technology because it is already well established. Newer touch-screens use multi-touch which improves the user experience. Using a depth sensing camera or multi camera tracking setup is the latest trend in interactive shop windows. Shoppers will no longer need to touch anything and can use their hands or body to control the information displayed on the shop window. New technologies allow multiple users and 3D gesture tracking. With such a system in place window shopper can look at products from any angle using simple gestures. Whenever there is a user and the system knows what the user is looking at, it can track this information. Using this information in an analytics system can give valuable information of which product is looked at most and for how long. This technology would allow to take a picture of each person who looks at a product and store this information together with the products the person looked at (even though under the current law in Switzerland this would be prohibited).

Further reading:

« Newer PostsOlder Posts »