Conclusion Technical Solution

The setup I choose to build the technical prototype helped me to develop quickly a running prototype and test the different functions. Having a technology like Adobe Flash for the visual and audio part made it easy to design and develop the prototype at the same time. Using processing together with the SimpleOpenNI wrapper made reading the values from Microsoft Kinect a very easy solutions. Using the micro-controller Arduino to control the light is an easy way to bring digital content in a connection with a physical attractor.

Flow Chart

Arduino and Electronics

For my prototype I displayed two products. Both of them are related to one scene which will be projected to the screen. To make the connection between the product more visual I used light. The light will higlight the product of the active screen.

To achieve this functionality I used Arduino and two relays. On the code side I used the serial listener which listens to the Tinker socketserver that controls switching the lights on and off.

Actionscript 3

The design of my shop window theme should be easy to change and quick in development time. Hence, I used Adobe Flash to achive this goal. The application is built in Actionscript 3 and is set up as an MVC based application. A socket server receives the values from the processing and passes them on to Flash where the values are handled by an appcontroller. The combination of pure Actionscript 3 coded in the Adobe Flash Builder Enviroment and the Flash IDE gave me the possibility to design quick prototypes and test them thouroughly before I had the final graphics.

Processing and SimpleOpenNI

For my final prototype I used SimpleOpenNI and Processing for the user- and hand tracking. using the Microsoft Kinect camera. Because I used Flash to display all Information on the screen I used the Server class from processing. The only values I send to the flash are the hand position and the center of mass of the closest user. These two values can easily be accessed through SimpleOpenNI.

Techniques in Computer Vision

When building an interactive system it’s important to look at the technologies available. What would be a perfect setup to build a prototype for an interactive system. What are the advantages and disadvantage of the different solutions? In this chapter I explain the technologies I used and why. I explain the basic functions of the prototype and the technology behind it.

Before building my prototype I decided to do some tests in different programming languages and techniques. I did tests three different environments:

  • Processing and OpenCV
  • Processing with SimpleOpenNI
  • Actionscript 3 with Kinect, OpenNi and NodeJS.

In the following videos I have some thoughts about each technology with its advantage and disadvantage

OpenCV in Processing
OpenCV is an open source computer vision library developed by Intel. I used the processing edition from http://ubaa.net/shared/processing/opencv/
OpenCV for Processing is a good tool develop a working prototype in a short time. The problem with Processing edition from OpenCV is, that it needs a lot of performance. In the next two videos we can see the difference with different resolution.

OpenCV and Processing 320 x 240
This resolution works quite good and the performance of the program works fine.
Read more »