OurNewBaby:Caseture
In recent months, we have expanded our interest in off-browser communication and experimented with gesture-based interaction. From various approaches and challenges, an in-house project emerged that aimed to seamlessly and excitingly combine elements such as technology, experience design, and product communication.
"A motion of the body that contains information. Waving goodbye is a gesture," is how Kurtenbach and Hulteen define the term gesture. Indeed, products also evoke certain emotions. From this standpoint, we explored the interaction between human-product interactions. Based on our findings, we identified 7 interactions that should serve as the basis for our concept and design process: Discoverer, Approacher, Explorer, Customiser, Experiencer, Reacher, and Player.
7 Interactions
Each interaction embodies a phase of the story with its gesture and the associated emotions it evokes. Thus, a specific dialogue takes place between the user and the screen (or product).
Emotions describe the core element of the design process of our experiment. In summary, our storyline for human-product inspiration begins with the (1) surprise over something that unexpectedly draws attention and provokes its discovery. This is followed by the aspect of (2) wonder, which leads the user to slowly approach the product. The (3) interest in learning more about the product rises until the state eventually turns into (4) desire. Imagination is stimulated and increases the longing. A certain connection to the product is built, it is tried out imaginatively, and a possible benefit is foreseen – the phase of (5) admiration sets in. The process is naturally followed by the (6) pursuit of the product and ultimately ends with the (7) acquisition of the product in the phase of satisfaction.
Progressive Framework
Caseture focuses on the potential of novel product and service communication. It embodies the various approaches to how products and experiences can be used as emotional stimuli. The gesture-based interactions that Caseture evokes exemplify solutions that can be suitable for many different contexts and provide added value to both users and brands.
Caseture was designed to serve as a basis for various concepts - whose scope seems almost limitless. The framework of this experiment can be a brand, a product, a service, or an independent experience. It is flexible and adaptable for one or even multiple users. While some interaction elements primarily address one person (Customiser), other models allow for a collective experience for multiple users (Player).
The visual language of Caseture is minimalist in nature. In this way, the product and service as central elements of the experience, as well as the interaction, form the focus. The design and branding of Caseture are subtle and aim to motivate direct interaction.
The simplicity of the icons and graphic elements reflects the simplicity of the gestures and interactions.
Technical Aspects
Caseture was developed using Unity – a platform used as a basis for computer games and interactive experiences. We worked with a Full-HD resolution at 60fps to achieve the desired high-end result we set as our goal. The project includes the integration of 3D objects, animations, timelines, and videos, accompanied by different learning curves and experiences.
Compared to other technologies, our model takes advantage of multi-layered capabilities. In addition to full-body recognition at long distances, hands can also be recognized that are moved very close to the display (about 1 meter). Thus, people can interact directly in front of the installation or be tracked in the environment.
For Caseture, we applied 3 types of gesture recognition:
- Position and speed of the body
- Position of the hand in relation to the body
- Head position tracking
With this information, hand movements such as swiping, dragging, pointing, and pressing can be recognized.
The Unity 5 Standard Shader and Real Time Global Illumination helped us perfect the interaction design. In addition to Unity's standard feature set, we developed a custom shader for "Experiencer" as well as an efficient multicore cloth simulation for "Discoverer" and "Player". Fusion and Unity communicate via an interface implemented with Google's High Performance Protocol Buffer and run over Websockets.
With the initiation of the Caseture project, we attempted to redefine the boundaries of innovative, digital experiences and stimulate the discussion around gesture-based interactions. While we continue to experiment, our framework already allows for almost limitless possibilities for a wide range of cases.