Traces: Embodied Immersive Interaction with Semi-Autonomous Avatars

Published Convergence: the journal of research into New Media Technologies, University of Luton press, Summer 2001.

Simon Penny
University of Portsmouth (UK) and Merz Akademie, Stuttgart, Germany,

Jeffrey Smith
Carnegie Mellon University Robotics Institute.

Phoebe Sengers
GMD - German National Research Center for Information Technology,

Andre Bernhardt
Karlsruhe University.

and Jamieson Schulte
Carnegie Mellon University

October 23, 2000


Traces is an artwork for the CAVE that uses a novel machine vision system to enable unencumbered full body interaction with a range of semi-autonomous agents without the imposition of any sort of textual, iconic or encoded-gestural interfaces and without physically restrictive wiring, pointing devices, or headgear. Furthermore, Traces does not consist of a world which is navigated; instead, the movement of the user through the space leaves volumetric and spatial-acoustic residues of user movement which slowly decay. This project was motivated by a desire to explore and critique four central issues in contemporary HCI: (a) embodied interaction with computational systems; (b) rapid and transparent learning of interfaces by untrained users (the autopedagogic interface); (c) immersive bodily interaction with software agents, (d) extension and elaboration of the general conception of interactivity itself. To explore these issues, we built an infra-red multi-camera machine vision system which constructs a volumetric model of the whole of the users body in real time. We have also developed custom 3D vision tools, graphical techniques and a range of techniques for generating and managing semi-autonomous agents in immersive environments. Traces was developed between November 1998 and September 1999 and exhibited in the CAVE at the Ars Electronica Center for Ars Electronica 99. Development was supported by the Cyberstar98 award, by GMD Sankt Augustin Germany, and by Carnegie Mellon University.

Download Paper [pdf]