Body Electric
Simon Penny and Malcolm MacIver
assisitants: Jeff Ridenour, Andrew Homyk
Body Electric was exhibited in the Neuro exhibition at the Wiliamson Gallery of Art Center college of Design, Pasadena, April 15-June 29, 2003, curated by Stephen Nowlin.
The Neuro Project was sponsored by the Center for Neuromorphic Systems Engineerring at Cal Tech, directed by Pietro Perona, supported by the Engineering Research Centers Program of the NSF under award EEC-9402726.
The goal of Body Electric was to give users a bodily experience analogous to the sensory capabilities of a weakly electric fish. It was an attempt at scientific education via direct experience of embodied simulation. The user entered the darkened interaction space. A full scale realtime 3D representation of the user, derived from the custom 3D machine vision system, was presented on large back projection screens on opposing walls. The figure was false colored based on the direction and proximity of an invisible autonomous agent which represented the prey or target. Eight channel spatialised audio also sonified the movement of the agent and the avatar of the user. The user's goal was to capture the prey. That is, based on visual reference to the mirror space represented on the screen, the user would move toward and reach out towards the location in real space of the virtual agent.
The interaction space is 12' square. It has a light trap doorway on one side and 10'x8' back projection screens set into opposing walls. At the Williamson, the backstage spaces were 14' long, and the projection image was folded via large mirrors hung on the far walls. One of these backstage spaces also functioned as the control room.
Eight Custom Infra red floodlights shine diagonally down and across the space from mountings at 10'. The four video cameras are mounted at five feet, one in each corner. The eight channel spatialised sound is played through eight speakers, on in each corner of the room.
Hardware and software:- Four IR machine vision cameras
- Eight infrared floodlights
- Two custom suse linux PCs
- Traces 3D vision system client and server
- One Mac G4 running Max
- Eight active speakers
- Two data projectors
The vision system is based on the traces vision system developed by Bernhardt and Penny in 1997. It generates a realtime 3D point-cloud body model from the four camera images. In Body Electric, the point cloud is skinned using a modified splatting approach. The body model or avatar is placed in a virtual environment of equivalent scale to the interaction space. Virtual cameras image the virtual environment from the sides, with live perspective correction based on the position of the users head. The camera views are sent out to the two projectors.
Autonomous agents, represented by green octahedra, also inhabit the space. These are analogous to the fishes prey. Their location and proximity is indicated by the changing color of the skin on the avatar, in a way analogous to the skin response of the fish. Agent location is also communicated by spatialised audio. The user's task is to 'capture' the agents.
In sensory neuroscience, the role of behavior in perception is especially clear in certain cases, one being the weakly electric fish, which hunt at night, in muddy Amazon rivers where vision is useless. They utilize a weak (less than one volt) self-generated field to sense their surroundings. Thousands of sensors covering the body surface become
stimulated when near the invisible objects they are hunting; they interpret these minute fluctuations in electrical activity of the skin to determine what the object is, and where it is.
In Body Electric, we simulate this fish's electrosensory system through a custom multimodal real-time sensing and display system. The 'spectactor' engages in a complex sensori-motoric exploration of a novel environment.
Until recently, the prevailing orthodoxy concerning perception was rooted in an elightenment-objectivist model which proposed that sensory systems take in the world passively, with information coursing into our brain to form rich internal representations of the world that we use to guide our behavior. This view of perception is embedded in scientific culture and as a result, is embodied in the hardware and software of our technologies.
A more recent view of perception has it that behavior is tightly coupled to the way we sense the world, that perception is a temporally extended process of active, embodied engagement with the world. Body Electric is informed by notions of active sensing, with respect to the subject matter of the installation, and in the way the interactive cultural experience is designed.