Brethren,
Welcome to the First Church of Jesus, Lord of Virtual Space, where you, like Thomas, can thrust your hand in the wound of Christ.
You can absolve your sins by taking up His cross on the way to Golgotha. The Church of Jesus, Lord of Virtual Space can deliver to you what the faithful and non-believers alike have yearned for for so many generations: Proof!
Here and now I want to extend an invitation to each and every one of you to join me, each Sunday, in the comfort and privacy of your own home, in the most scientific, the most up to date, religious experience that is available today. Join me every sunday as we live the life of Christ together.
You can witness the sermon on the mount, the betrayal in the garden of gethsemany; you'll hear the cock crow, you will be there!
Don your holy eyephones and come with me, through the chapels of the stations of the cross, suffer with your savior, take Christ into your heart, and step into the body of Christ. Then join me in virtual communion, where angels will pass you golden goblets direct from the flowing wounds of the crucified christ!
For only $599 and a low monthly rental, you can visit the Kingdom of the Lord (or something very like it), every Sunday. Our Fathers' house has many mansions, and I want to show you just a few of them.
Picture if you will, this scene: in living rooms across the country, a parish of believers wearing eyephones, negotiating invisible obstacles, falling on their knees. But inside the eyephones its a different world, the streets are sandy, the sun near- eastern hot, the crowd is yelling as Pilate hands down his judgement. A Christian virtual theme park.
Virtual Reality, like any other technology, is embedded in a cultural history which lends to the enterprise a worldview. In the first part of this paper I will attempt to unearth aspects of that system by, firstly, constructing a pre-history of VR; and secondly, examining VR's comtemporary cultural context. I will then go on to discuss some of the issues that might arise as VR embeds itself into western culture. The scenario above is an attempt to indicate the likely modes in which the technology will inhere to a culture such as ours, and also to suggest the cultural specificity of our VR to a Graeco-Roman tradition, as opposed to a VR that might arise in a non-western culture, if such a thing is possible.
Jaron Lanier has announced that virtual reality is the culmination of culture. This is a somewhat conceited judgement given that he is a major developer of the technology. But my concern is more with the cultural specificity of that remark. The abhorrence of the body is inherent in christian doctrine, and christianity has served as the basis for western philosophy until last century. Philosophical ideas such as the Duality due to Rene Descartes are based in christian doctrine. Alluquere Rosanne Stone has observered that in the Greek New Testament, the word endyo is used in the context of narratives of christian conversion, to mean `to put on christ' in the sense of putting on an overcoat. [1] This condition of `stepping into' is very similar to the condition of being in VR. Such suggestions strengthen the assertion that the cultural history of VR is as old as western culture itself.
Given that technological progress continues at the rate we have become accustomed to, and there is every indication that it will for a decade at least;[2] the applications of VR technology will be as wide as offset printing is today. It will function as educational media, entertainment and job training; it will have applications in the judicial system and in the rehabilitation of the criminally insane. In twenty years, many of us may go to work everyday in a virtual office, the physical body of the person at the desk opposite will be in Tokyo. We will teach classes to students distributed around the planet, we will teach about the outdated cinematic technology by hauling a virtual 16mm projector out of a virtual closet, we'll plug it in to a virtual outlet and load a virtual film reel. Our virtual projector will then project a virtual copy of Un Chien Andalou on a virtual screen.
This medium is larger than the codified forms we are used to. It cannot be considered as `expanded cinema', fancy arcade games or interactive TV. It is not a videophone or a conference call. It is not a dream, nor is it `electronic LSD'. It is both enclosing and interactive. It bears many characteristics in common with previous cultural and media forms but presents an entirely new combination with extended possibilities.
In VR, unlike previous communications technologies, the proscenium is dissolved. Through painting, sculpture, drama, cinema, TV, the separation of audience from art was complete. VR effects a melding of experience and representation rather than the separation effected by the proscenium.[3] We viewed through the renaissance window, we held the privileged singular viewpoint. One might argue that VR technology has `automated' renaissance perspective, increasing its effective range from a mere 10 degree slot to a full wrap around experience. But the notion of the privileged position of the viewer persists. VR also perverts relationships of the dramatic tradition, one is simultaneously actor and director, one is both constructing a narrative, a protagonist of that narrative, and its audience.
VR offers a condition paradoxical with respect to our familiar art-forms, because it is simultaneously a picture and a bodily experience. That is, a cultural experience to be consumed, at a distance, as it were, by the eye, a gulf of space and time between the viewer and the viewed, with no potential of active interplay. And simultaneously, it is as immediate and physically engulfing as a game of squash. This simultaneous occupation of the realm of the symbolic and the physiological is particulary fascinating. The question underlying this paper is: what kind of cultural practice can we imagine for this radically new (non)site.
As early as 1967, Guy Debord observed that in modern societies : "... all of life presents itself as an immense accumulation of spectacles. Everything that was directly lived has moved away into representation. "[4] He quotes Feuerbach: "But certainly for the present age, which prefers the sign to the thing signified, the copy to the original, fancy to reality, the appearence to the essence,...illusion only is sacred, truth profane. Nay, sacredness is held to be enhanced in proportion as truth decreases, so that the highest degree of illusion comes to be the highest degree of sacredness."[5]
This desire for simulation over the real, for the spectacular, the simulated, runs deep in western culture, and has always been carried as far as the available technology would allow. I will string together a loose collection of historical examples, with no pretense of constructing a linear, causal history. There are ways in which these activities have a common theme, others in which they are radically different.These cases will serve as viewpoints from which VR may be considered. I will leap between disciplines because the desire for the simulated has been expresseded in just as many, and because VR is a combines aspects of all these different disciplines.
The developers of VR have (unwittingly?) inherited a humanistic world view (an attitude to life and a way of making pictures) which places the eye of the viewer in a position of command, a priviledged viewpoint on the world. Asian imagery offers us alternative ways of looking or of picture construction, medieval european imagery offers another. Television offers a third, with its multiple viewpoints and rapid cuts which dissolve the body. This historical examination implicitly asks the following question: What if VR had developed along pictorial principles other than renaissance humanism? Could we feel we could inhabit it at all? Western perspective, or any system of pictoral representation, is in no sense innate, but a learned (often arduously) convention. In other words, how much is any so called VR dependant upon culturally acquired knowledge in order to be decipherable? Numerous experiments in visual perception performed on non-western people attest to the cultural specificity of our particular way of pictorially representing space and distance, relative scale etc. I've heard reports that New Guinea highlanders were unable to `see', to indentify, their image in polaroid photographs taken at the time.
What if VR developed in a culture with a different attitude to the body? A recent article on Indian dance relates: "The sense of space was wholly different...: no long runs or soaring leaps or efforts to transform the stage into a boundless arena, a kind of metaphysical everywhere, but content within the realm of the body, comfortable with dimension and gravity, all ease, all centered." The author described the attitude of the teacher to the body "...no sense of elevation or extension... body self contained ...inwardness, inwardness ...In Hinduism" said the teacher "there is no beyond. "[6] Compare this attitude to that inherent in VR. The corporeal sense of touch requires immediate physical contact with its object, not so the eye. VR arms the eye, it gives the eye a hand of its own, propelled (appears to be propelled) by the gaze itself. The authoritative viewpoint of renaissance pictorial space is actively enpowered: action at a distance. The entire body is propelled by scopic desire.
VR, as curently formulated, is a direct continuation of the tradition of illusionistic pictorial representation which was already in evidence in Pompeii. It was developed in the renaissance in concert with the development of humanistic philosophy and optics in the west and gained a time dimension with the development of cinema. Along the way it became technologized, first through the use of optical drafting devices (cameras lucida and obscura) through the development of photography, both mono- and stereo-scopic; and the projection of moving pictures. It is also a sucessor to the long tradition of grand theatrical spectacles and of world's fairs, amusement piers and theme parks: a Coney Island of the mind.
The pneumatic automata of Hero of Alexandria are an example from the late Greek world of the utilisation of the most sophisticated contemporary technology to realize a persuasive and articulated simulation of reality. [7]
Grand theatrical spectacles were regular occurences through the renaissance and the baroque. In 1548 the Queen of Hungary welcomed Phillip II of Spain with a grand two day event spectacle which began as a dance tournament. `Savages' attacked the dance and carried off a number of the women when repelled. The next day `knights' attacked the castle in which the `savages' had barricaded themselves. During the battle that ensued, Phillip was served a banquet by nymphs and niaids in the centre of the battlefield.
Baroque ceiling painting must properly be included in this history of technologies of simulation. The vertigo inducing draftsmanship constructs for the viewer the sense of `peering up into heaven', while simultaneously dissolving the physical architecture. This presentation of a reality other than the one immediately visible is a recurrent idea in christian art . This historical utilisation of advanced simulation technologies by the church prompted my invention of the Church of Virtual Space, of which these paintings are clear precursors.
The latter part of the nineteenth century saw an extraordinary explosion is invention in technologies we might call proto-cinematic. It was not until 1838 that the British scientist Wheatstone built the first stereographic image projection system. The first multiple user stereographic projection system was exhibited in Lyons on 1890. Along with the well known time lapse photography of Marey and Muybridge, Edisons' kinetoscope and the cinematograph of the Lumiere brothers; numerous more or less bizarre optical/mechanical theatres were constructed.
Daguerre's Diorama was one such mechanized theatre, in which the audience was propelled on a revolving viewing platform, past enormous scene paintings which were painstakingly painted with differing degrees of transparency, such that by controlling lighting from the front and the back, the illusion of the transition from daylight to dusk to night, could be effected. After a faltering career as a hack realist and theatrical scene painter, Daguerre found great sucess in this invention, the revenue from which funded his photographic experiments.
The World Exhibition in Paris in 1900 sported several of these optical mechanical theatres. The Mareorama simulated a sea voyage from Nice to Constantinople via Venice. During the simulation, two screens, 40ft high and 2500 ft long were to be unrolled while the viewers stood on a pitching ships deck. The inventor of this system was yet another minor realist painter, Hugo d'Alesi, who spent a year on board ship painting the sections of the screens. A contemporary newspaper report trumpets: "Few visitors to the Exhibition will be able to resist the temptation ... to make an inexpensive voyage which involves no hazards whatsoever, yet is so natural.... even on the high seas, amid raging elements, one can get out and tread on terra firma at any moment. "[8]
Other mechanical theatres at the Exhibition included `Stereorama' or `Poeme de la mer' and a simulated Trans-Siberian Railway. The railway was placed strategically nearby the Russian and Chinese pavilions and was built by the Compagnie Internationale des Wagons-Lits. There is recognisable here a certain historical continuity in the use of high-tech for corporate PR. The EPCOT scheme is not significantly different.
About the same time, the scholarly Marquis de Selby seems to have been engaged in experiments into a more truly `virtual' tourism:
"During his stay in England, he happened at one time to be living in Bath and found it necessary to go from there to Folkestone on pressing business. His method for doing this was far from conventional. Instead of going to the railway station and enquiring about trains, he shut himself up in a room in his lodgings with a supply of picture postcards of the areas which would be traversed on such a journey, together with an elaborate arrangement of clocks and barometric instruments and a device for regulating the gaslight in conformity with the changing light of the outside day. What happened in the room or how precisely the clocks and other machines were manipulated will never be known. It seems that he emerged after a lapse of seven hours convinced that he was in Folkestone and possibly that he had evolved a formula for travellers which would be extremely distasteful to railway and shipping companies." [9]
It should be noted in passing, that Virtual Reality is discussed here as if it exists. At the time of writing, VR in the civilian domain is a rudimentary technology, as anyone who has worn a pair of eyephones will attest.That the technology is advancing rapidly is perhaps less interesting that the fact that virtually all commentators discuss it as if it were a fully realized technology. There is a desire in our culture for VR that one can fairly characterize as a yearning.
VR has lingered pre-natally in Sci Fi and the Star Trek Holodeck for a generation or two but now it is being born. It will slip frictionlessly into our culture because our culture has prepared us for it. I have suggested that every significant media technological development since the renaissance has been employed to create theatres of simulation. This idea was not lost on Andre Bazin, who noted mid-century that: " The guiding myth...inspiring the invention of cinema, is the accomplishment of that which dominated in a more or less vague fashion all the techniques of mechanical reproduction of reality in the nineteenth century, from photography to the phonograph, namely an integral realism, a recreation of the world in its own image, an image unburdened by the freedom of interpretation of the artist or the irreversibility of time." [10]
This readiness for VR has been prepared (most recently) by Disneyland, Hollywood, Liposuction and Nintendo. Conceptually vacuous theme parks, anesthetizing cinema, interactive games that perpetuate the myth of the individual and the cult of violence-as-liberty. And perhaps most significanlty, the acceptance that the body may be customized at will like some kind of hotrod.This culture customizes its bodies like it customizes its cars. The body is a representation only, an external appearance, and may be adjusted to suit the taste of the owner. The absolute malleability of the Virtual Body is different only in degree. During early April 1992, daytime TV host Jeraldo Rivera had liposuction live on TV in front of a studio audience. Gobs of yellow fat were sucked from his buttocks and injected into his lips and around his eyes. The attitude to the surgical customizing of the flesh, "body sculpting" and the designing of the Virtual body both assume and reinforce the Cartesian duality by restating the body as pure representation. Thus VR is an easy step because the body is already a representation.
How real is VR? The cultural underpinnings are already in place to lubricate the general acceptance that VR does adequately represent `reality'. VR may be considered as a kind of `instrumentation' of the body. The interchangeability of visual consumption with `experience', which we are encouraged to believe occurs via TV has certainly colored exectations of the virtual environment. Simultaneously, the modern notion of the functions of the automobile has informed the shape of VR. Iggy Pop defined this condition in `The passenger': "...he travels under glass... all of it is yours and mine... so lets ride and ride and ride and ride and ride". It's a very limited interactivity, I can travel, observe, but I cannot act. Nor can my environment act upon me. A white man driving through Chicago ghettos in a plush rental car on a hot saturday evening with the AC and the Stereo on, with tinted windows and the doors automatically locked is in VR. The paradigm of VR is informed by this paradigm of motor car use. It's a paradigm of the powerful gaze, of monitoring but remaining undetected: infra red night vision: it's a military intelligence model. And why should we be surprised about that!
To wear a watch is to be `instrumented'. When I wear a watch I am linked to a large organising grid. I can know by looking at my watch whether an office on the other side of town is closed for lunch, or whether my mother on the other side of the planet is still asleep. I can prepare my trajectory to arrive at the dentist within minutes of my appointment time. I the sense that a large social system is coordinated according to one grid system, a system which I can predict and plan with reference too, it is a virtual world.
We are taught to believe that you can "experience" the countryside from inside an airconditioned car travelling at 60mph. This `belief' prepares us for the VR condition. Virtual Reality is as real as a picture of a toothache. A reality in which you can walk through walls with impunity, a reality which has no odor or temperature isn't very real. But the construction of more and more complex and expensive interfaces is beside the point. It's the kind of obsessive project that characterizes the activities of engineers in the realm of cultural production. A discussion of what the `bandwidth' of reality is, is folly. Our preparation is cultural. We will accept VR as a representational scheme, no matter what its verisimilitude, in the same way that we accept a map of a city or the pieces on a chess board.
Technological development has always defined the location of frontiers. Medieval principalities were limited in scale by the speed of communication and the rate at which troops could be deployed, horsespeed and peoplespeed respectively. The atlantic coast of europe remained the edge of the world until explorers were liberated from coast hugging travel by accurate navigational technologies and robust ships. The American west was claimed and held only when the the steam locomotive, the telegraph and the conoidal bullet combined into one technological complex. More recently the space race advanced as soon as the technology was available.
With geography filled up, and the dreams of space colonization less viable every day, the drive to the frontier has collapsed in on itself. The space remaining for colonization is the space of the technology itself. No longer the tool by which the frontier was defined, the body of technology is now itself under exploration. Back in the early 60's, one of the pioneers of computer graphics, Ivan Sutherland, declared that the goal was to "break the glass and go inside the machine."[11] More recently, Jaron Lanier has said of VR that: "the technology goes away, and all that's left is the cultural component." The technology `goes away' because we are inside it.
It is important to distinguish between the history of technological development of VR, the pragmatic applications in the military, medical and architectural sectors; and the evangelical trumpeting, the technophilic rhetoric, of its populists and apologists. The kind of euphoria that characterizes these outpourings is remarkably consistent with previous technophilic hype that seems to have been an aspect of technological PR since the beginning of the industrial revolution, as is evidenced by this piece of C19th doggerel:
Lay down your rails, ye nations near and far--
Yoke your full trains to Steam's triumphal car.
Link town to town; unite with iron bands
The long estranged and oft embattled lands.
Peace, mild-eyed seraph--Knowledge, light divine,
Shall send their messengers by every line...
Blessings on Science, and her handmaid Steam!
They make Utopia only half a dream.[12]
Theodore Roszak notes the "salvational longings ... entwine themselves around new technology"[13] and give rise to artistic pinnacles such as these. This verse is an encapsulation of the Enlightenment calculus: Peace, Knowledge, Science, Technology sum to Utopia.
Thomas Edison imagined that his phonograph would find it's niche as acoustic `happy snaps', a way of preserving the voices of beloved relatives afer they died. He had no conception of the uses that corporate capitialism would put his invention to: the commodification of aural culture. Bazin makes similar observations regarding the cinema: " Those who had the least confidence in the future of the cinema were precisely the two industrialists Edison and Lumiere. Edison was satisfied with just his kinetoscope and if Lumiere judiciously refused to sell his patent to Melies it was undoubtedly because he hoped to make a large profit for himself, but only as a plaything of which the public would soon tire."[14] Brecht eulogized over the emancipatory potential of radio.[15] Television, contrary to the idealistic rhetoric of the early years, evolved not into an ideal democratic information network, but into a fantastic way to sell commodities and inculcate values. VR has inherited the liberationist, democratic rhetoric that has surrounded these previous waves of new technologies. Sadly, in these cases, the rhetoric stands as a bleak counterpoint to institutionalized application of these technologies, which tend to result in a greater degree of domination, manipulation and control. We must recognise that the current condition of utopian euphoria for VR represents a stage in the familiar history of the development of technologies in a laissez faire capitalist economic context. The utopian rhetoric, no matter how heartfelt by the inventor community, is ultimately very useful PR for the corporate merchants, it often obscures the military origins of the technology.
At the now legendary Virtual Reality panel at Siggraph'91, VR `came out' at least to a community of `25,000 of it's closest friends', as Siggraph members refer to the conference. During question time, I suggested to the panel that, to my knowledge, there had never been a case in the history of the world when a ruling group did not avail itself of the most advanced technology in order to consolidate or expand its power. I asked the panel why they thought Virtual Reality would be any different. I was not particularly surprised when the question was politely sidestepped.
It is common knowledge that tobacco and junk food corporations pay substantial amounts of money to have their product appear on Hollywood movies. Back in the `70's Richard Serra noted that in order to recieve the delivered television broadcasts, the consumer pays $40 for every dollar invested by the networks.[16] In the face of this, only a fool would wallow in the illusion that Virtual Reality will be any different. One can imagine possible Disneyland-style consumeristic virtual worlds with interactive and quasi- intelligent cans of Coke and cuddly giant hot-dogs with sexy giggles. A virtual supermarket where the products lean out at you from the shelves imploring you to buy them, explaining how you will be happier, healthier, sexier, wealthier...
The overt utopian rhetoric on the consumer front seems always to run parallel with, and obscure, simultaneous covert development in the military. The development of radio and transistorized electronics were spearheaded (together) by the military.[17] So it is with VR. It has been acknowledged that civilian VR is even now, 10 years behind the condition of the military machinery. Before 1985-86, all VR research was military. Although the Polhemus inductive sensor was originally developed for miners in Virginia, the military has been researching inductive sensing since 1967. Currently available commercial inductive sensors are 1% the ability of military sensors of 10 years ago.[18] [19]
Not unpredictably, the communities of engineers designing military simulations and theme park amusements are the same community; it comes as no surprise that the combat theme is such a consistent structuring strategy in interactive games. A VR virtual themepark called Battletech has existed in Chicago since mid 1991, it simulates futuristic warfare in which remote users do battle in simulated robotic war machines. In a quaint twist, each `player' is informed that he or she is the righteous vigilante fighting against the forces of evil. These `forces of evil' happen to be the other players, who are all laboring under the same delusion. Why does this scenario sound appallingly familiar?
Machine tools, including computers, are devices for exercising power over objects and sometimes people, but the role of the user in VR is essentially submissive. In VR one submits to the representation and the limited freedoms it offers: a postmodern capitalist paradise! Curiously this `submission' is analogous to the submission of the people who are employed to actually manufacture the hardware that runs these virtual worlds: In electronic sweatshops in Taiwan, Malaysia, Mexico and El Salvador, people (primarily women) labor to produce these goods which they will never consume. Alluquere Rossanne Stone remarked that Rene Desartes was able to `forget the body' only because he had sevants to attend to the needs of his. [20] Similarly users of VR implicitly expoit the labor of these 3rd world workers.
Can we interpret in this swing in VR technology from a paradigm of domination to a paradigm of submission as a `fin de siecle' malaise, a simultaneous decline in the will to control, and an acceptance of the overarching power of technology? Gilles Deleuze has built a new argument on similar terms, in which he outlines a general movement in the C20th from societies of overt discipline such as Foucault has described, to "ultrarapid forms of free floating control." Deleuze poignantly remarks: "Man is no longer man enclosed, but man in debt."[21] He describes: " ...a city where one would be able to leave one's apartment, one's street, one's neighbouhood, thanks to one's... electronic card that raises a given barrier; but the card could just as easily be rejected on a given day or between certain hours: what counts is not the barrier but the computer that tracks each person's position... and effects a universal modification. "[22] Not so much the spectre of a machine voice decreeing "Access Denied" as internalized control based on knowledge of one's electronic status.
It has been suggested recently (Joan Marks, Human Genome project) that human genetic testing based on a complete gene map will make it possible to detect genetic propensity to certain inherited disorders. It has also been suggested that medical insurance companies would offer `incentives' to encourage `wellness behavior' in such people. Such incentives might include not incurring a higher premium rate for having such propensities (propensities which may never materialize into a treatable disorder). Combining this idea with the projection that the average family of four in the US in the year 2000 will be paying $15,000 p/a in health insurance suggests highly effective strategies for behavior control.
Campuses around the country are currently instituting a combined debit and ID card (referred to in some places as an `all in one' card) with which students may borrow books, do laundry, work in the computer lab, buy food. Such a card will allow almost real time tracking of each card owner.
In the computerized workplace, real-time surveillance via the computer has been a reality for some time. Paranoia aside, your computer is watching you. Text workers are monitored in terms of keystrokes per minute, telephone salespeople in terms of calls per hour.
If such a vision is possible in the corporeal world, the prospect of realtime surveillance is so much more simply facilitated in VR. Not only will the computer know where you are, but what kind of information you are accessing, and where your various bodyparts are at the time. As digital media become increasingly encapsulating, so the possibility of permanent real-time surveillance becomes real. In a time when rapidly expanding access to electronic information also makes possible automated and invisible systems of surveillance, we must note that the point at which the protection of liberty in the electronic terrain becomes an invasion of privacy is hazy indeed.
In terms of corporate economics, VR serves the computer industry very well. It is intuitive (no learning curve, no consumer resistance) and calls for unlimited computer power. It thus fulfills the industry's need for technological desire.The transference of libidinal desire onto fetish objects which offer the promise of ecstasy but never finally consumate, driving the consumer to the next purchase in an unending coitus interuptus.
We have no reason to delude ourselves that any new technology, as such, promises any sort of socio-cultural liberation. History is against us here. We must assume that the forces of corporate consumerism will attempt to fully capitalize on the phenomenon in terms of financial profit, and that the potential for surveillance and control will be utilised by corporate and state instrumentalities. As educators, we have a responsibility to develop a critical consciousness of these possibilities, the better to prepare our students to deal with the highly technologized lifestyle of the early decades of the 21st century. As they say, the future isn't what it used to be.
One of the claims made of VR is that it constitutes a liberation from the mind-body duality. It is argued that VR achieves this by sidestepping the process of translation into, and out of, symbolic representation. (This is referred to as post-symbolic communication in VR cognitive science circles.) This claim is, in my view, questionable. One does not take one's body into VR, one leaves it at the door. VR reinforces the Cartesial duality, replacing the body with a body image, a creation of mind, as all objects in VR are a product of mind. As such it is a clear continuation of the rationalist dream of disembodied mind, part of the long western tradition of denial of the body.
But the term `body' should be clarified. When we discuss the body, it seems to be as two quite different perceptual roles. We can discuss the body as a thing which is perceived, (and understood to be the physical manifestation attached to the observing mind), and we can also discuss the body as the thing which does the percieving of other things outside the body. This distinction applies to the corporeal body and it also applies to the body representation in VR.
Lanier argues that "the way you talk to your body doesn't use symbols"[23]. Fair enough, but what is then suggested to be a logical corollary doesn't follow: "you can make a cup that someone else can pick up...without ever having to use a picture or the word `cup'...you create the experiential object `cup' rather than the symbolic object"[24] But a cup in VR is a representation, it is a stereographic image.You can't drink out of it.
Consider three modes of communication of a piece of mathematical information concerning a geometrical form: a written equation, a plaster model, and a VR model. Now it is fair to say that handing a co-participant in a shared VR a virtual object in the shape of a hyperbolic paraboloid is a more directly communicative gesture that to offer a written complex algebraic equation which must be decoded by the mind of the receiver. A corporeal plaster cast of the form communicates information that is directly perceived due to our understanding of three dimensional form, but no mathematical analysis is available. The VR representation is doubly useful as the object is itself a visual representation of the data encapsulated in the equation, and this mathematical information remains available.
William Bricken maintains that all the operations of symbolic logic can be performed in VR without recourse to symbolic languages, that logic is equivalent to inference in visual programming. Set theory, number theory, algebra can all be represented as objects in space that is non-symbolic and totally math-rigorous! Binary logic can be represented as open and shut doors, knot theory as fish swimming upstream over dams. "All computation is algebraic pattern matching and substitution (proven)" [25]
There is clearly a paradigm shift in the VR experience, but it does not bypass the symbolic and replace it with an experience which is indistinguishable from corporeal experience. The VR representation is an interactive stereographic representation, it is an automation of pictorial representation. The appelation Virtual Reality is unfortunate, it makes the same sort of untenable claims for the technology that the term Artificial Intelligence did for that discipline. I would prefer to discuss VR as a special augmented case of representation, such that the object is simultaneously a representation and an experiential phenomenon. VR directly interfaces with the body, the kinesthetic, bypassing textual language; but it remains a pictorial representation and is thus subject to critical analysis as such. What is required is a new critique, a way of thinking about the meeting point between the immediate physiological reality of the body as lived in, and culturally specific conventions of representation.
The technology `goes away', the interface becomes invisible. This is the stated goal of VR from Ivan Sutherland on. To interface with pure information where the act of `driving' the technology disappears. For the moment the simulation is incomplete, the interface is apparent. In the (technological) limit all sense inputs will be synthesized and simulated. But there is a paradoxical aspect to increased verisimilitude of simulation: as the representation becomes increasingly complex, the gap yawns: the greater precision only more clearly defines the ways and degrees in which the representation will not stand for the reality. This is all rather reminiscent of Arthur Dent's tea dilemma:
"After a fairly shaky start to the day, Arthur's mind was beginning to reassemble itself from the shell-shocked fragments the previous day had left him with. He had found the Nutri-matic machine which had provided him with a plastic cup filled with a liquid that was almost, but not quite, entirely unlike tea."[26]
That VR is incomplete is clear at even a cursory inspection. As a representation, VR is an abstraction. The question is not: how abstract a representation will the mind/body accept as one which will stand in for reality?, because I have suggested that the use of the `R' word in this context is obfuscating. The question becomes: what constitutes a continuous interactive representation? What arrangement of images and interactive cues cohere into a system with syntactic order? This question is made complex due to the confounding malleability of the mind, what William Bricken refers to as "cognitive remodelling". The mind, it seems, is very willing to restructure itself to compensate for or adapt to, a changing `reality'. There is a peculiar cognitive feedback loop here: VR, standing in for `reality' begins to shape the way the mind describes its experience to itself. The current state of the VR image is extremely simple, built as it is from several thousand polygons. Even so, William Bricken reports that as one interacts with a virtual world, one comes to accept the polygonal representations. It becomes as valid a world as the `real'. VR people refer to this as `cognitive plasticity'.[27] Thomas Furness relates that if you spend a lot of time in VR, you begin to dream in polygons! [28] Jaron Lanier's oft-quoted definition that "reality is what is on the other side of the senses" is validated by these experiences.
The VR representation is ultimately as schematic as that of a map or a chess board. These are schematic representations which are culturally learned and from which we draw meaning. One of the techniques of virtual world design, as in other computer interface design, is to utilise familiar symbols and terminology to indicate to the user that the computer system has been modelled on a familiar `real world' system (ie: the folders and trash can of the Mac interface). The learning curve is less steep because relationships are familiar. Reduced to this level, the problematics of representation in VR evaporate.
There are two alternatives for VR systems. Either the virtual world is `mapped onto' the real world in such a way that the characteristics of the virtual world and the real world are not in conflict, so no dislocation is experienced. This is the theory of the `head-up' display systems first used by the military and now being developed for VR medical diagnosis expert systems. Alternatively, the visual and auditory world is shut out, and replaced with another. This leaves one part of what we might call the sensorial body in the corporeal world, and the other in the virtual world. Can the body/mind cope with such fragmentation? Numerous studies and anecdotes testify to the wondrous maleability of the mind. Systems without even internal consistency seem to be adopted seamlessly, given time. Adaption indicates also that the mind will willingly close down sensory channels at odds with other more dominant channels. There do seem to be a problems when the closed down channels are being reactivated.
Simulator sickness arises from disconnected sensory modalities. "sitting still' in a flight simulator in which the image material presents the visual experience of `rolling' requires the mind to preference the visual input and ignore the kinesthetic information from the semi-circular canals, to switch it off. This the mind happily does, but it takes at least 24 hours for the sense of balance to `re connect'. During that time people tend to fall over a lot. As a result, the navy prohibits the piloting of a plane within 24 hours of being in a flight simulator.
To simulate all sensory input in a coordinated way might be referred to as "total body representation". The quest for total body representation is "the kind of obsessive project that characterizes the activities of engineers in the realm of cultural production" (see above). But for the sake of argument it is instructive to examine what it implies. If in VR, I am confronted with a cast iron chair, a typists chair and a lounge, I must not only be able to sit, but the sensation of texture must change. This implies a full `force-feedback'[29] suit unimpeding and light enough to be unnoticable by the wearer. In 1974 Theodor Nelson coined the term `teledildonics' to denote sex experiences utilising such futuristic hardware. But sex at a distance with a teleoperated robotic partner will remain sci-fi for the forseeable future.
In order for a fully simulated representation of the body to be complete, the realm of the kinesthetic, proprioceptive sense and sense of balance must be catered for. The internal body senses must be `represented': How can we electronically simulate the sense of a distended stomach? Sense of taste and smell are also absent. When can I eat virtual food and excrete virtual shit? Clearly no amount of external gadgetry will facilitate total body representation. For better or worse, direct neural jacks seem to loom on the horizon, and this technology will radically change the terms of this discussion (see below).
The VR condition that we are thus discussing is the limited case of a simulated interactive steroscopic visual (and occasionally auditory) environment, in which the body is represented purely visually. The prospect of a partial but coordinated and articulated representation of the body raises the question of the repercussions (both psychological and cultural) of a double body. VR replaces the body with two partial bodies: the corporeal body and an (incomplete) electronic `body image'. In terms of the rhetoric there is no question which is in the ascendant. This is a kind of sensory apartheid. VR leaves the meat body on the chair. A confirmation, rather than a liberation from, Cartesian Dualism. VR is thus about dislocation and dissociation. Simulator sickness testifies to this dislocation, it is the first virtual illness. The body representation of VR fragments the body: a powerful eye mounted on a fractured body.
In 1896-7, GM Stratton devised an experiment to invert the visual field, using an inverting lens attached to the eye. In an eight day experiment, he found that by the 5th day he could move about the house with ease. This experiment suggests, to use a gross mechanical analogy, that the entire optic nerve bundle was re-patched! He notes that when the lenses were removed that the scene had a bewildering quality, but there was no sensation that the field was upside down. So perhaps the mind develops a `conditional' model of the world, under the effects of the inverting lenses, but reverts to the `normal' model readily . This would be rather like riding a bicycle as opposed to walking. This idea is corroborated by experiments by J and JK Paterson. They found that the adaptions made by subjects in order to function with the inverting lenses were instantly recalled when using them again after a lapse of 8 months. You don't forget how to ride a bicycle.
Stratton developed another experiment using a mirror arrrangement mounted on a shoulder harness which threw his body image out horizontally in front of him. He reported: "I had a feeling that I was mentally out of my own body..." This report is corroborated by medical records of a patient who had vertebrae removed in surgery and reported that he felt that he was viewing the world through his forhead.
People who have a lesion of the parietal lobe in one hemisphere of the brain "have been known to push one of their own legs out of a hospital bed because they were convinced it belonged to a stranger. Such behavior shows that the damaged area normally imparts a signal that says: "This is my body, it is part of myself.""[30] There is a rare neurological disease which entirely eradicates the kinasthetic sense, (the internal sense which alows you to know where your hands are with your eyes closed), the tragic upshot of this being that you have no sense of inhabiting your own body.
Stratton summed up his experiments by saying : "The different sense-perceptions, whatever may be the ultimate course of their extension, are organised into one harmonious spatial system. The harmony is found to consist in having our experiences meet our expectations...The essential conditions of harmony are merely those which are necessary to build up as reliable cross reference between the two senses." [31]
Where do these expectations come from? From being in the world, experientially cross-relating sense information in early chidhood, one supposes. Richard Gregory observes that for compensation to take place in a condition of altered sense information, it is essential that the subject make active corrective movements. He cites the experiments of Richard Held, who finds that active arm movement (striking a target with a finger) is necessary for effective adaptation. Gregory asks: "is the adaptation perceptual or proprioceptive...[?]"[32] He does not pursue this question with but it is central to the consideration of the cognitive psychology of VR.
The interconnectedness of the `sensorial body' or as Stratton would have it, the `harmonious spatial system', is borne out in another experiment by Held. Two kittens, reared in total darkness, were fitted in a gantry arrangement with two baskets. One basket had holes for the legs such that the physical movements of one kitten would drive both animals through roughly similar spatial experiences. The kitten which could associate visual information with its own physical movement developed effective vision, the other remained functionally blind.[33]
These examples have some bearing on the understanding of the body in VR. William Bricken has proposed `super-binocular' vision in VR in which the (virtual) eyes are six feet apart, allowing increased depth perception. In such cases, the existence of a virtual hand would not be simply a useful addition, but a necessity if the perpectually different virtual world was to make sense at all. Without the active exploratory hand, this world would remain unintelligeble. The construction of a new `sense of body' (even of the limited body of one hand and a pair of eyes) is effected by the physical exploration of this new perceptual space with the hand, and the cross correlation of internal kinesthetic data with visual input. Meredith Bricken has suggested that VR "allows as many dimensions as we need".[34] But our bodies and senses are structured for navigation of 2 1/2 dimensions only. I question the possibility of learning to navigate in multi-dimensional space.
As all objects in a virtual world are constructed, so is the body image itself. In `designer reality, the shape and style of the body you take into VR is an open choice. One can design a body with numerous limbs, say a giant lobster, and by attaching additional sensors to knees and elbows to control the extra limbs, one can comfortably inhabit a body with double the regular complement of limbs.The mind maps to this new body almost effortlessly. That is, you begin to instruct your left knee to move, fully knowing that it is in fact the third foot down on the left side. In the case of the Giant Lobster, Lanier reports that it takes only 2-3 minutes to remap arbitrarily placed sensors as controllers for extra limbs, ie: sensors on chin or knee. It is difficult to reconcile this report with the experiments in visual perception discussed above, as it suggests that the mind can quite quickly draw a new `internal body representation' to allow control of the new body. This effect seems to be somewhat at odds with the notion of the neurological homunculus inscribed on the brain. The arbitrary body suggests a way of understanding virtual body articulation as `hyper- marionettry', with the homunculus functioning as a temporary map or I/O program, as opposed to `hardwired' circuitry.
Randy Walser and Eric Gulichsen (who should know better) have recently been quoted as saying: " In cyberspace there is no need to move about in a body like the one you possess in physical reality. As you conduct more of your life and affairs in cyberspace, your conditioned notion of a unique and immutable body will give way to a far more liberated notion of `body' as something quite disposable and, generally, limiting. You will find that some bodies work best in some conditions while others work best in others..." This is a confusion, there is no need for a body at all in VR except for narcissistic or gaming purposes. All one requires is an indication of the location of your VR effectors with respect to your virtual viewpoint. As the entire physical body is represented in VR by a larger and larger array of interface points, the potential diversity of one's image in VR will become more limited. The variety is possible now only because you can put any shape between the image of the glove and the virtual viewpoint. Walser and Gullichsen continue: "... The ability to radically and compellingly change one's body image is bound to have a deep psychological effect, calling into question just what you consider yourself to be" [35] Indeed!
The experiments in visual perception cited above, and the VR experiences, seem to suggest that our `sense of self', our sense of place in the world, remains consistent and continuous purely because external reality has a certain continuity to it. That we have no internal continuous self image. If self image is volatile and only a stable `reality' enforces a stable self image. (My front door tends to be the same size, shape and color each time I go through it, my left hand usually has five fingers.) What then are the effects of long term immersion in VR, of adopting alternative bodies, and what are the effects of `paddling' (in out in out) of a variety of bodies in a variety of worlds? Could the Walser and Gullichsen experience induce schizophrenia ?
One might argue (somewhat simplistically) that the technologies of robotics, space-travel and virtual reality itself, have their roots in literature. Robotics in Kapek's RUR and Fritz Lang's Metropolis (etc), space travel in Buck Rogers and the early Gernsback Sci-Fi (etc), virtual reality in Disneyland.[36] Another persistent aspect of our sci-fi mythology is the prospect of symbiotic integration of human organism and machine. I want to consider some of the implications of the development of the `direct neural jacks' that William Gibson has envisaged, from the perspective of neurology and studies in perception.
The mind cannot distinguish between actual heat applied to the hand, and the artificial stimulation of nerves someplace between the hand and the brain. Already nerve interface chips are being developed such that amputees will drive robotic prosthetics through articulation of the `ghost' limb. So Laniers' definition of reality as being that which is `on the other side of the senses' may only be conditional.
The neurophysiological reality of cognitive remapping is indicated by data from the applied science of prosthetics. Current technology allows the control of motors in a prosthetic arm by external electrodes on the skin adjacent to the muscle mass. This technology is referred to as Myoelectrics. Even in such hamfisted approaches, the selection of which muscle is somewhat arbitrary, muscles can be retrained such that the mind will trigger a muscle in the thorax to control the prosthetic wrist, and this transfer or re-training becomes `second-nature'. Performance artist Stelarc demonstrates this forcefully with his ability to write simultaneously with three hands, one of which is a robotic prosthetic attached at the right elbow.
Several years ago Bernard Widrow and his colleagues at Stanford developed experimental implantable microchips, not yet in the brain, but in the stumps of amputees. These chips contain fissures through which severed nerve-ends were encouraged to grow. The chips contain analog to digital converters which would allow the digital electronic control of servo-motors in robotic prostheses.[37]Experiments on the control of robotic prostheses from nerve impulses from the stumps of amputees enlarge on this understanding. There seems no necessity to locate precisely the nerve for the control of the contraction of the thumb in order to control a servo motor in a robotic prosthesis to control the robot thumb. Any nerve will do! It might be the on that controls rotation of the wrist, it doesn't matter. The mind re-patches at its end of the nerve bundle.
How does this neurological research impact the virtual body problem? How readily will the mind map to the arbitrary virtual body form? How long will it retain that mapping as a memory and how many maps will the mind accomodate simultaneously? These and related questions remain unanswered.
The location of the site in the nervous system or brain where these remapping operations occur is extremely fugitive. Recent literature is apt to induce vertigo as one plumbs a conception of mind that is incompatible with these gross mechanical metaphors of plugs and sockets. Ronald Melzack has recently noted that phantom limb pain seems not to originate in the amputees stump, nor in the spinal chord or the brain stem, but in a complex parallel network of higher brain functions.[38]
Back in the world of gadget- science, Thomas Furness relates that the cumbersome eyephones will be made obsolete by laser scan direct onto the retina via a miniature hybrid laser chips which will appear as small dots in the center of each lens of a pair of very dark RayBans. This image is always in focus, located at ocular infinity. He predicts that this technology will be developed in a few years, the entire system will weigh a few ounces and cost $2000.[39]
The prognosis is that the VR interface gadgets will get smaller and lighter before they disappear. While they remain attached to the outside of the body and track actual physical movement we can continue to speak about a dislocation between the virtual body and the corporeal body. But when direct neural jacking synthesizes the entire body experience, the terms of this discussion will radically change.
Having discussed the representation of the body in VR, it is appropriate to discuss the environment which this body inhabits. The virtual world is a product of design, nothing is there by accident. That no representation is absolute or value free is a familiar idea, but in VR, attitudes are embedded not only in the construction of the image, but in the design of the tools and pathways for interaction. It remains the work of a particular human mind. Terry Winograd and Fernando Flores discuss at some length the dependance of a `representation' upon a background of (culturally specific) assumptions.[40] Imagine, to take a simplistic example, a VR in which the only tool for interactivity was a gun. One might dispose of enemies and hit targets, but the prospect for a caress or intellectual research are extremely limited. In entering someone's VR, it is accessible to us to the extent that we share a cultural background.
Interactive media inhabit a nether-world between two conditions: simulations, which purport to represent the world in some sense, and interactive fiction. Simulation systems are employed increasingly to demonstrate and to inductively `prove' in science and mathematics. The difficulty is that as the verisimilitude of the representation increases, the simulation can be approached in a self contained game-playing mode, where to goal is to attain a `maximum score' in the terms of the model, complete separation being achieved from the corporeal situation the model purports to represent. We might call this the `Nintendo syndrome', the closed lop of self serving simulation which Harun Farocki has referred to as "pure military fiction". [41] The simulation is taken to be `true' on the basis of its coherent structure of representation, while the database that provides the source statistics for the representation may be inaccurate or incomplete.
Beneath the veil of the rhetoric of `free' interaction, interactive fiction media offer varying degrees of heavily determined relating. Though not necessaily structurally hierachical, the networks of paths and links are rock solid, defined by the maker. A recent developers paper on interactive fiction asserted that giving the user a sense of existing in a virtual world "...must be done in such a way as to leave the user with an undiminished feeling of free will"[42] But encouraging a viewer to believe that he has created the experience is subterfuge amounting to covert manipulation. What does it mean to have "an undiminished feeling of free will" ? When I visit the zoo I am happy to walk on the paths and I understand why I am prevented from entering the rhinoceros enclosure.
Although little research has been done on the psychology of interaction in VR, some researchers have addressed the subject in non- virtual interactive media. Grahame Weinbren observes: "I reject... the paradigm of interactive cinema as a machine to invent stories to match the viewers desires." [43] Such a machine "...connects to the viewers brain and externalizes his fantasies, a machine that produces psycho-cinema without any real production." So such a psycho-cinema offers a spectre of unrestrained libidinal fantasy without the responsibility of `reaction' [44] . The machine cannot gossip or blame, it offers `deniability' In the words of Bart Simpson: "I didn't do it and anyway nobody saw me." Unfortunately hollywood cinema already produces this kind of psycho-cinema, albeit on a demographic basis, as does any market driven production.
Weinbren continues: "I am not interested in an interface which would cause the viewer to believe that he has created the story. Not only is this a false belief, but it presupposes an invalid esthetic. Art functions to widen one's perception and experience, not to narrow it, to show the viewer a vision of the world, not to make him imagine he has invented one...I believe the requirement that the viewer make choices is not a liberation but a burden. My position on this issue goes against the purported link between choice and freedom fundamental to western ideology. The connection between freedom and responsibility is glossed over in the same ideology, but it is the feeling of responsibility for what appears on the screen (without having chosen it) that interests me. The general paradigm that I propose is that of a machine that answers to a viewers response. " [45]
Winograd and Flores base their discussion of human-computer interaction on the premise that the computer is essentially an extension of language: "The rationalistic tradition takes language as a representation,-a carrier of information- and conceals its central social role. To be human is to be the kind of being that generates commitments, through speaking and listening. Without our ability to accept (or decline) commitments we are acting in a less than fully human way, and we are not fully using language." "This key role [of speech act theory] develops from the recognition that computers are fundamentally tools for human action. Their power as tools for linguistic action derives from their ability to manipulate formal tokens of the kinds that constitute the structural elements of language. But they are incapable of making commitments and cannot themselves enter into language." [46] "In [computer systems literature] there is pervasive misunderstanding based on the failure to recognize the role of commitment in language."[47]
It is to the great credit of these authors that they remind us that computer systems are ultimately designed by people, and that they are, or ought to be, devices for facilitating communication between people. Winograd and Flores neutralize the possibility of putting the blame on an external object, the computer, as if it had intentions, and place the responsibility squarely on the shoulders of the designers (and users) of such systems.
I hope to have indicated that the ideas that have constructed Virtual Reality are not new. Not simply that many of the issues I have raised are relevant to a broader spectrum of computerized culture than simply VR. Rather, that the conception of VR has deep roots in our culture. It is therefore important to bring a cultural critique to bear on the matter.
Historically, technological development projects have been considered by their developers as non-continuous with the world of everyday experience. Virtual Realities must not be considered in this way, nor should the developers of these environments be encouraged to think in such a way. It is the fabric of everyday culture that lends, and confines, meaning in these virtual worlds. The developers and their worlds are immersed in, and informed by, the contemporary culture which is itself informed by cultural history.
Simon Penny March 1992