Experiment is Accident

Interview with activist, artist and inventor Steve Mann, by Arjen Mulder, published in "The Art of the Accident," 1998.

Experiment is Accident

The Art of the Accident, 1998

I do not talk to strangers.

Therefore you must slide a government-issued ID card through the slot on my head if you want to talk to me. These SAFETYGLASSES prevent me from seeing or hearing you until you identify yourself! Until you provide positive ID, the camera and microphones on my head will not be connected to my head mounted display set. Your time is very important to me, so please wait for my next available moment!

If you would like to try to sell me a new product, press 1.
If you would like to ask me to fill out a form, press 2.
If you would like to show me an advertisement, press 3, and slide your credit card through my slot to purchase my attention. If you would like to inform me that photography is not permitted on your premises, press 9, and wait for my next available moment. For quality-control and training purposes, this conversation may be recorded or monitored.

Arjen Mulder: Steve, what is all that stuff on your head and body?

Steve Mann: Wearable computing facilitates a new form of human-computer interaction comprising a small body-worn computer system that is always on and always ready and accessible. Wearable computing will now be defined in terms of its three modes of operation and its six attributes. There are three operational modes in this new interaction between human and computer:
Constancy: The computer runs continuously, and is "always ready" to interact with the user. Unlike a hand-held device, laptop computer, or personal digital assistant (PDA), it does not need to be opened up and turned on prior to use. The signal flow from human to computer, and computer to human, depicted in Fig 1a runs continuously to provide a constant user-interface. [http://wearcam.org/wearcompdef/fig1a.gif]
Augmentation: Traditional computing paradigms are based on the notion that computing is the primary task. Wearable computing, however, is based on the notion that computing is not the primary task. The assumption of wearable computing is that the user will be doing something else at the same time as doing the computing. Thus the computer should serve to augment the intellect, or augment the senses. The signal flow between human and computer is depicted in Fig 1b. [http://wearcam.org/wearcompdef/fig1b.gif]
Mediation: Unlike hand held devices, laptop computers, and PDAs, the wearable computer can encapsulate us (Fig 1c). [http://wearcam.org/wearcompdef/fig1c.gif]
It doesn't necessarily need to completely enclose us, but the concept allows for a greater degree of encapsulation than traditional portable computers.
There are two aspects to this encapsulation:

  • Solitude: It can function as an information filter, and allow us to block out material we might not wish to experience, whether it be offensive advertising, or simply a desire to replace existing media with different media. In less severe manifestations, it may simply allow us to alter our perception of reality in a very mild sort of way.
  • Privacy: Mediation allows us to block or modify information leaving our encapsulated space. In the same way that ordinary clothing prevents others from seeing our naked bodies, the wearable computer may, for example, serve as an intermediary for interacting with untrusted systems, such as third party digital anonymous cash "cyberwallets." In the same way that martial artists, especially stick fighters, wear a long black robe that comes right down to the ground, in order to hide the placement of their feet from their opponent, wearable computing can also be used to clothe our otherwise transparent movements in cyberspace. Although other technologies, like desktop computers, can help us protect our privacy with programs like Pretty Good Privacy (PGP), the Achilles tendon of these systems is the space between us and them. It is generally far easier for an attacker to compromise the link between us and the computer (perhaps through a so-called Trojan horse or other planted virus) than it is to compromise the link between our computer and other computers. Thus wearable computing can be used to create a new level of personal privacy because it can be made much more personal, e.g. so that it is always worn, except perhaps during showering, and therefore less likely to fall prey to covert attacks upon the hardware itself. Moreover, the close synergy between the human and computers makes it harder to attack directly, e.g. as one might peek over a person's shoulder while they are typing, or hide a video camera in the ceiling above their keyboard. Furthermore, the wearable computer can take the form of undergarments that are encapsulated in an outer covering or outerwear of fine conductive fabric to protect from an attacker looking at radio frequency emissions. The actual communications between the wearer and other computers (and thus other people) can be done by way of outer garments, which contain conformal antennas, or the like, and convey an encrypted bitstream. Because of its ability to encapsulate us, e.g. in embodiments of wearable computing that are actually articles of clothing in direct contact with our flesh, it may also be able to make measurements of various physiological quantities. Thus the signal flow depicted in Fig 1a is also enhanced by the encapsulation as depicted in Fig 1c. To make this signal flow more explicit, Fig 1c has been redrawn, in Fig 1d, where the computer and human are depicted as two separate entities within an optional protective shell, which may be removed or partially removed if a mixture of augmented and mediated interaction is desired. [http://wearcam.org/wearcompdef/fig1d.gif]
    Wearable computing is a framework for enabling various degrees of each of these three fundamental modes of operation. Collectively, the space of possible signal flows giving rise to this entire space of possibilities, is depicted in Fig 2. [http://wearcam.org/wearcompdef/fig2.gif]
    While individual embodiments of wearable computing may use some mixture of these concepts, the signal path depicted in Fig 2 provides a general framework for comparison and study of these systems. The signal paths typically each, in fact, include multiple signals, hence multiple parallel signal paths are depicted in this figure to make this plurality of signals explicit.

AM: But wait a minute ...

SM: There are six informational flow paths associated with this new human-machine synergy. These signal flow paths are, in fact, attributes of wearable computing, and are described, in what follows, from the human's point of view. Two additional attributes follow from these six, and are also associated with wearable computing. The six attributes of wearable computing are as follows:

  1. Unrestrictive to the user: ambulatory, mobile, roving, "you can do other things while using it", e.g. you can type while jogging, etc.
  2. Unmonopolizing of the user's attention: it does not cut you off from the outside world like a virtual reality game or the like. You can attend to other matters while using the apparatus. It is built with the assumption that computing will be a secondary activity, rather than a primary focus of attention. In fact, ideally, it will provide enhanced sensory capabilities. It may, however, mediate (augment, alter, or deliberately diminish) the sensory capabilities.
  3. Observable by the user: It can get your attention continuously if you want it to. Almost-always-observable: within reasonable limitations (e.g. that you might not see the screen while you blink or look away momentarily) the output medium is constantly perceptible by the wearer.
  4. Controllable by the user: Responsive. You can grab control of it at any time you wish. Even in automated processes you can manually override to break open the control loop and become part of the loop at any time you want to (example: "a big Halt button you want as an application mindlessly opens all 50 documents that were highlighted when you accidentally pressed "Enter" would make a computer more controllable). Infinitely-often-controllable: the constancy of user-interface results from almost-always observability and infinitely-often controllability in the sense that there is always a potential for manual override which need not be always exercised.
  5. Attentive to the environment: Environmentally aware, multimodal, multisensory. (As a result this ultimately gives the user increased situational awareness).
  6. Communicative to others: Can be used as a communications medium when you want it to. Expressive: allows the wearer to be expressive through the medium, whether as a direct communications medium to others, or as means of assisting the production of expressive media (artistic or otherwise).

Two additional properties that follow from the above are:
Constant: Always ready. May have "sleep modes" but never "dead." Unlike a laptop computer which must be opened up, switched on, and booted up before use, it is always on and always running.
Personal: Human and computer are inextricably intertwined.

  • Prosthetic: You can adapt to it so that it acts as a true extension of mind and body; after time you forget that you are wearing it.
  • Assertive: can have barrier to prohibition or to requests by others for removal during times when you wish such a barrier. This is in contrast to laptop computer in briefcase or bag that could be separated from you by a "please leave all bags and briefcases at the counter" policy of a department store, library, or similar establishment.
  • Private: others can't observe or control it unless you let them. Others can't determine system status unless you want them to, e.g. clerk at refund counter in department store where photography is prohibited can't tell whether or not you are transmitting wireless video to a spouse for remote advice, in contrast to camcorder technology where it is obvious you are taking a picture when you hold it up to your eye. Note that a computer mediation device with sufficient bandwidth can synthesize or even heighten the augmentational aspects. For example a sufficiently attentive computer can sustain a sufficient illusion of being unmonopolizing that it could encapsulate the user and still provide the same experience as system running in the augmentational mode of operation. Similarly, a sufficiently communicative machine, especially if "machine" is broadened to include mechanical mediation devices such as motorized exoskeletons, can synthesize the unrestrictive attribute.

AM: But what I want to ...

SM: The most fundamental issue in wearable computing is no doubt that of personal empowerment, through its ability to equip the individual with a personalized, customizable information space, owned, operated, and controlled by the wearer. While home computers have gone a long way to empowering the individual, they only do so when the user is at home. As the home is perhaps the last bastion of space not yet touched by the long arm of surveillance - the home computer, while it does provide an increase in personal empowerment, is not nearly so profound in its effect as the wearable computer which brings this personal space - space one can call one's own - out into the world. Consumer technology has already brought about a certain degree of personal empowerment, from the portable cassette player that lets us replace the music piped into department stores with whatever we would rather hear, to small hand held cameras that capture police brutality and human rights violations. However, wearable computing is just beginning to bring about a much greater paradigm shift, which may well be equivalent in its impact to the invention of the stirrup, or that of gunpowder. Moreover, this leveling of the playing field may, for the first time in history, happen almost instantaneously, should the major consumer electronics manufacturers beat the military to raising this invention to a level of perfection similar to that of the stirrup or modern handguns. If this were to happen, the time scale over which technology diffuses through society will have decreased to zero, resulting in a new kind of paradigm shift that society has not yet experienced.

AM: Doesn't your body protest against these prostheses?

SM: Rather than attempting to emulate human intelligence in the computer, as is a common goal of research in Artificial Intelligence (AI), the goal of wearable computing is to produce a synergistic combination of human and machine, in which the human performs tasks that it is better at, while the computer performs tasks that it is better at. Over an extended period of time, the wearable computer begins to function as a true extension of the mind and body, and no longer feels as if it is a separate entity. In fact, the user will often adapt to the apparatus to such a degree, that when taking it off, its absence will feel uncomfortable, in the same way that we adapt to shoes and clothing to such a degree that being without them most of us would feel extremely uncomfortable whether in a public setting, or in an environment in which we have come to be accustomed to the protection that shoes and clothing provide. This intimate and constant bonding is such that the combined capabilities of the resulting synergistic whole far exceeds the sum of either. Synergy, in which the human being and computer become elements of each other's feedback loop, is often called Humanistic Intelligence (HI).

AM: Aha! Now any new philosophy consists of a set of new ideas and practices, and a critique of older ideas and practices. What do you criticize?

SM: The perceived "success" of video cameras in banks has led to their use in department stores, first at the cash register and then throughout the store, monitoring the general activities of shoppers. "Success" there has led to governments using ubiquitous surveillance throughout entire cities to monitor the general activities of citizens. (In Baltimore, throughout the downtown area, the government installed 200 cameras as part of an experiment that, if "successful," would mean other cities would also be so equipped.) Businesses such as the Sheraton Hotel have used hidden video cameras in their employee locker rooms, and the use of hidden cameras by both businesses and governments is increasing dramatically. The recent proliferation of video surveillance cameras interconnected with high-speed computers and central databases is moving us toward a high-speed "surveillance superhighway," as cameras are used throughout entire cities to monitor citizens in public areas. As businesses work alongside governments to build this superhighway and expand it into private areas as well, there is a growing need to develop methodologies of questioning these practices.

I propose "Reflectionism" as a new philosophical framework for questioning social values. The Reflectionist philosophy borrows from the Situationist movement in art and, in particular, an aspect of the Situationist movement called détournement, in which artists often appropriate tools of the "oppressor" and then resituate these tools in a disturbing and disorienting fashion. Reflectionism attempts to take this tradition one step further, not only by appropriating the tools of the oppressor, but by turning those same tools against the oppressor as well. I coined the term Reflectionism because of the "mirrorlike" symmetry that is its end goal and because the goal is also to induce deep thought ("reflection") through the construction of this mirror. Reflectionism allows society to confront itself or to see its own absurdity. One of my goals in applying Reflectionism to the surveillance problem is to allow representatives of the surveillance superhighway to see its absurdity and to confront the reality of what they are doing through direct action or through inaction (blind obedience to a higher and unquestionable authority).

AM: What's that got to do with wearable computers?

SM: My WearComp invention (wearable computer with visual display means) formed the basis upon which I built a prosthetic camera called WearCam, which was worn rather than carried and could be operated with both hands free - and thus while doing other things. A wireless connection to the Internet provided offsite backup of all image data, facilitating another aspect of the Reflectionist philosophy - namely, as far as destruction is concerned, to put the pictures beyond the reach of totalitarianist officials. Just as an individual cannot rob a bank and then destroy the video record (because the video is recorded or backed up offsite, or is otherwise beyond the bank robber's reach), my apparatus of détournement put the images beyond the destructive reach of members of the establishment, because of the Internet connection, which allowed for offsite backup of all images at various sites around the world.

WearCam-on-the-WWW extends this "personal safety" infrastructure and further deters representatives of an otherwise totalitarian regime from being abusive: on one hand, I have collected the indestructible evidence of hostile totalitarian actions, and on the other, my friends and relatives are quite likely to be watching, in real time, at any given moment. This process is a form of "personal documentary" or "personal video diary." "Wearable Wireless Webcam" challenges the "editing" tradition of cinematography by transmitting, in real time, life as it happens, from the perspective of the surveilled. Furthermore, because I am merely capturing measurements of light (based on the photometric image composite, which represents the quantity of light arriving from any angle to a particular point in space), which are then yet to be "rendered" into a picture, I may choose to leave it up to a remote viewer operating a telematic virtual camera to make the choices of framing the picture (spatial extent), camera orientation, shutter speed, exposure, etc. In this way I may absolve myself of responsibility for taking pictures in establishments (such as department stores) where photography is prohibited, for I am merely a robot at the mercy of a remote operator who is the actual photographer (the one to make the judgment calls and actually push the virtual shutter release button). In this manner, an image results, but I have chosen not to know who the photographer is. Indeed, the purpose of these personal documentaries has been to challenge representatives of the video surveillance superhighway who at the same time prohibit photography and video.

AM: Could you give an example?

SM: ShootingBack was a meta-documentary (a documentary about making a documentary). Since I am a camera, in some sense, I do not need to carry a camera, but in ShootingBack, I did anyway. This second camera, an ordinary hand-held video camera, which I carried in a satchel, served as a prop with which to confrontmembers of organizations that place us under surveillance. First, before pulling the camera out of my satchel, I would ask store representatives why they had cameras pointing at me, to which they would typically reply "Why are you so paranoid?" or "Only criminals are afraid of the cameras." All this, of course, was recorded by my WearComp/WearCam apparatus concealed in an ordinary pair of sunglasses. Then I would open up my satchel and pull out the hand-held video camera and point it at them in a very obvious manner. Suddenly they had to swallow their own words. In some sense, ShootingBack caught "the pot calling the kettle black." To further the Reflectionist symmetry, I also experimented with wearing some older, more obtrusive versions of WearComp/WearCam, which I described to paranoid department store security guards as "personal safety devices for reducing crime." Their reactions to various forms of the apparatus were most remarkable. On one occasion, an individual came running over to me, asking me what the device I was wearing was for. I told him that it was a personal safety device for reducing crime - that, for example, if someone were to attack me with a gun or knife, it would record the incident and transmit video to various remote sites around the world. I found that by taking charge of the situation and throwing the same rhetoric back at them, even though photography was strictly prohibited I could overtly take pictures in their establishment, while telling them in plain wording that I was doing so. My approach, which essentially forced them to swallow either their words or their policy, left them tongue-tied, unable to apply their "photography prohibited" policy, confused, bewildered, in what I believed was a state of deep thought - at least they finally began to think about the consequences of their blind obedience.

AM: You seem to be a sort of ghost rider.

SM: I think many people don't understand the danger of centralized surveillance. Perhaps others are on the wrong side (all is relative), and need someone on the right side to confront, to tell them they're on the wrong side? All a matter of perspective.

AM: Sure. But somehow your "audiences" in the shops, these people that seem to be so used to surveillance cameras and also to cameras of the big media, seem to crash mentally when you appear with your wearcam and wearcom.

SM: It is probably, in part, due to the unfamiliarity of my approach, so in this way I have succeeded in re-situating everyday familiar objects (cameras) in a disturbing, disorienting fashion (what I call "surveillance situationist" approach). People have become so numb to traditional surveillance that they don't seem to see it anymore. What they need is something to force the issue to an edge, and confront the matter. They don't seem to mind being watched by strangers, but maybe don't realize that someone could be getting to know them through the medium of surveillance, and possibly bring harm (e.g. like the peaceful protesters are often rounded up, detained, murdered, etc., as a result of surveillance videos which were supposedly for "public safety").

AM: So you're the accident invading their personal safety?

SM: I see the accident as a means of seeing the walls of hegemony's cell. The forces of hegemony often manifest themselves as control systems to maintain social order, much like the obedience collars worn by dogs. Obedience collars are said, by their manufacturers, to produce an "electrical corrective signal" when the dog deviates from its confinement space. In addition to being a pleasant euphemism for "painful electric shock," the notion of a "corrective signal" describes quite well the dehumanizing "control systems" approach to maintaining social order. The "control theory" approach to maintaining social order might, at first, appear to maximize happiness for all, but at some point one must ask whether Disneyfication leads to Prisonification, or whether Singapore leads to Sing Sing. Is the idyllic world a prison? Perhaps the prison grows around us so slowly that we don't see it, until an accident happens and we hit the walls of our cell, like Jim Carrey of the Truman Show who sails to the edge of the earth and crashes into its wall, suddenly discovering his confinement vessel - the television studio as a prison with an illusion of freedom - a prison one cannot see until an accident happens. The accident is something the makers of the confinement vessel never envisioned. When Reflectionism is successful we smash painfully into the mirror it has held up to us (society). At first it appears as an idiot or drunk, driving on the wrong side of the road, until we realize it is a mirror image of ourselves. It is not the detached grotesque cyborg entity in its tangled mess of circuits and wiring, but, rather, it is the idyllic society we have built that says "Please wait, while I steal your time, your life, and your soul."

Steve Mann, currently professor at the University of Toronto, has, since the eighties, almost permanently been wearing a wireless computer and camera upon his body (so-called wearcom and wearcam), of which he created different, increasingly user-friendly versions. He not only uses these techniques because they are convenient in everyday life, but also to create performances that consist of recording with his own camera those who film him with their surveillance cameras; which leads to bizarre confrontations in department stores, malls and other scanscapes. Mann uses the term "Reflectionism" for these situationist actions. Through his long-lasting fusion with the computers on his body, Mann can, like no other, describe and interpret the psychological and social effects of that fusion.

© 1998 Arjen Mulder / V2_

Document Actions
Document Actions
Personal tools
Log in