The Self Referentiality of Perception: the ear does not hear, the eye does not see

Lecture by Seiko Mikami during the Machine Aesthetics Student Seminar (1997).

My recent projects have included works that used the visitor's pulse as a medium and a VR installation whose title was Molecular Informatics, which traced the movements of the user's eye and translated them into virtual molecular structures which the user could see as they evolved. These structures were created via input directed by the visitor's sight line or gaze. My latest project World, Membrane and the Dismembered Body uses the audience's heart and lung sounds which are amplified and transformed within space. These sounds create a gap between the internal and external sounds of the body. My works fragment the body into data, employing them as interfaces. I chose to organize these works in such a way as to elaborate how the structure of "interface" exists within the body itself. The set-up externalizes the body's mechanisms, hence, "The ear is not a merely a thing that hears; the eye is not merely a thing that sees."

I would like to share with you my views and later show some videos of my projects. First I would like to present the latest project, the permanent exhibition for Intercommunication Center, Tokyo. This project uses the audience's heart and lung sounds which are amplified and transformed within space. This is a soundproof room also known as an echoic room. The soundproof room is a special space where sound does not reverberate. Upon entering this room, perhaps because sound is not at all reflected, it's as if the visitor's ears are no longer living while paradoxically you also feel as though all of your nerves are concentrated in you ears. In some cases the participants' ears have turned red because of the heat generated. Ultimately, the visitor has the impression of being inside a huge ear or the feeling that you are immersed in the membrane of the ear. This leads to an overall perception of your universe being turned inside out, because the soundproof room utilizes the quality of suspendedness to artificially create a situation in which the visitor is made aware of the mediation of sound in the interaction of auditor and environment. Therefore, when you enter this room, you can not help but have an overwhelming sense of your auditory organ, or more precisely, the lack of your sense of hearings.

In the virtual realty or multi media world, we tend to use highly technical means to create 3-d worlds and images, however, most of the work that goes into developing these worlds centers around the image. The acoustic senses are largely ignored in most cases. For instance, I recently experienced a very polished VR work about Architecture. Even though this was a highly detailed 3-d world where you could walk around and into all of the separate rooms, audio was not used in any way to help the viewer orient themselves in the VR space. I think sound can and should be addressed in a much more comprehensive fashion. Unfortunately in the world of virtual reality and also the ART world itself, acoustics often take a subordinate role to the visuals. In my opinion however, the eye can only attain a high level of awareness, or focus, to a narrow fraction of the space to which its attentions are constantly being attracted. The ear, on the other hand, is able to take in information from a larger space and many signals can be transmitted via sound. In a normal environment, the visitor can orient themselves almost unconsciously by taking in the sounds of footsteps, voices, and other types of audio cues and thereby gain an understanding of the size and types of materials that make up the space he or she occupies.

However, human ears don't work according to one's will. While the eye can intercept the flow of information by closing, the ear does not have the same power. The acoustic sense extends its feelers to take account of places that the ears can't "see" and numericizes those distances. The ears of the exhibition visitor register the sounds emitted from his or her own body through the body's membranes, which have been set to vibrating by noises originating therein. Unlike the eye, the ear does not lend itself as easily to metaphorical expressions, such as "keep your eyes open"; in the case of the ear there is no sense that one "keeps open" one's ears. But in the soundproof room, the environment that surrounds the visitor provides nothing for him/her to react to or interact with.

This project makes clear that perception largely takes place behind people's backs, so to speak. This room looks like a membrane existing inside our body. The soundproof room is a special space where sound does not reverberate. Yet it is not silence but sound that exists in this space; that of the body's own internal noise. If the visitor remains in the soundproof room for a long time they are given the illusion of being dominated by the various sounds of the body, such as the heartbeat, lungs, pulse and especially the rumblings of your stomach which becomes similar to the sound of distant thunder. They soon realize "I myself produce noise." John Cage also said a similar thing after he entered a soundproof room. One could say that the heartbeat itself is the most fundamental form of self-expression. In addition, the sounds of the heart, lungs, and pulse beat are also numericized by the computer and act as parameters to form a continuously transforming 3-d polygonal mesh producing images that are projected in this room. Therein, two situations are effected in real time: the slight sounds produced by the body itself reverberate the body's internal membranes, and the transfigured resonance of that sound is amplified in the soundproof room; a time-lag exists in this process. 

Neither the body nor the environment is cast as the object of representation; rather, the "ear" that intervenes signifies a kind of inter-medium that serves as the perceptual link, or code, between the acoustic sense and the space of the room. This in between "ear" is the abstract expression of the work's claim that "the ear is not merely a thing that hears; the eye is not merely a thing that sees." When you enter the soundproof room and start to hear your heartbeat, you are suddenly thrown into a very heightened bio feedback loop. The participant is suddenly stuck with the amplification of their internal heart sounds and typically has a psychological reaction of trying to control it. This is similar to what can happen during a lie detector test when the test subject becomes overly aware of their reactions and trys to modify them and then finds that they are having even more uncontrolled reactions, and so a battle between the self and the body ensues. Thus, a fundamental gap is born between the body's response, when the heart is made to palpitate and undergo change in the soundproof room, and its result as expressed in the movementof sounds emitted from the body. A gap occurs between this event and the resulting desire; thus, the visitor is overcome by the feeling that a part of his corporeality is under erasure.

The visitor exists as abstract data, only his perceptual sense is aroused. The visitor is made conscious of the disappearance of the physical contours of his subjectivity and thereby experiences being turned into a fragmented body. The ears mediate the space that exists between the self and the body. .. My present works deal with perception as a whole, however when you talk about all of perception it is a vast and wide-ranging theme, so for my current projects I decided to break down the different senses and concentrate on them individually. Again, for example, I am experimenting with chaotic themes of reversal such as "the eyes can touch, the ear can smell, or the nose can see." The topic of the "human interface" is similar in its complexity and since, in my opinion, we won't have enough knowledge about our own body and physical functions, this also is why I concentrated on just the auditory senses in this project and another project which concentrated mainly on the sense of sight. I think the key experience in those, and other VR works, is the issue of perception and memory.

About perception: I recently went to the Philadelphia Museum of Art in the United States where I saw Marcel Duchamps final piece "Etant Donnés". What struck me at this show was my perception before and after looking at his piece. His installation was at the far end of the museum, so after I saw it I had to backtrack through all the exhibits of paintings I had already seen, and I felt like I had a very heightened sense of perception, and that those pieces became more profound for me than they were when I had viewed them just moments before. This is, obviously, the power Art can hold to alter perception and I found it very interesting when later I read that Duchamp had said, "Man can see what man sees, but Man can not hear what he hears." I am impressed with that work, but not the physical object itself which he created, but by how his work can expand our perception of the eye. I think he also expanded the art world's view of other painters and artists work. I had a similar experience while working in and around the soundproof room. Every time I left that environment I would experience different very detailed sounds that I had not perceived before.

On "memory": I once experienced a VR work inside a large simulator. This "world" was navigated via a mouse and the whole simulator would also move as a result of my direction. As I was navigating through this space, the system crashed and I was suddenly hit with a confusing conflict between the real world and the virtual world. Even though I knew I could easily exit this space/program, for a moment my senses took over and reacted to the sudden crash. This was due to the lag between my senses sending a signal of danger and my memory, which knew that I was actually safe. I think media/digital art programs need more of this unpredictability. I feel like this would reflect the real world more accurately. The real world contains all sorts of potential negativity. In other words, I would like to see programs where death and accidents can play a role. This could especially be interesting as it concerns AI work. Virtuality and reality both exist within our memories. Digital technology has existed for a long time inside our own body/minds mechanisms, from the cellular level up to the higher brain functions. Also, our perceptions mediate the self and the body. There is always a split between the "thing/object" and the "viewer". It is this space that I am interested in in my work. I am not interested in the physical objects that might arise from my work, but with making material this invisible space that arises between the body and the object. This is known as the inter-communication or mediation that occurs between things. In his "epistemology", Lacan says, "this is the split between the eye and the gaze." He says, "The eye and gaze this is for us the split in which the drive is manifest at the level of the scopic field. (Lacan,73) .

The next thing, I would like to talk about is the eye-tracking work I presented at ARTLAB TOKYO in 1996. This project was called Molecular Informatics--morphogenic substance via eye tracking, It was designed for one person, and attempted to show the gap between one's conscious and unconscious process. After that, here at DEAF 96, I presented the work with a new version designed for two people. This piece concerned itself with the territory set up between the two participants. Each participant wears VR glasses equipped with an eye tracking sensor. The glasses are first calibrated to match different eye sizes and pupils. The virtual world can be navigated simply via the users eye movements. There are no icons on the screen. When the viewer's gaze stays in the center of the screen the view slowly scrolls through the space, towards the horizon. When the viewer looks to the left or right for several seconds the view is scrolled in that direction. The focus point of the viewers eyes is set about two meters in front of them in the virtual space. Molecules are automatically generated according to the traces of one's eye movements. When the viewer looks back, they can see the traces of their views. The audience in the exhibition can watch a projection that shows what the participant is viewing through the eye tracking glasses. At the same time at the back of the exhibition space another projection system acts as a telepresence recorder of what the participant is seeing. This projector is on a motorized base that tilts and pans with the participant's eye movements, creating not just a visual record of the viewers movements, but also a physical replication of these movements. 

I am exploring the world as it is generated and modified through the exchange of invisible information, under the concept of "bio informatics," which traverses the genres of biology or technology. This exhibition is based on the theme of "the interface of mathematics (factor) and perception (morphology). It is not the type of art which deals with the aesthetic values of visible objects, but art which is made possible only in an information exchange relationship. The content arises from the interaction with the program (factor), and the resulting change of the visible morphosis. Molecular Informatics focuses on the transformability and morphoses of molecular structure and movement. The viewer experiences a virtual world created and manipulated by their own eye movements. This movement, the human, physical part, is transformed into data that represents the viewers location within the 3-dimensional space, therefore real-time change and generation occurs within the virtual world. Once the viewer dons the VR eye tracking glasses, they are thrown into a world constructed solely of molecules. They navigate through this world simply via their eye movements, and new molecules are created or mutually influenced in real-time due to their eye movements. The participant is represented as abstract data in the virtual space, and newly generated molecules reflect how they view and react to the virtual world. Humans do not consciously control most of their eye movements and in this project these unconscious movements play a major role in the creation of the piece. These uncontrolled movements are transformed into form and structure in the virtual world, and the molecules generated by the previous viewer are left behind for the next participant. These multiple traces of eye movement co-exist and create a multi-layered space. 

Michel Foucault wrote in his book, The Birth of the Clinic - an archaeology of medical perception (Naissance de la Clinique): "One hears certain words at the moment he perceives a scenery." The act of seeing is a passive and obscure activity involving complicated physical functions." In other words, the world that is seen reflects the viewers physical complexity and multiplicity. The creation of Molecules by random eye movement, the Molecular Informatics, ceaselessly changes the virtual world and creates a world that changes dramatically from moment to moment. It is both an infinite succession of images and an endless sequence of senses. I said before "the eye does not see, the ear does not hear," therefore using eye-tracking, the eyes mediate the space that exists between the self and the body.

Document Actions
Document Actions
Personal tools
Log in