Tuesday, August 24, 2010

Seeing, Hearing and Emoting in Virtual Worlds

Recent posts by John “Pathfinder” Lester and Grace McDunnough discussed how sight and hearing in a virtual world impact our perception of each other. In You Look Marvelous, Pathfinder wrote about the "uncanny valley" phenomenon that makes us feel "creeped out" when depictions of human faces are not quite right. In Coding our Faces for the Crowd, Grace discusses the power of sound and the spoken word to communicate subtleties that are lost in text-only speech.

I've been long fascinated by the interesting blend of character simulation and human extension/expression found in virtual worlds. Humans never comprehensively and directly perceive anything in the external world. Contact is always mediated through the senses. They pass a limited range of information to the mind/body which translates the patchwork of sensory data into our internal mental model of the external world.

When a human is psychologically immersed within a virtual world, his or her brain/biology and subconscious mind/psychology pretty much treat sense impressions streaming from the virtual world in the same way they translate input from the physical world. And what they do well is to "fill in the blanks", replacing missing information with content from an individual's existing mental model. Compared the the physical world, virtual worlds are what Marshall McLuhan termed a cool media, which is a form of media with relatively low resolution and incomplete data.

So on one hand, as Pathfinder wrote, we miss the nuances of facial expression and body language. But as Grace brings up, filtering body language may avoid miscommunication through culture-specific (mis)interpretation.

Just as as those who are blind often develop a richer and more nuanced sensory experience of hearing,  experienced users of virtual worlds have extended text chat to compensate for missing visual and aural cues physical. I wrote about this in Erotic Chat as an Exemplar of Sense Extension in Virtual Worlds.

In any case, it is going to be interested to see how our experience of virtual worlds shift as haptic interfaces become more common and increasingly sophisticated software allows for photo-realistic, high resolution avatars. I leave you with my cautionary motion comic "Primates in Virtual Worlds". It gives a little narrative oomph to the idea that our minds don't do well at distinguishing virtual and physical experiences and that our emotions are often triggered by sensory input:


Anonymous said...

There is plenty of truth to this video. Our minds are stimulated by what we see and experience. A movie will cause us to react emotionally to what we see on the screen, however, our emotions are greatly augmented when we are the ones actually forming a part of that story.

Some may think that what we experience in the virtual world does not affect us in real life, but they are completely wrong. Our brains are constantly being stimulated by what we see, hear, touch, smell, and sense. In a virtual world those senses are limited but yet our brains accept it as if we were experiencing this in real life. So you will feel strong emotions of love, hate, disgust, lust, fear, and even arousal. Separating these two worlds is, in my opinion, impossible.

Botgirl Questi said...

You're preaching to the choir! I've written quite a bit here over the years that people's attempts to wall-off their virtual lives are not sustainable. Even with two isolated identities, there is only one brain/body. Time and time again, despite people's very strong intentions, virtual life bleeds into physical life, for better or for worse.