Recent posts by John “Pathfinder” Lester and Grace McDunnough discussed how sight and hearing in a virtual world impact our perception of each other. In You Look Marvelous, Pathfinder wrote about the "uncanny valley" phenomenon that makes us feel "creeped out" when depictions of human faces are not quite right. In Coding our Faces for the Crowd, Grace discusses the power of sound and the spoken word to communicate subtleties that are lost in text-only speech.
I've been long fascinated by the interesting blend of character simulation and human extension/expression found in virtual worlds. Humans never comprehensively and directly perceive anything in the external world. Contact is always mediated through the senses which detect a limited range of information and then through the brain and body which translates the patchwork of sensory data into a mental model that we perceive as a seamless whole.
When a human is psychologically immersed within a virtual world, his or her brain/biology and subconscious mind/psychology pretty much treat sense impressions streaming from the virtual world in the same way they translate input from the physical world. And what they do well is "fill in the blanks", replacing missing information with content from an individual's existing mental model. Compared the the physical world, virtual worlds are what Marshall McLuhan termed a cool media, which is a form of media with relatively low resolution and incomplete data.
So on one hand, as Pathfinder wrote, we miss the nuances of facial expression and body language. But as Grace brings up, filtering body language may avoid miscommunication through culture-specific (mis)interpretation.
Just as as those who are blind often develop a richer and more nuanced sensory experience of hearing, experienced users of virtual worlds have extended text chat to compensate for missing visual and aural cues. I wrote about this in Erotic Chat as an Exemplar of Sense Extension in Virtual Worlds.
In any case, it is going to be interested to see how our experience of virtual worlds shift as haptic interfaces become more common and increasingly sophisticated software allows for photo-realistic, high resolution avatars. I leave you with my cautionary motion comic "Primates in Virtual Worlds". It gives a little narrative oomph to the idea that our minds don't do well at distinguishing virtual and physical experiences and that our emotions are often triggered by sensory input:
4 comments:
Yes, facial expressions are so important. I forget who said this (i'll try to look it up when i'm not so busy), but something like 90% of human communications is none verbal. I've been interested in this subject since the early days of MIT'S COF project and specifically “Kismet” by Cynthia Breazeal and her team. I think there misunderstandings in SL that begin with inappropriate facial expressions and physical recognition factorst too. I'd love to hear more and more about this subject.
sowwy, COG project
"Her brain doesn't know the difference." You summed it up perfectly. Nice educational comic!
On the topic of "miscommunication through culture-specific (mis)interpretation," I think that's a fascinating area for exploration.
The meanings behind facial expressions and body language can vary a great deal between different cultures. It's a huge challenge for cross-cultural communication.
If we're really clever in creating systems that can detect and codify facial and body language, I can imagine the eventual development of "social signal translators." Changing (in realtime) how our RL facial expressions and body language are displayed to other folks so they use the other person's cultural norms.
Standing in front of my video wall in my living room, I wave in a friendly way to the avatar of a Japanese businessman. But instead of my avatar being cued to reflect my wave, my avatar bows at the precisely correct angle. Translating my intent in a way he understands, perfectly.
-Pathfinder
Yordie: For sure! Text chat misses both body language and voice tone and pacing. And voice chat in virtual worlds not only misses facial expression and body language (as when speaking on a phone) but also depicts an avatar who may be either providing body language that conflicts to some extend with that of the speaking human, or at best be neutral.
Here's a link related to channels of communication and meaning.
Cunning Pathfinder: That would be very cool. We need some sort ontology so as to construct a Services Oriented Architecture of virtually represented body language :)
Post a Comment