Question 1: Tell us about yourself. What is your background and training, and what projects are you currently working on?

I was formally trained in Cognitive Science, as an undergraduate at the University of Michigan, and as a graduate student at Northwestern University. My dissertation was a mathematical model of human reasoning and argument. In 1999, I decided to drastically change my research focus: I accepted a post-doctoral fellow position at the University of California, Santa Barbara, in the Research Center for Virtual Environments and Behavior. There I learned to build (from a computer science standpoint) and study (from a social psychological standpoint) virtual human avatars and embodied agents. I have now created the Virtual Human Interaction Lab (http://vhil.stanford.edu) at Stanford University. Our work is dedicated towards building immersive collaborative virtual environments and studying interaction among human avatars and embodied agents.

Question 2: How do you define "virtual reality"? Is your definition of virtual reality shared by your colleagues and the research community?

My favorite definition of Virtual Reality is by William Gibson: "A consensual hallucination". This definition is especially appealing because it operates independently of any specific technology. Jaron Lanier, who coined the term Virtual Reality, offers a similar definition: "a new post-symbolic paradigm which circumvents representation with a direct experience".

Question 3: When most people think of virtual reality, they envision a user with goggles and data gloves. Is that image now archaic?

Unfortunately not. The goggles are still heavy and expensive.

But our work at the VHI-lab is not to engineer lighter gear. Instead we study social interaction, specifically the nuances among digital representations. The new issues that arise in communication and interaction via virtual reality transcend any specific technological implementation. The ability for humans to transform identity and behaviors in real-time (i.e., Transformed Social Interaction, described below) during digital communication is inherent to any type of digital virtual reality system.

Question 4: The best visual supercomputers can render 5 billion polygons per second. How close is this to reality? How many polygons-per-second would be needed to accurately simulate reality in a photorealistic manner?

Our work has demonstrated that you can elicit huge amounts of presence (feeling like one is in VR, not in the physical world) and copresence (feeling as if an avatar is really another human being) even if the world is rendered simply using low numbers of polygons. What is crucial is to implement to create presence are realistic social behaviors.

Question 5. Tell us more about Transformed social interaction.

TSI is a research paradigm that examines the disjoint between human characteristics and behavior that exist in physical space with the characteristics and behaviors that are rendered to others in virtual reality. Because behaviors are tracked and rendered in VR, as opposed to directly transmitted via an analogue-type of information stream, interactants have the ability to filter, augment, or block their own behaviors from the eyes and ears of the conversational partners. Previous research on TSI has examined the effects of being able to look directly in the eye of more than one other person at once, at mimicking the nonverbal behaviors of other interactants, and at morphing the faces of interactants to absorb facial features of conversational opponents. All of those studies have demonstrated that a) TSI is extremely difficult to detect, and b) TSI makes speakers more persuasive, more well-liked, and better teachers.

Question 6: The Sims2 PC game has become a hugely successful game, and some claim that software like this will discourage face-to-face social interaction. Will virtual socializing largely replace physical socializing in the 21st century?

This remains an extremely far-fetched vision. In my opinion, it will be a long time before digital simulations replace face-to-face interaction. Humans enjoy real physical contact.

Question 7: It would seem that virtual reality could become an excellent training tool. Have you done any research into the use of VR as a training aid?

We have conducted a number of studies about using VR as a tool for training. Using Transformed Social Interaction, a teacher would be able to 1) look at 30 students eye-to-eye all at the same time, 2) subtly engage each student via differentially tailored avatars that simultaneously mimic nonverbal behaviors from each one, and 3) utilize automatic algorithms that detect student confusion, boredom, and inattention.

Question 8: Although enormous strides have been made in the field of > computer graphics during the past several decades, people still find computer-generated faces to be somewhat unrealistic. Can computer-generated faces project the necessary nuances and expressions to be effective in virtual reality?

Faces are no longer unrealistic. In collaboration with the Federal Judicial Center, we are using avatars as a way to replace the typical photographs used in police lineups. Current police lineup procedures rely predominantly on mug photographs when conducting lineups. We believe that conducting lineups using three-dimensional, digital busts of heads and bodies offers a number of advantages: witnesses can view the suspects and distracters from any angle (not just front and profile), witnesses can move as close or as far from the suspects as they desire, test administrators can use mathematical models to distance the distracters from the actual suspects by simply morphing the underlying mesh model of the suspect, the lineup can occur within the virtual crime scene (i.e. the suspects and foils can lineup in the context of the actual crime, and finally, the virtual suspects and distracters can be animated, not just static heads. Our previous research has proven that 3d heads and faces created using photogrammetric software capture over 95 percent of facial structure.

Question 9: Is Virtual Reality more limited by hardware constraints or software limitations?

Probably hardware, although there are not enough graphics software packages specifically designed for real-time animation.

Question 10: Have you done any research on telepresence? Could a combination of robots and telepresence obviate the need for a human presence in space?

I have conducted many studies on telepresence, specifically on how one measures telepresence. Our work demonstrates that the best way to measure telepresence is through behavioral measures: do people act consistently with the virtual world (e.g., they refuse to walk over a virtual cliff) or the physical world (e.g., they remember there is a floor beneath them in the physical world). If the former occurs, then there is high telepresence.

In regards to space, I think humans like going into space, so, while we will have robot partners, I doubt we will be replaced.

Question 11: Do you believe that a thriving virtual reality industry will emerge within the next decade? If not, then when (if ever) can we expect a robust VR industry to emerge?

Some form of interaction among digital human representations will become extremely prevalent, whether it is immersive virtual reality or other types of forums. Due to the extreme popularity of chatrooms, online games, and the use of the internet, the escalation to more sensory rich simulations is highly probable.

Question 12: What are your professional plans for the next decade?

Over the next ten years, the mission of the Virtual Human Interaction Lab is to understand the dynamics and implications of interactions among people in immersive virtual reality simulations (VR). Researchers in the lab are most concerned with understanding the social interaction that occurs within the confines of VR, and the majority of our work is centered on using empirical, behavioral science methodologies to explore people as they interact in these digital worlds. However, oftentimes it is necessary to develop new gesture tracking systems, three-dimensional modeling techniques, or agent-behavior algorithms in order to answer these basic social questions. Consequently, we also engage in research geared towards developing new ways to produce these VR simulations.

Our research programs tend to fall under one of three larger questions:

1) What new social issues arise from the use of immersive VR communication systems?

2) How can VR be used as a basic research tool to study the nuances of face-to-face interaction?

3) How can VR be applied to improve everyday life, such as legal practices, and communications systems.