Photo-Realism: An Exact Science?

Featured image from Jeremy Birn’s CG Society Lighting Challenge #4, June 2006.

This article was originally published on 2 Sep. 2002 in Illumin, the University of Southern California School of Engineering’s Engineering Writing Program online magazine. It has been reproduced here for preservation.

---

Occasionally, it is the job of special effects artists in film and television to manipulate reality in order to present an image that audiences believe they would see, as opposed to what they would actually see in real life. This concept and practice incorporates not only the mathematics and engineering of special and visual effects, but the psychology of what people expect to be reality as well as the art of filmmaking. Together, these fields create images that make a story more believable, even though it is now actually further from reality than before. However, this procedure leads us to question whether it is right to continue to propagate incorrect expectations in audiences’ minds, as opposed to the reality of what actually exists. Which is correct: the accurate presentation of reality, or what the human mind expects to be reality? In due course, the two may be resolved in a fashion appropriate to the nature of visual effects for entertainment.

Introduction

In the movie world, special effects artists are often given the same challenge over and over again–the re-creation of some aspect in the real world that, for reasons ranging from financial to safety concerns, cannot be filmed or recorded in a normal fashion. The artists must then incorporate their work seamlessly into the rest of the film so that no distinction can be made between the two pieces.

During the past few years, however, this has become increasingly difficult because many effects are now being done on a computer in 3D. Though this gives the effects team more control over the shot, it simultaneously introduces the problem of making the 3D elements appear real. For some objects appearing in the physical world, such as static, hard surfaces like brick or stone, this can be done easily–it is much more difficult to lend a photo-realistic look to something organic or dynamic, such as fur or water. That is where much of today’s 3D graphics research lies: making an element like water appear to move, interact with other elements, and reflect and refract light like real water. It is a mix of science and engineering along with artistic creativity and vision. The ultimate goal is the creation of an on-screen product that is indistinguishable from the other elements of the movie.

Can Everything Be Calculated?

Every major special effects house in the world has a Research and Development (R&D) team that supplies the rest of the company with new ways to make things photo-real. The engineers and mathematicians that work in R&D are constantly using mathematical equations to understand the actions of elements; this offers a more realistic imitation of life. It is similar, in a sense, to the final theory of everything, which says that one day we will be able to determine how everything in the universe interacts.

To imitate water, for instance, R&D engineers must determine how it will flow around objects by using fluid dynamics, how it will splash and scatter by calculating surface tension, and how objects will look when submerged by determining angles of reflection and refraction. They must then write software that calculates all of this over time and allows the virtual water to interact with other elements in the scene [1]. The result can sometimes be very convincing, such as in the film The Perfect Storm, which was nominated for a Best Visual Effects Oscar in 2000. Even with extensive research, though, the resulting outcome will not fool everyone because many elements, such as water, are things that we are all exposed to on a daily basis.

Reality vs. Perception

A 3D image that looks and moves in mathematical accordance with reality can still look wrong to a viewer on a television screen or in a movie theater. Audiences expect things to look a certain way, even though that may not be the way in which it is seen in real life. Thus, watching a physics-based simulation often conflicts with the viewer’s expectations, to the point where the viewer will not believe that the simulation is realistic. This presents a different problem: do we continue research in hopes of achieving greater accuracy in the representation of reality, or do we change the equation or algorithm so the viewer sees it as a more correct representation, even though, in reality, it may not be.

For instance, in the recent film Harry Potter and the Sorcerer’s Stone, the effects animators portrayed the infamous Quidditch match, involving students flying on broomsticks around a stadium at high speeds. In order to achieve the proper look, the animators simulated the characters flying at speeds sometimes over 300 miles per hour. The key problem resided in the fact that the simulated clothing of the characters did not work at those speeds. Eric Armstrong, Sony Imageworks animation supervisor, explained, “If we exceeded 300 miles per hour, it would literally rip their clothes off and leave them behind at the other end of the field!” [2]. Visual Effects Supervisor Rob Legato also described that even at slightly slower speeds, the cloth didn’t seem to be moving correctly to audiences [4]. In order to solve the problem, the animators simulated the cloth at only 60 miles per hour instead of the 300 that was used for the rest of the character [3]. Animators then combined the two together to give the audience the proper feel that the clothes were flapping in the wind. Even though physics dictates cloth movement, this was not in agreement with viewers’ perceptions of reality; so instead, the animators changed reality.

Ethically Correct?

This area of graphics involves mathematics and engineering to create more photo-realistic portrayals, as well as art and psychology to cater to the human perspective. When developing scenes for Apollo 13, Legato and his team would first interview people to see what they remembered and what they thought the rocket launches looked like; in the end, they based the simulation on this information rather than raw NASA footage [4].

Is such reinforcement of incorrect views and tinkering with people’s perceptions of reality ethically correct? Is the work of these artists for pure entertainment value, or do audiences leave with incorrect beliefs of how things should appear in the real world? Does it really matter if people know how cloth should move at 300 miles per hour, or is it just for a fantasy world? How should research continue–in the pursuit of absolute physics-based simulation and photo-realism, or in giving viewers images that enhance their imagination and assist in the telling of a story? Perhaps it would be wise to continue in both areas, and in the future, the two will merge. The capability of simulation in accordance with formulas may be tempered with an artistic perspective. After all, the most appropriate image gives the best impression rather than the most lifelike depiction.

Current Research

The results of such research would greatly affect the future of graphics and its use in the world. Many major advances in technology and upcoming concepts are based on having accurate, photo-realistic graphics. At the University of Southern California, the Integrated Media Systems Center is researching a future form of communication, using avatars to represent the parties involved.

However, people will not utilize such technology unless they can become comfortable with seeing a virtual, yet photo-realistic representation of another person. If anything is not accurate, however, or does seem to be accurate, even though it very well might be, participants will not want to continue communicating, because they will feel that something is not “right.”

Despite this, is there a way to develop avatars that may not be completely photo-realistic and anatomically correct but still convey the information from one person to another? Chris Crawford, an expert in computer design and interactive storytelling says, “We should relax our obsession with optical reality and concentrate more attention on perceptual reality, at least in images concerned in any way with human emotions”. He also believes that the focus should not be on what most appears to be real, but what most is perceived to be real. He comments that in most computer graphics, “The images are excellent representations of optical reality, but they just don’t cut the mustard in terms of perceptual reality” [5]. This further supports the fact that one must study how people are seeing those simulations in addition to the development of better algorithms for simulation.

Conclusion

It is essential to continue creating representations that are indistinguishable from reality, both in appearance and in movement, keeping the human’s psychological interpretation of reality foremost in our endeavors. The final product will then fit right in, and no one will notice the difference.

References

Related Images: