Photogrammetry: A Study of Image-Based Modeling

If you have the Quicktime plugin [1], you can view clips of photorealistic models by simply clicking the image.

Although advancements in recent modeling techniques have advanced by leaps and bounds, the process of creating photorealistic models remains a somewhat tedious and time-consuming task depending on the circumstance and whom you ask. Plus it may not always be beneficial to spend extensive time creating complex models such as humans, cars or any other real-world subjects in a day-to-day production pipeline. With the advent of image-based modeling (IBM), the technique has caused quite a stir in the industry and rightfully so. With the ability of artists to explore objects and scenes captured from real world photography, it has become extremely attractive, but it does carry its share of limitations, which we will explore later in this article.

Although image-based techniques cover a wide range of topics, including animation, inverse lighting, light field rendering, layered depth images, to name a few, I will cover the process of creating 3D models from normal everyday photography. I will attempt to help those who don’t understand exactly what IBM is, and explore its pros and cons.

[2]

Tour [2] this 3D environment created with ImageModeler. © www.animations-and-more.com

Understanding Image-Based Modeling
IBM is the process of having the user take photographs from different points of view and then using these photographs as a "starting point" in the process of using point-to-point measuring tools in an image-based modeling application. By using a typical consumer digital camera to take these photographs, it makes this technique accessible to a very wide range of users. Although IBM is relatively new, researchers such as the renowned Paul Debevec and artist Greg Downing have made very impressive discoveries utilizing image-based techniques. Furthermore, more advanced image-based techniques have already made their way into production environments. Let's take a look at how this technique can assist your projects.

Some Uses of Image-Based Modeling
There's already quite a growing list of uses for IBM in production environments; some of the more evident roles it can play are the exact recreation. For example, architecture or archaeology sites where you can evidently immerse your viewers into a virtual 3D world with pinpoint accuracy, or the matching of camera perspectives for overlaying 3D over 2D elements for your compositing needs. In addition, some video game developers have adopted image-based solutions utilizing these techniques for fast creation of photorealistic backdrops or humanoid models.

The Software That Makes This Happen
Although IBM software has been around for quite a few years, the technology only recently matured enough to be considered production ready. This resulted in a number of prominent applications, which take a lot of the guesswork out of creating detailed models. The user essentially loads a photograph they wish to model, assign common reference points in the given software and the software analyzes the photos and creates the model based on that image.

There are several capable applications available to date that focus on IBM -- let’s take a look.

REALVIZ ImageModeler 3.5
Realviz has created a powerful IBM application that is now widely known amongst artists, photographers and architects alike. REALVIZ ImageModeler 3.5 has created a solution that measures your photographs from a few points of view and creates the 3D scenes based on those using polygonal methods. It uses special advanced algorithms that extract the 3D information using point-to-point measurements and at the same time it automatically takes the texturing info from photographs, which drastically reduces time. www.realviz.com

PhotoModeler
PhotoModeler is another versatile application that basically uses the same approach as ImageModeler; the software uses NURBS, among other modeling tools, for object creation and does in fact support a variety of output file formats for usage in other mainstream 3D or other software. By using PhotoModeler's Point, Line and Edge tools, you are able to mark features on imported photographs, which will then be used to generate the physical 3D models. www.photomodeler.com [5]

Façade
Paul Debevec has conducted extensive research in image-based techniques and has used this knowledge to create an unreleased research prototype IBM dubbed Façade. Although it’s still in the prototype stages, its technology and functionality have been used as the source of inspiration for many products, including MetaCreation's product Canoma and RealViz's ImageModeler 2.0. As an additional note, Façade was used to model the interior of St. Peter's Basilica for Debevec’s film Fiat Lux in the SIGGRAPH 99 Electronic Theater.

[6]

Artist Greg Downing used ImageModeler 3.5 and Stitcher to recreate a 3D model of the Tribunal Plaza in Nice, France that a viewer can "walk" [6] around in. © Greg Downing [7]

In addition to the above-mentioned software, an artist by the name of Greg Downing has developed a technique by creating image-based 3D models from panoramic images. Utilizing REALVIZ ImageModeler 3.5 and Stitcher (a panoramic creation software) to “stitch” a series of panoramic images from inside the Tribunal Plaza in Nice, France, Downing re-created a very accurate 3D model of the Plaza. This was then taken a step further to be used to "walk" around in, as if in a game.

There maybe more software not listed here capable of producing similar results, but essentially they all use a similar process to achieve the final results. So as you can see, these applications do have the ability to cut down significantly on the time it takes in production to create models, unwrap and texture them, but is it all as good as it sounds?

So What are the Pros and Cons?
While it may appear to some of you that these techniques and applications are just what you’re looking for to add to your production pipeline, before you make your purchase let’s dig a little deeper than the surface. While this technology can work with basically any photograph you can throw at it, not all of the aforementioned products do exactly what you may you expect them to do. As an example, the software PhotoModeler may work divinely for your architectural projects, but it’s not as efficient for modeling organic objects such as people. The reason for this is because PhotoModeler and a majority of the other applications mentioned require users to manually assign reference points for areas that only can be seen from your images. Each of these reference points essentially forms a vertex in the final model, thus creating the object, but because of the complexity of organic subjects that require quite a few more vertices or faces, etc., the amount of reference points you would have to place manually to create a quality model would be astronomical, let alone very tedious and time consuming work. Also, it can be quite difficult to accurately place precise reference points on photographs that lack sharp edges to use as reference.

On another note, using IBM solutions have been widely successful because we can now create 3D imagery based on a few photographs taken from your ordinary consumer digital camera or camcorder. But although there have been many advancements in this area, there’s still the lack of usability while modifying or manipulating these objects once rendered.

One of the major advantages of IBM is the ability to create arbitrary geometry, but as with anything else, you will have to practice taking the correct angles of your subject/s to ensue proper calibration within your respective IBM software. The reason for this is to lessen the distortion that may occur from improperly taken photographs.

The Future of IBM
Image-based techniques have been used for the facial animation and modeling in The Matrix Reloaded, as well as numerous elements in Fight Club, such as the "Mid-air Collision" and the "Gun Shot," which also mixed principal photography. So, as you can see, industry experts are finding essential uses for these techniques.

In fact, as we progress, I believe IBM will become a normal practice in more feature film and television work, and will eventually find its niche as the technology continues to advance, which may be dependant on your artistic skill set or production requirements. But no matter how much it advances, there will always be a need for seasoned artists, who will continue to play a vital role.

Tito A. Belgrave is currently a senior 3D artist for DKP Effects in Toronto and freelance writer, who recently completed work on the first-ever network primetime CG series, Game Over, which debuted this month at 8:00 pm on UPN. When not involved in a tight day-to-day schedule, he spends time with his lovely fiancée, and works on his short film, The Tale of Kar [10].