Search form

The Power of 3D in Biomedical Visualization

Mary Ann Skweres examines the connections between the visual effects industry and medical visualization.

Advances in the medical and entertainment vfx fields often contribute to future breakthroughs that benefit both areas. Above is 3D rendering of a torso by BioDigital. All BioDigital images © BioDigital System Llc.

What does the visual effects industry have to do with medical visualization?

Chris Ruffo, Alias industry marketing manager for digital publishing (including the area of medical visualization), who works with animators in both arenas, explains: A lot of animators in the biomedical space realize that the same tool sets that they are using in the entertainment arena are very useful for medical visualization and telling stories in that space as well. For instance, in the movie Hollow Man, the character decomposes through various physical states from flesh through muscle through bone. The animation was all done with Maya, Alias 3D animation and vfx package that has been used on most major vfx pictures. Ruffo continues, The artists at Sony Pictures Imageworks were interested in making it look as accurate and realistic as possible. In a sense, it is almost like a medical visualization if you look at the footage. It turns out that the modeling, animation and effects tools in Maya are really appropriate for medical visualization.

According to Christopher Ruffo of Alias, it turns out that the modelling, animation and effects tools in Maya are appropriate for both the medical and entertainment visualization arenas. © Alias Systems Corp.

In fact, a lot of artists working in the medical industry are also trained animators. Although the primary focus of Alias is creating 3D animation and visual effects software for the film, video and game industries, members of the medical animation community offer feedback on software. They are involved in Beta forums, testing software in pre-release cycles, submitting bugs and offering suggestions. Some features have made it into the software such as polygon modeling tools and enhancements to the paint effects technology. Advances in one field often contribute to future breakthroughs in the other.

One of the flagship projects, contributing to, promoting and enabling advances in 3D biomedical visualization has been the Visible Human Project a government program funded by the National Institutes of Health (NIH). Established in 1989, the Visible Human Project provides detailed, three-dimensional representations of normal male and female human bodies. The project involves the use of images acquired by transverse CT, MRI and cryosection a donated frozen cadaver sliced into 1mm sections that were scanned in computers to build a digital image library of volumetric data representing complete, normal adult male and female anatomy. These data sets, available at the National Library of Medicine (NLM), are designed to serve as a reference for the study of human anatomy, for testing medical imaging algorithms and for the construction of network accessible image libraries. Applications include a variety of educational, diagnostic, treatment planning, virtual reality, artistic, mathematical and industrial uses.

Production quality is up in medical animation. Above is a visualization of the red and white blood cells in circulation from an award-winning animation called The Metabolism of Methotrexate. All BioLucid images © 2004 BioLucid Prods.

Several applications have been developed at or under the direction of the NLM, including: Insight Toolkit (ITK) an open source public software system developed to support image analysis research in segmentation, classification, and rigid deformable registration techniques to process high dimensional medical data. Functional Atlas of the Visible Human, developed by the University of Colorado, is a website consisting of six functional anatomy-teaching modules enhanced by additional clinical and surgical information and intended to serve as a model for educational applications based on the Visible Human Project data sets.

As scanning technologies such as MRI, CT and Ultrasound have improved, there has been an increasing use of three-dimensional images for clinical medicine and biomedical research. Unlike X-Rays that can cause damage to a living body, scientist Mira Ramen (daughter of VFXWorld Digital Eye columnist Peter Plantec) explains that MRI uses radio frequencies and a large magnet to create 2D scans without cell damage. Ultrasound also has interesting features, according to Ramen. It shows realtime functions. You can see the blood flowing. It is a valuable, inexpensive medical tool. Superior scanning technologies have made pre-operative imaging a tool for surgeons to plan the location of lesions before a patient is opened up.

BioDigital Systems co-founders Aaron Oliker (left), John Qualter (right) and Frank Sculli use real medical data to create 3D medical visualizations, animations and simulations.

Although government and educational institutions have been developing 3D medical projects for some 25 years, the area of 3D medical software has only been around commercially for the past five years. Add improved animation tools and artistry fuelled by the popularity of visual effects in the entertainment industry, and the result is rapidly evolving software that strives for photo-real, three-dimensional images that are useful and easily adapted to the biomedical field.

The Visible Human Project opened the flood gates, states Aaron Oliker of BioDigital Systems, when describing his business and the industry. Were still in the process of educating the pharmaceutical and medical communities about the power of 3D in their work. It also takes time for a company in an emerging industry like 3D medical animation to gain credibility. Oliker and BioDigital Systems co-founders John Qualter and Frank Sculli are software engineers, in-the-trench animators and leaders in the emerging field. Bio Digital Systems uses real medical data to create 3D medical visualizations, animations and simulations. Their software of choice is Alias Maya, rewritten to better function with their specific applications. In a short time the team has gained a solid reputation through projects with Novartis, Cornell Universitys Medical Center, New York University School of Medicine and St. Lukes Hospital.

In conjunction with NYU Department of Cardio-Thoracic Surgery, BioDigital Systems created the worlds most advanced, scientifically accurate 3D-animated beating heart. Scientific data from the Visible Human Project was used to build an anatomically correct three-dimensional, but static heart. Then, using actual additional scientific data obtained from the radioactively sutured heart of a live sheep, mapped over time a process somewhat like motion tracking BioDigital animated the heart using a program that Oliker wrote to reconstruct the natural movement of a beating heart. This heart could then be manipulated by other data to react to a heart attack using animation to morph the healthy heart into a damaged heart. Using MRI body scans, BioDigital also partnered with St. Lukes Hospital to reconstruct full, contiguous, extremely detailed body compositions.

3D bio medical visualizations that contain all real data are especially effective learning tools and are used extensively by various medical schools and organizations.

BioLucid animators work from highly detailed storyboards in the concept phase of a project before hitting the computer, where they build models, add textures and create animatics. Above progenetor cells become white and red blood cells.

The Smile Train is a charitable organization that provides free cleft surgery and related treatment for children who would otherwise never receive it a simple cure that has been around for decades that takes as little as 45 minutes, for a cost as little as $250. Unlike organizations that send visiting medical teams, The Smile Train empowers local doctors to provide safe, high quality care to the children in their own communities for a fraction of the cost of medical missions.

A major tool in achieving The Smile Trains mission are Virtual Surgery CD-ROM Training Videos the first surgical educational tool to use virtual technology and advanced 3D animation software. The most complex surgical techniques are presented as easy to understand animations, enabling doctors in the poorest, far-flung corners of the world to learn advanced cleft surgical techniques from the foremost experts in the field. And the software used to create the videos is being expanded to compliment new developing technologies that are being used to bring the future closer.

Using the latest animation and three-dimensional techniques, Court Cutting, M.D., director of The Smile Train Virtual Surgery Laboratory, cleft repair experts from New York University and leading medical animators are collaborating to create a realtime virtual surgery environment that can be controlled interactively by surgeons to learn, practice, and perfect techniques in cleft lip & palate repair, while measuring proficiency and accelerating the time it takes to become skilled at these procedures.

BioLucid Prods. is a full-service medical animation studio that creates mechanism of action animations for the biotech and pharmaceutical industries. According to BioLucid founder, Jeff Hazelton, animations illustrate far more dramatically than dry text in a medical journal, How a product fights disease on a microscopic or molecular level. Hazelton uses eye-catching 3D-animated movies to demonstrate new treatments for life-threatening illnesses. BioLucid clients, such as Pfizer, Amgen, Edwards Life Sciences and other pharmaceutical and biotech companies, use their video and still presentations for trade shows, websites, sales force training tools and informational presentations for physicians and consumers.

BioLucid founder Jeff Hazelton finds animations illustrate far more dramatically than dry text in a medical journal. His company uses eye-catching 3D-animated movies to demonstrate new treatments for life-threatening illnesses.

Hazelton has a degree in biology and a passion for computer graphics and animation. For the animations to be scientifically accurate, BioLucid employs PhD level medical writers. Animators come from come from schools, Hollywood and internships. Hazelton likes young animators two to three years out of school, Were better off training animators because of the scientific side. We are the translators of this language. The team follows the production pipeline of a typical animation studio, including a high level of quality and detail. To produce convincing medical animations, they work from highly detailed storyboards in the concept phase of a project before hitting the computer where they build models, add textures and create animatics. All animations are 100% CGI.

Production quality is up in medical animation, with productions composing music and adding sound effects. Last year BioLucid got into stereoscopic 3D, a purer form of 3D that splits up images in a way that looks better. BioLucid has used Maya to animate since its introduction in 1998. An abundance of plug-ins, tools like fluid effects, particle effects and organic modeling, ensure animation realism. Hazelton just started using Real Flow, a fluids plug-in, to create the realistic movement of fluids. After Effects is used for compositing. The ability to quickly change and correct already rendered animations to customer specifications make it possible for the animation staff to finish the work accurately and in a timely fashion.

BioDigital Systems has created the worlds most advanced, scientifically accurate 3D-animated beating heart. Oliker wrote the program to reconstruct the natural movement of a beating heart that could then be manipulated by other data.

The U.S. is not the only country involved in this emerging technology. Three-dimensional medical visualization using animation tools is also being developed in Europe. Hanna Reuterborg and David Ortoft, medical students at Karolinska Institutet in Stockhom, Sweden, initiated a medical visualization that resulted in the 3D Embryo. The project was first used during a course in developmental biology for medical students in the fall of 2001 and is currently being used by MD, Biomedicine and the Biomedical laboratory analyst programs at the Institutet. The visualization uses animation to follow the embryos day-by-day development. The 3D Embryo is a tool that simplifies learning and teaching early embryology, which is intended to be integrated into lectures and serve as a complement to literature.

As scientific advances in medical imaging improve in speed and resolution and animation programs offer increasingly adaptable and realistic tools, the applications for 3D medical visualizations will expand. If in the first five years of this emerging technology is any indication, its not too far fletched to imagine the inner frontiers we will soon be able to explore in this fantastic voyage of medical discovery.

Mary Ann Skweres is a filmmaker and freelance writer. She has worked extensively in feature film and documentary post-production with credits as a picture editor and visual effects assistant. She is a member of the Motion Picture Editors Guild.