SCAD Embraces the ‘Bleeding Edge’ of Technology with New LED Volume Stage

At 40 feet wide, 20 feet deep, 18 feet high, and comprised of almost 600 LED panels, the new XR stage, or LED volume, at the Savannah College of Art and Design (SCAD) Atlanta certainly commands a room. And, according to SCAD professors, it’s also commanding the future of entertainment production. 

“This technology is so incredibly important because this is where the industry is going,” says Quinlan Orear, Associate Chair of Film & Television at SCAD Atlanta, who gave a presentation on the LED volume in the school’s Digital Media Center at SCAD AnimationFest with Jud Estes, Associate Dean for the School of Film and Acting, School of Animation and Motion, and School of Creative Technology. “It is bleeding edge new… and our students are not just learning on it. They're also learning from the masters.”

Under the guidance of their professors, mentors from Industrial Light & Magic and Meptik, and XR/VFX industry experts, SCAD students are now creating commercials, films, and experimental VR projects with the LED Volume. The stage and large background screen setup use extended reality technology that combines live-action performance on stage and real-time environments – animated by SCAD students in Epic Games’ Unreal Engine – projected on the LED panels. 

The stage uses a 6K Komodo camera with a Redspy tracking system on a 15-foot Jimmy Jib that is programmed to synch up with sensors (in the form of white dots) on the ceiling that allows the 3D/CG background scene to automatically shift as the camera moves. Adjustments to the background – such as location of landscape features, speed of passing clouds, color and vibrance, and more – can be adjusted in real-time during filming. 

“We have incredible flexibility here to change anything we want on the fly, which gives our filmmakers more creative power than they've ever before had,” says Orear as he shows one of the students’ recently created snowscape environments for the volume. “This is a three-dimensional environment, and those mountains are as far away in our virtual world as they would be in the real world. We can step deeper and deeper into infinity, as long as our engineers design it that way. And everything here is being rendered in real-time, meaning it's responding as our camera is moving.”

SCAD is the only university in the world with two LED volumes, SCAD’s first XR stage having opened at Savannah Film Studios in 2021. SCAD’s LED volumes are also the largest at any academic institution in the United States. 

“I really believe that video is the future,” says Nick Rivero, co-founder of Meptik, a creative visual production company, based in Atlanta, focused on innovating extended reality and IC (in-camera) VFX. Meptik has worked with Fortune 100 companies, musical artists, films, and broadcasters to create CES keynotes, music videos, and everything in between. Now, they’re helping usher SCAD into the future of visual experiences and entertainment. 

“I'm very honored to have the chance to bring what we've been doing in the industry, both as a team with ‘disguise’ systems as well as here at SCAD, building not one but two incredible virtual production facilities,” says Rivero, addressing the audience at his AnimationFest panel, Impact: The Evolution of XR Production Technology with Meptik. “You have the chance to go out and create the future with technology that didn't exist two years ago. And that is just mind-blowing to me. And we are excited to be a part of that. But we are excited you get to be a part of that.”

This XR technology, which differentiates itself from VR and AR by merging the real and virtual worlds, was developed for shows like The Mandalorian, and it’s been used on movies like The Batman and Thor: Love and Thunder, along with TV series like Obi-Wan Kenobi. 

“We have people coming in talking about their advancements, like, ‘Okay, this is what we learned on Season One of The Mandalorian, and this is what we changed for Season Two,’” says Orear. “This means that our students are going out not just competent, but as thought leaders.”

In addition to the snowscape, SCAD Atlanta students – mostly juniors, seniors, and graduate students – have developed roughly six other environments with an additional six in the works. These include a magical forest with bioluminescent plants; a gothic library; a land of mushrooms where the actors are the size of a fairy; one called “Large Creatures” with a setting of sand dunes and red rock formations with flying whales that slowly pass by; and another called “Elevated City” where the actors are on an elevator going from the lower section of a planet that's underneath clouds and where citizens don't ever see sunlight up to an ornate city above the clouds where the upper class lives.

The students have also developed an experimental virtual reality piece using the LED volume. 

“Someone actually plays a game in virtual reality on the stage and participants can watch them and see what they're playing against on the volume,” explains Eric Allen, Associate Chair of Interactive Design and Game Development and Associate Chair of Motion Media for SCAD Atlanta. “But the participants that are the observers can also interact with an iPad touchscreen device and they can help the person that's playing in virtual reality to keep away the onslaught of these robots in the game. It has old Nintendo style graphics, but the entire thing is built with modern day engines running on this extremely modern technology.”

And all the new solutions and ideas birthed from the minds of SCAD students are relayed back to Epic Games and disguise, directly informing changes they're making to their software. “We're able to steer and help develop the language of what virtual production is going to be like for the next decade or so,” says Dan Bartlett, Dean of SCAD’s School of Animation & Motion.  “It’s a lot of responsibility, but it’s also a privilege.”

Luckily, Bartlett says SCAD students are “fearless” regarding the potential burdens that come with working on the bleeding edge. 

“SCAD students have this energy and ambition that makes them fearless,” says the dean. “So, when we introduce a new technology, like the XR stage, or even just bring in something as simple as Unreal Engine to the animation program, they attack it fearlessly. They're not intimidated by it at all. And that's great, because it means they just dive in and just start creating and building ideas. And actually, a lot of the work that we do as faculty is to bring them down a bit, you know, to kind of temper the ideas and help them streamline their creative thinking so that they can realize projects.”

While Meptik has been working with XR stages since 2018, the LED volume technology didn’t really take off until the pandemic hit in 2020.

“Because of the pandemic, all the production studios in the country had to figure out a way to still travel and still shoot the things that they needed to shoot,” says Estes. “But we were all on lockdown, and you couldn't travel. I think that really kicked this technology into high gear.”

And SCAD was quick to respond, as well. 

“The speed at which this happened, that's what we call ‘SCAD-speed’ as well,” says Andra Reeve-Rabb, Dean at SCAD’s School of Film and Acting who has been a big part of helping get student actors trained on the XR stage. “We actually had designated space down in Savannah and up here in Atlanta to build studios. Not necessarily XR studios or LED Volumes, but when we saw what was happening in industry after the pandemic, that was our call to action and we built not one, but two of these volumes in a matter of months, because we wanted to support our students, and truly train them for what’s next.”

Estes adds, “This is an unstoppable economic force. And I would say that we're probably not going back to the old way.”

Rather than spending the time and money to travel to six different countries in six months –depending on the shots needed on a production – Estes says it’s preferable to utilize an option where a team can get all the shots they need in one day, in one place.

“You don't have to take your entire 65-person or 85-person crew to China, and then to the beach in Savannah, or wherever else you have to take them, and pay for their airplane tickets, hotels and their transport,” says Estes. “With this new technology, our students are able to go to Mars and then to the beach in Savannah, all within one day… if not within a few minutes.”

Orear adds, “One thing that I think gets overlooked is the lighting. If you know anything about filmmaking on location, if you want to shoot a five-minute scene that needs to take place at sunset, you're going to be coming back probably at least two or three times because sunset lasts 45 minutes. Well, here, we can just load up sunset and shoot for 12 hours.”

But one of the biggest innovations the LED Volume contributes to the entertainment industry has nothing to do with time or money, and everything to do with community building within the production pipeline. The volume has acted almost like a center of gravity for SCAD, pulling all the departments together in a way that’s rarely seen, if ever, in live-action productions. 

“It’s got this gravity that's pulling everybody into it,” says Allen. “There's a lot of collaboration that has to happen, and with that comes challenges. But, at the same time, once you bring students together, and they see what's possible, they see projects that have been executed, then all of a sudden, a light comes on and bigger and better things follow. It's really a positive loop.”

Production design and game design students work on the virtual art department side, building assets, getting the right textures, and producing the photogrammetry work that's needed for the digital materials. Visual effects and animation students come in to build sequences within Unreal Engine, such as birds flying, cloud patterns, or even the cycle and movement of the moon and sun. Sculpting students build the stage sets, Fabrics students build the actor costumes, Film and Acting students, of course, come on board for shooting. 

Naturally, all these departments have been included on live-action projects before, but rarely are they working together at the same time as they are with the XR stage. 

“The process is really inverted where visual effects, animation and design students are working ahead of production and testing things out with the actors before the shoot rather than in post-production,” says Aaron McComas, Associate Chair of Animation and Associate Chair of Visual Effects for SCAD Atlanta. “So, you'll be testing that real-time render, and optimizing it, in preparation for the day of shoot. It does mean that their schedule has changed a lot. You're doing some stuff in post, like editing of course, but most of the visual work has to be done early.”

He adds, “It’s going to be more like the old Star Wars movies. The core group that was there in the 70s, the art directors and the prop builders, they were a really tight-knit community and bringing the visual effects and the developers on site for the shoot, working so closely together with the directors and the talent, that's really going to be a huge benefit for everybody.”

Since the actors are not on a greenscreen, all the work normally done in post has to be done before shooting, which means there’s a lot more collaboration happening between all the departments. No one is fixing mistakes or adding new content to what’s already there. They are building separate puzzle pieces that have to fit together perfectly when they’re all presented to each other at the same time, or else no picture will get made. Despite the pressure, McComas believes this will actually lead to less stressful working environments. 

“Because you know right away if something isn’t going to work,” he says, “there’s a lot less stress and there’s no guesswork, wondering if your eight hours of rendering will actually work. The only stress is coming from making sure you get things setup right and getting it ready in time for the shoot.”

McComas says this has been especially beneficial for his visual effects students, who are so used to working on their own, post-production island in the filmmaking process. 

“The visual effects students really enjoy the fact that they're not relegated to one facility anymore,” shares McComas. “They’re not siloed. They're working in conjunction with the directors, with the animators and the storytelling side with the film students. It’s more like being on set as a visual effects supervisor and getting to see the process as they go.”

He continues, “The collaboration that you get between everyone in an environment like this, where all departments are working together, is just so much stronger. So, I think that we'll see a lot more effects artists understanding animation a little bit better and vice versa.”

This heightened teamwork, partnered with advance visualization technology, allows filmmakers to not only further understand the process of departments they’ve never spoken to before now, but to also show the worlds they’ve always imagined to the people they’re working with. Especially the actors, who can see the monitors as they perform and thus watch themselves interact in these environments, seeing in real-time what the director is seeing.

“In the past, we've been constrained by technology,” says Allen. “Technology has driven storytelling. But, with this, it’s opening the floodgates. Never before have you been able to create anything you want, and have your actors live in that world as they act. That kind of synergy that the volume bringing together is going to create visuals and experiences and performances that we haven't seen before.”

During Meptik’s panel at AnimationFest, plenty of students voiced concerns about the unprecedented power that XR is having on the industry. Will the lack of production travel be isolating? Will the world be so focused on the virtual reality that it forgets about the physical world? As AI becomes more advanced, will the desire for a human touch to creative work disappear? Rivero, while acknowledging that any step forward has its risks, believes that the “future of experience is focused on one key thing: creating compelling content.”

“We communicate visually” says Rivero. “We process visuals 60 to 1000 times faster than text, and people learn better with visuals. But the way we do that is not just through the visuals, but through the narratives, through the story we tell with those visuals. Humans are storytellers and stories have been with us and passed down to us since the beginning of time. The most fascinating aspect of this technology is the ability to continue telling these stories, but to tell the stories visually in ways that we could never achieve before.”

He continues, “It’s no longer pages in a book. It's actually about standing in those places. Star Wars has been putting us in these ‘other worlds’ for 40 years, but now I think we have the ability to take it so much further. We can go to not just places in time, but we can go to abstractions, we can stand in a comic book, in a painting, we can preserve history and create the future. Those are things that you couldn't do before. And these students are only at the beginning of what's possible with this tech.”

While many of the projects are still in production, SCAD Atlanta is hoping to feature their students’ work with the LED volume at their TVfest, which takes places February 9-11, 2023.

Victoria Davis is a full-time, freelance journalist and part-time Otaku with an affinity for all things anime. She's reported on numerous stories from activist news to entertainment. Find more about her work at victoriadavisdepiction.com.