Imagine exploring Paris through a digital window in your living room, using a camera to capture a 3D blueprint of your house to plan your next remodel, or trying on clothing in a virtual dressing room that shows how fabrics flow with your every move. These are a few potential applications of visual computing, a fast-growing field of technology that combines photorealism, HD video and audio, interactivity and computational modeling to enable real-time, lifelike immersive experiences.
The demand for visual computing is accelerating in parallel with increasing computational power that enables the processing of complex visual data, the proliferation of mobile devices and the demand for improved user experiences. In the future, people will use their computing platforms to interact with the world in richer ways, letting them enjoy realistic simulated experiences that blur the line between the physical and virtual worlds.
Investing in the Future
The Science and Technology Center for Visual Computing, launched in early 2011, is focused on the development of innovations in lifelike computer graphics, natural user interfaces and realistic virtual humans that will in the future make people’s technology experiences more immersive. The goal is to drive visual computing applications that not only look, act and feel real, but also accelerate the pace at which these innovations reach consumers.
The project is a collaboration between experts from eight top U.S. universities: Cornell, Harvard, Princeton, Stanford, UC Berkeley, UC Davis, UC Irvine and the University of Washington. Stanford is the hub of the virtual center, which drives projects that will explore fundamental research problems and potential applications in this emerging space.
While much of the research of the Center will focus on the development of systems and application software to enable visual computing, the results will be used to guide the development of future hardware platforms. The work of the Center is divided into four overlapping research themes: scalable real-time simulation, perception of people and places, content creation, and graphics and systems.
Real-time Simulation
The simulation of physical phenomena is central to visual computing -- from light transport and appearance to the dynamics of fluids and solids, the movement of virtual characters and increasingly the sounds these systems make. Such multisensory simulations are notoriously expensive to compute, making it difficult to realize real-time simulations and develop simulation applications for mobile computing.
This theme will address physics-based simulation and multisensory rendering in an integrated and connected manner: light, motion and sound. Research will focus on the unique challenges that simulated virtual characters pose. In addition, this theme will explore the computational challenges of scalable simulation as it relates to multi-physics and multisensory software integration, parallel and distributed computing, interactive and hard real-time computation, model complexity (e.g., planetary-scale simulation), the spectrum of computing platforms (from mobile to the cloud), and ever-present memory and bandwidth concerns.
Perception of People and Places
The proliferation of digital cameras, coupled with explosive progress in computer vision, has led to major breakthroughs in sensing technologies. These technologies will impact our everyday lives -- in cameras, maps and search capabilities -- with many more uses on the way.
Advances are due mainly to the recent development of extremely accurate and robust low-level computer vision algorithms for feature detection, matching and 3D measurement. The next wave of breakthroughs will be defined by the ability to infer high-level functional and semantic information about people and environments from images and video. Beyond determining if a person is present, next-generation systems will reliably perceive who it is and what that person is doing, down to the level of actions and activities.
Such capabilities are already beginning to transform gaming experiences through 3D camera-based motion detection and control, but this is just the beginning. Next-generation 3D vision systems will go beyond raw depth measurement to perceiving the functional and semantic content of the scene. While current 3D-modeling methods represent the scene as an unorganized mass of points or triangles, next-generation systems will recognize what is in the scene -- doors, chairs, stairs, sidewalks, windows, tables and other components. Beyond scene visualization, these new capabilities will enable applications -- such as home remodeling pre-visualization and the building of searchable, functional 3D city models from online imagery and light detection and ranging (LIDAR) data -- that will enhance online virtual worlds.
Content Creation
Creating visual content is difficult. Consider any visual medium: photographs, video, 3D models and so on. The software tools for creating such media usually include sophisticated interfaces that provide precise and flexible control over the content. Yet, they are dauntingly difficult to learn to use effectively.
Although cameras now allow anyone to easily capture photos and video, tools for manipulating such media and creating other forms of visual content remain accessible only to experts. Two recent trends offer the promise of enabling a significantly wider set of everyday users -- of all skill levels -- to produce visual content: collaboration via the Internet and new sensing hardware for input.
Graphics and Systems Research
To create the future architectures needed to support advances such as those outlined in the above three themes, researchers at the Center for Visual Computing will explore next-generation architectures and tools that address the following four crucial technology trends:
Creating the Future
It’s difficult to imagine all the potential visual computing experiences that will be possible in 10 years, but by bringing together the top minds in the field -- in Europe and now across the United States -- The Science and Technology Center for Visual Computing has made a substantial commitment to advancing the future of visual computing, for the benefit of researchers and consumers alike.