3D Gaussian Splatting for Real-Time Radiance Field Rendering

Description

Radiance Field methods have recently revolutionized novel-view synthesisof scenes captured with multiple photos or videos. However, achieving highvisual quality still requires neural networks that are costly to train and render,while recent faster methods inevitably trade off speed for quality. Forunbounded and complete scenes (rather than isolated objects) and 1080presolution rendering, no current method can achieve real-time display rates.We introduce three key elements that allow us to achieve state-of-the-artvisual quality while maintaining competitive training times and importantlyallow high-quality real-time (≥ 30 fps) novel-view synthesis at 1080p resolution.First, starting from sparse points produced during camera calibration,we represent the scene with 3D Gaussians that preserve desirable propertiesof continuous volumetric radiance fields for scene optimization whileavoiding unnecessary computation in empty space; Second, we performinterleaved optimization/density control of the 3D Gaussians, notably optimizinganisotropic covariance to achieve an accurate representation of thescene; Third, we develop a fast visibility-aware rendering algorithm thatsupports anisotropic splatting and both accelerates training and allows realtimerendering. We demonstrate state-of-the-art visual quality and real-timerendering on several established datasets.

Abstract

International audience

Additional details