Abstract


This paper targets high-fidelity and real-time view
synthesis of dynamic 3D scenes at 4K resolution. Recently, some
methods on dynamic view synthesis have shown impressive rendering quality. However, their speed is still
limited when rendering high-resolution images. To overcome this problem, we propose 4K4D, a 4D point cloud representation that supports hardware
rasterization and enables unprecedented rendering speed. Our representation is built on a 4D feature grid so
that the points are naturally regularized and can be robustly optimized. In addition, we design a novel
hybrid appearance model that significantly boosts the rendering quality while preserving efficiency.
Moreover, we develop a differentiable depth peeling algorithm to effectively learn the proposed model from
RGB videos. Experiments show that our representation can be rendered at over 400 FPS on the DNA-Rendering
dataset at 1080p resolution and 80 FPS on the ENeRF-Outdoor dataset at 4K resolution using an RTX 4090 GPU,
which is 30x faster than previous methods and achieves the state-of-the-art rendering quality.

Read More