AESplat: Advancing Pose-free Feed-forward 3D Gaussian Splatting via Decoupled Appearance Modeling

TL;DR

• AESplat shifts from a unified appearance modeling paradigm to a decoupled one for pose-free feed-forward 3D Gaussian Splatting, substantially improving rendering quality.
• We propose an efficient decoupled appearance modeling strategy: the zeroth-order SH coefficient (view-independent appearance) is derived directly from input images without training (I2DC), while higher-order SH coefficients (view-dependent appearance) are predicted by a shallow MLP with two 3D-aware inductive biases: GVSRE and WCM.
• AESplat achieves state-of-the-art novel view synthesis, with a 0.8 dB PSNR gain over the pose-free method NAS3R and a 1.1 dB gain over the pose-required method DepthSplat on RealEstate10K.

Demo

3D Reconstruction

Interactive 3D Gaussians exported as .glb (RealEstate10K). Drag to orbit, scroll to zoom.

Abstract

Pose-free feed-forward 3D Gaussian Splatting (3DGS) has demonstrated remarkable potential for generalized novel view synthesis. However, existing methods typically predict Gaussian appearance attributes represented by spherical harmonics (SH) in the same manner, overlooking the fundamental distinction between view-independent and view-dependent appearance, which results in suboptimal rendering quality. In this paper, we present AESplat, a novel and general framework for pose-free feed-forward 3DGS that introduces an effective decoupled appearance modeling strategy based on an analysis of SH, enabling higher-quality rendering. Specifically, AESplat directly derives the zeroth-order SH coefficient, which represents the base view-independent appearance component, from the input images without training. The higher-order SH coefficients are subsequently predicted by a shallow multilayer perceptron equipped with two efficient 3D-aware inductive biases to model view-dependent appearance variations. Extensive experiments across multiple datasets demonstrate that our method significantly outperforms state-of-the-art approaches, achieving a 0.8 dB improvement in PSNR over the pose-free method NAS3R and a 1.1 dB improvement over the pose-required method DepthSplat on the RealEstate10K dataset.

Method

Given unposed images, AESplat jointly predicts camera poses and Gaussian primitives in a single forward pass, with appearance attributes obtained through decoupled appearance modeling. Leveraging the strong observation provided by the input image itself, the zeroth-order SH coefficient of each pixel-aligned Gaussian is directly derived from its corresponding pixel RGB value via Image-to-Direct Component (I2DC), without training. Higher-order SH coefficients are predicted by a view-dependent appearance head that takes two effective 3D-aware inductive biases—Gaussian-to-Views Spatial Relation Embedding (GVSRE) and Warped Color Map (WCM)—enabling more accurate modeling of view-dependent appearance. Built upon a pose-free feed-forward backbone, this decoupled strategy produces higher-quality Gaussian representations and better novel-view rendering.

Quantitative Results

Qualitative Results

Qualitative comparison on RealEstate10K. The leftmost column shows the two-view context images.

Qualitative comparison on ACID. The leftmost column shows the two-view context images.

Cross-dataset generalization (trained on RealEstate10K): DL3DV (first four rows), ACID (fifth and sixth rows), and ScanNet++ (last two rows).