Skyfall-GS: Real-Time City-Scale 3D from Satellite Images via Diffusion-Guided Refinement

Skyfall-GS generates immersive, city-block-scale 3D urban scenes solely from multi-view satellite imagery. It first reconstructs a 3DGS scene with pseudo-depth and illumination modeling, then iteratively refines it via a curriculum-based dataset update using a T2I diffusion model. The approach yields more accurate geometry and photorealistic textures than prior work, enabling real-time 3D exploration.

Key Points

Introduces Skyfall-GS, a city-block-scale 3D scene synthesis framework from satellite imagery that requires no 3D annotations and renders in real time.
Uses 3D Gaussian Splatting with pseudo-camera depth supervision to overcome limited parallax and an appearance model to handle multi-date illumination changes.
Proposes a curriculum-driven Iterative Dataset Update (IDU) that employs a pre-trained T2I diffusion model with prompt-to-prompt editing to iteratively refine training renders.
The iterative process improves geometric completeness, cross-view consistency, and photorealistic textures while reducing visual artifacts.
Experiments show Skyfall-GS outperforms state-of-the-art methods in geometry and texture realism for large-scale urban scene synthesis.

Sentiment

The community is generally impressed by the research and sees genuine promise, particularly for catastrophe modeling and flight simulation. However, there is widespread agreement that the current quality claims are overstated — results break down significantly at close range. The tone is constructive rather than dismissive, with many suggesting paths forward like hybrid approaches and drone data.

In Agreement

The technology is impressive for what it achieves from satellite data alone, and the approach will continue to improve over time
Using cheaper data sources like satellite imagery instead of licensed Street View data is valuable for production scenarios
Practical applications exist in catastrophe modeling, urban heat island analysis, and building height/volume estimation
Drone data instead of satellite imagery could significantly improve detail and make results truly immersive
Hybrid approaches combining Gaussian Splatting coarse geometry with traditional rendering methods could overcome current limitations

Opposed

"Explorable" and "immersive" oversell the current quality — artifacts are very obvious below building level, worse than traditional photogrammetry
Gaussian Splatting is reconstructive, not generative — it cannot fix missing data without introducing unfaithful hallucinations
Gaussian Splatting is unsuitable for games because collision geometry cannot be derived from gaussian splats
Microsoft Flight Simulator already achieves similar results using traditional photogrammetry from satellite photos
The project lacks practical accessibility — no simple demo interface, only training scripts