Skullptor: High-Fidelity 3D Head Reconstruction in Seconds

The Challenge of Capturing a Human Face

Creating a realistic digital human head for video games comes at a hefty price—not just in terms of money, but also in time and infrastructure. The industry gold standard relies on photogrammetry: surrounding an actor with dozens, sometimes hundreds, of synchronized cameras to triangulate every pore, wrinkle, and facial contour into a dense 3D mesh. The results are impressive, but the setup requires a dedicated light stage, extensive processing time, and skilled artists to clean up artifacts, especially around hair and fine facial structures.

One alternative has been to rely on AI. Recent AI foundation models can reconstruct a 3D face from a single image in seconds. Unfortunately, this efficiency comes at the expense of person-specific details. The subtle geometry that makes a face uniquely identifiable—the exact depth of a laugh line, the specific fold of an eyelid—gets averaged out. The result is plausible, but not accurate.

With Skullptor, we asked: what if we didn't have to compromise?

A Hybrid Approach: The Core Idea

Skullptor is based on a simple but powerful insight: data-driven foundation models and direct optimization are not competing approaches. Instead, they are complementary.

Foundation models are fast, and they encode a deep understanding of human faces from large-scale training data. Optimization-based methods are precise and can lock onto person-specific geometry by directly minimizing a geometric error. The weakness of one is the strength of the other.

So, we built a two-stage pipeline that uses them in sequence.

In the first stage, a multi-view neural network takes a sparse set of images—ranging from three to ten—and predicts surface normal maps for each viewpoint. Surface normals represent the orientation of every point on the skin surface, essentially encoding a dense geometric blueprint of the face. Crucially, this stage runs with a single feed-forward pass, taking roughly 1.5 seconds.

In the second stage, these predicted normals become the input to an inverse rendering optimization. Starting from a sphere, a mesh is iteratively deformed so that its rendered normals match the predicted ones, with adaptive remeshing applied at every step to recover fine surface detail. The whole optimization runs in under 30 seconds.

The result is a complete, detailed 3D head mesh—with wrinkles, skin folds and person-specific surface geometry—reconstructed in seconds from a handful of cameras.

Ensuring Geometric Consistency of AI Predictions 

Using a foundation model for normal prediction comes with a catch. These models were designed to process one image at a time. When fed the same face from three different angles, each prediction is made independently, with no awareness of the others. A skin fold in front view and the same fold in side view will produce normals that disagree in 3D space. When trying to reconstruct geometry from these conflicting normals, these features average out and are smoothed over.

To solve this, we built a model based on DAViD, a monocular foundation model for facial normal estimation, extending it with view-aware cross-attention. Between each transformer block, we inserted a cross-attention layer that allows every viewpoint to attend to all others simultaneously. Feature tokens in each image represent the query; the keys and values are constructed from the concatenated features of the entire set of images. Camera pose is encoded as a positional embedding and injected directly into this process, so the model knows from which viewpoint each token originates.

This way, the model predicts normals that are geometrically consistent across all viewpoints. This consistency is what makes the downstream optimization high-quality and stable.

From Normal Maps to a 3D Mesh

Once we have consistent normal maps, we can start the optimization stage. We initialize a mesh as a unit sphere and iteratively optimize vertex positions to minimize the difference between the mesh's rendered normals and the predicted ones.

Two design choices make this approach work well in practice.

First, we weight the normal loss based on camera-facing angle. Normals predicted for surface regions directly facing the camera are more reliable than those at grazing angles, and the optimization reflects this by ensuring that the frontal regions contribute more to the reconstruction loss.

Second, we use the Continuous Remeshing optimizer throughout the reconstruction. At each iteration, edge splits, collapses, and flips are applied to adapt the mesh resolution dynamically to local geometric complexity. Additionally, this optimizer adapts to the scale of the local geometry at each vertex. This prevents the mesh from degrading into self-intersections or collapsed faces, and allows fine details like wrinkles to emerge cleanly.

The full optimization runs for 300 steps and completes in under 30 seconds on a single consumer-grade GPU.

Why the Hybrid Approach Matters

The key reason for the success of this project was choosing to use surface normals rather than depth maps or explicit 3D geometry.

First, surface normals are incredibly well-suited for AI foundation models. Unlike complex 3D geometry, normals can be easily rendered from synthetic data to create high-quality training sets. Because predicting them involves a dense, per-pixel regression task, standard dense prediction transformers (DPTs) can be naturally adapted to the problem.

Second, normals provide a superior geometric representation for multi-view optimization. While depth maps only indicate how far away a point is, normals encode the orientation on the surface. This means they capture higher-frequency details—like the fine edge of a wrinkle or the sharp contour of an eyelid—far better than depth can. Crucially, normals are scale-invariant. They describe local shape rather than absolute distances, which side-steps the scale-matching issues that tend to break multi-view reconstruction pipelines.

By using geometrically consistent normals as our optimization target, Skullptor bridges the gap between AI speed and classical precision. It allows us to operate in a regime where traditional photogrammetry fails entirely—for example, by using only three cameras—while still recovering the sharp, identity-defining details that make a digital human feel real.

Results

We evaluated Skullptor on two publicly available datasets: NPHM, which contains high-quality structured-light scans, and Multiface, which provides lightstage multi-view video captures.

Across both datasets, Skullptor matches photogrammetry-level geometric quality, while using a fraction of the input views and completing reconstruction in seconds rather than minutes. It substantially outperforms recent Gaussian splatting-based reconstruction methods (2DGS and SuGaR) on all metrics, including depth accuracy, normal angular error, and preservation of high-frequency surface detail.

We wish to highlight the sparse-view ablation: as the number of input views drops from 23 to 3, Skullptor degrades gracefully, and quality stays high even using three cameras. Classic photogrammetry, however, degrades sharply below 16 views and generates near-unusable results using three cameras.

Qualitative comparisons tell a similar story: Skullptor recovers wrinkles, skin folds, and person-specific surface variation that Gaussian splatting methods cannot reproduce.

Conclusion

Skullptor demonstrates that combining a data-driven normal predictor with a fast inverse rendering optimization bridges the gap between the speed of AI foundation models and the quality of dense-view photogrammetry. From fewer than ten cameras, it produces complete, high-fidelity 3D head geometry with wrinkle-level detail in under 30 seconds.

The current method is designed for controlled capture conditions, synchronized cameras and reasonably consistent lighting. Strong specular reflections, motion blur, or occlusions can affect the final result. Next steps include extending the framework toward full appearance capture, jointly predicting normals and albedo, and incorporating lighting estimation to enable relighting.

We are releasing the code to support future research. More results, including 4D sequence reconstructions, are available on the project page.

Additional Information

For more information on this research, please refer to the following article: 

Artru, N., Hussain, R., Got, E., Messier, A., Lindell, D. B., & Dib, A. (2026). Skullptor: High Fidelity 3D Head Reconstruction in Seconds with Multi-View Normal Prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).