A framework for high‐resolution, high‐fidelity, inexpensive facial scanning

We present a framework for high‐resolution, high‐fidelity, inexpensive facial scanning. The framework combines the speed and cost of passive lighting scanning systems with the fidelity of active lighting systems. The subject is first scanned at the mesoscale, the scale of pores and fine wrinkles. The process is a near‐instant method for acquiring facial geometry and reflectance with 24 DSLR cameras and ten flashes. The flashes are fired in rapid succession with subsets of the cameras, which are specially arranged to produce an even distribution of specular highlights on the face. The total capture time is less than the mechanical movement of the eyelid in the human blink reflex. We use this set of acquired images to estimate diffuse color, specular intensity, and surface orientation at each point on the face. With a single photo per camera, we optimize the facial geometry to maximize the consistency of diffuse reflection and minimize the variance of specular highlights using an energy‐minimization message‐passing technique. This allows the final sub‐millimeter surface detail to be obtained via shape‐from‐specularity, even though every photo is from a different viewpoint. The final system uses commodity components and produces models suitable for generating high‐quality digital human characters. The mesostructure is enhanced to include microgeometry through the scanning of skin patches around the face. We digitize the exemplar patches with a polarization‐based computational illumination technique which considers specular reflection and single scattering. The recorded microstructure patches can be used to synthesize full‐facial microstructure detail for either the same subject or a different subject with similar skin types. We show that the technique allows for greater realism in facial renderings including a more accurate reproduction of skin’s specular reflection effects. A microstructure database is provided for easy cross‐subject synthesis during the enhancement stage. Additionally, a multi‐view camera calibration technique is introduced. This new technique can be accomplished with a single view from each camera of a cylinder wrapped in a checkerboard pattern. It is fast and resolves extrinsic and intrinsic camera parameters to a sub‐pixel re‐projection error.