Материал: [2.1] 3D Imaging, Analysis and Applications-Springer-Verlag London (2012)

Fig. 1.9 Left: calibration targets used in a camera calibration process, images courtesy of Hao Sun. Right: after calibration, it is possible to determine the positions of the calibration planes using the estimated extrinsic parameters. Figure generated by the Camera Calibration Toolbox for Matlab, webpage at http://www.vision.caltech.edu/bouguetj/calib_doc/, accessed 22nd Dec 2011. Page maintained by Jean-Yves Bouguet

board image. The same corner position input data can be used to calibrate a stereo rig, whereby two sets of intrinsic parameters are established and the extrinsic parameters define the 6 degree-of-freedom rigid pose of one camera relative to another.

1.5.11 3D Shape Matching: Heat Kernel Signatures

One problem with spin images and other local shape descriptors is that they are encoded in Euclidean space and are only valid for rigid local shapes. Understanding shapes in terms of their geodesic distances23 can yield more generic approaches that are not degraded by the bending of the shape (i.e. they are isometric invariant). In 2009, Sun et al. [48] presented a multi-scale shape signature that is isometric invariant. It is based on the properties of the heat diffusion process over a meshed surface and belongs to a class of methods known as diffusion geometry approaches. The concise signature is obtained by restricting the heat kernel to the temporal domain. This technique and other approaches involving diffusion geometry enable high performance 3D shape matching under isometric deformation and are thus finding their way into 3D object recognition and 3D shape retrieval (search) applications. Figure 1.10 shows heat kernel signatures extracted on two instances of an isometrically deformed shape.

23A geodesic distance between two points on a surface is the minimal across-surface distance.

Fig. 1.10 Heat kernel signatures calculated on two isometric shapes. The top row shows that signatures at corresponding points look very similar. The bottom row shows that signatures at different points on the mesh differ. Figure reproduced from Chap. 7, courtesy of Benjamin Bustos

1.5.12A Seminal Application: Real-Time Human Pose Recognition

In 2011, Shotton et al. [46] presented a system that was able to segment the whole body into a set of parts based on a single range image (i.e. no temporal information is used) and thereby determine, in real-time, the position of human body joints. This process is illustrated in Fig. 1.11. A design aim was high frame rates and they achieved 200 frames per second on consumer hardware. The system can run comfortably using an inexpensive 3D camera and the algorithm forms a core component

Fig. 1.11 From a single depth image, range pixels can be labeled as belonging to certain body parts, which are colored differently in the figure. From this labeling, 3D joint positions can be inferred. Figure courtesy of [46]

of the Kinect-XBox gaming platform. To achieve high frame rates, extremely simple and computationally cheap 3D features were employed based on depth comparisons and randomized decision forest classifiers were trained and implemented to run on a GPU. The training data uses 100,000 captured poses, each of which generates a synthetic set of 15 base meshes, spanning a range of different body sizes, shapes, viewpoints, clothing and hairstyle. This system is an archetypal modern, successful 3D recognition system that shows what can be achieved when machine learning techniques, a large corpus of training data and fast computational architectures are brought together.

1.6 Applications of 3D Imaging

Probably the most mature techniques in range sensing are 3D scanner applications in geodetic, architectural and industrial surveys. Airborne LIDAR systems are being used routinely now to survey building structures in cities leading to the generation of digital elevation maps (DEMs). High-precision DEMs of rural regions and forest areas are also produced for various purposes. Multiple reflection response LIDAR systems record and evaluate multiple reflections in forest areas that arise from height differences between the solid forest floor and the foliage. These systems are combined with photogrammetric DEM estimation and semantic segmentation from aerial imagery to automate map generation (e.g. for navigation) and topographic map creation. Figure 1.12 depicts an example of a 3D DEM, mapped with texture from a satellite image. Chapters 9 and 10 will explicitly deal with these applications.

Ground-based 3D scanner stations are used to reconstruct 3D structures, such as buildings (indoor and outdoor) for the purpose of documentation and mensuration. One application area is preservation and electronic documentation of cultural heritage, such as scanning of 3D models from famous buildings, old castles, churches and archaeological artifacts. The conversion of these data into 3D models and their

Fig. 1.12 Bird’s eye view of a mountainous area from a 3D DEM mapped with texture from a satellite image. Copyright METI/NASA, reprinted with permission from ERSDAC (Earth Remote Sensing Data Analysis Center), http://www.ersdac.or.jp/GDEM/E/2.html

Fig. 1.13 Example 3D scans from the Bosphorus dataset with occlusions, expression variations and pose variations [41]. Figure adapted from [11]

presentation by electronic means will help to preserve cultural diversity and will enable online access to such sites. Another application area is the precise 3D documentation of industrial sites and fabrication plants for as-built control and quality control. It is very important to document the changes in building structures and industrial plants, since this will greatly facilitate planning and integration of new structures. A challenge for reconstructing complete sites from range imaging devices is the need to fuse all surface data into a consistent 3D representation and to extract functional objects from the 3D points. This involves the registration of partial surfaces into complete models, as was illustrated in Fig. 1.7.

The handling of dynamic and time-varying 3D shapes is another emerging field which heavily relies on 3D imaging techniques. With the advent of range video cameras that capture depth at video frame rates, like the ToF 3D range cameras or the Kinect sensor, it becomes possible to observe deforming shapes over time and to model these deformations accordingly [27]. Of particular interest is the tracking and modeling of human motion, expression and behavior. Much work investigated traditional dynamic shape techniques, such as active motion capture systems using markers, or markerless multiview shape from silhouette for human motion capture and much has already been achieved regarding motion and behavior modeling. Novel and improved range imaging technology is now facilitating and significantly influencing motion and behavior modeling. As already discussed, a recent example is the very successful combination of the Kinect sensor with the XBox human motion capture system24 for interactive games. Prior approaches that utilized 2D images only are significantly inferior to the detailed tracking of body and limbs that is becoming available with range data.

In addition to human pose and gesture tracking, the analysis of human faces and face recognition systems are gaining importance. In particular, face recognition can be improved significantly if 3D face models and 3D face data are available. Systems based on 3D face shape are much more robust when dealing with changing illumination and pose, and facial views can be normalized for better recognition. Figure 1.13 shows a set of rendered face scans from the Bosphorus dataset [41]. Current research aims to provide 3D face recognition under occlusion, facial ex-

24Kinect and XBox are trademarks of Microsoft Corporation.