Материал: [2.1] 3D Imaging, Analysis and Applications-Springer-Verlag London (2012)

post production. Figure 1.14 shows the camera system, one of the original views, the depth and color views from the enhanced scene, including real and virtual content. Without range data, such scene composition is a tedious manual task, as used in traditional video post processing today. With the use of depth information, there is a wealth of opportunities to facilitate and to automate data processing.

Although many different techniques have been proposed for a wide variety of 3D imaging applications, many of them work well only in constrained scenarios and over a subset of the available datasets. Many techniques still fail on challenging data and techniques with more robustness and better overall performance still need to be developed. Deeper insights into various 3D imaging, analysis and application problems are required for the development of such novel techniques.

1.7 Book Outline

We conclude this chapter with a roadmap of the remaining chapters of this book. Although there is a natural order to the chapters, they are relatively self-contained, so that they can also be read as standalone chapters. The book is split into three parts: Part I is comprised of Chaps. 2–4 and it presents the fundamental techniques for the capture, representation and visualization of 3D data; Part II is comprised of Chaps. 5–7 and is concerned with 3D shape analysis, registration and matching; finally, Part III is comprised of Chaps. 8–11 and discusses various application areas in 3D face recognition, remote sensing and medical imaging.

1.7.1 Part I: 3D Imaging and Shape Representation

Chapter 2 describes passive 3D imaging, which recovers 3D information from camera systems that do not project light or other electromagnetic radiation onto the imaged scene. An overview of the common techniques used to recover 3D information from camera images is presented first. The chapter then focuses on 3D recovery from multiple views, which can be obtained using two or more cameras at the same time (stereo), or a single moving camera at different times (structure from motion). The aim is to give a comprehensive presentation that covers camera modeling, camera calibration, image rectification, correspondence search and the triangulation to compute 3D scene structure. Several 3D passive imaging systems and their realworld applications are highlighted later in this chapter.

In Chap. 3, active 3D imaging is discussed. These systems do project light, infrared or other electromagnetic radiation (EMR) onto the scene and they can be based on different measurement principles that include time-of-flight, triangulation and interferometry. While time-of-flight and interferometry systems are briefly discussed, an in-depth description of triangulation-based systems is provided, which have the same underlying geometry as the passive stereo systems presented in Chap. 2. The

characterization of such triangulation-based systems is discussed using both an error propagation framework and experimental protocols.

Chapter 4 focuses on 3D data representation, storage and visualization. It begins by providing a taxonomy of 3D data representations and then presents more detail on a selection of the most important 3D data representations and their processing, such as triangular meshes and subdivision surfaces. This chapter also discusses the local differential properties of surfaces, mesh simplification and compression.

1.7.2 Part II: 3D Shape Analysis and Processing

Chapter 5 presents feature-based methods in 3D shape analysis, including both classical and the most recent approaches to interest point (keypoint) detection and local surface description. The main emphasis is on heat-kernel based detection and description algorithms, a relatively recent set of methods based on a common mathematical model and falling under the umbrella of diffusion geometry.

Chapter 6 details 3D shape registration, which is the process of bringing together two or more 3D shapes, either of the same object or of two different but similar objects. This chapter first introduces the classical Iterative Closest Point (ICP) algorithm [5], which represents the gold standard registration method. Current limitations of ICP are addressed and the most popular variants of ICP are described to improve the basic implementation in several ways. Challenging registration scenarios are analyzed and a taxonomy of promising alternative registration techniques is introduced. Three case studies are described with an increasing level of difficulty, culminating with an algorithm capable of dealing with deformable objects.

Chapter 7 presents 3D shape matching with a view to applications in shape retrieval (e.g. web search) and object recognition. In order to present the subject, four approaches are described in detail with good balance among maturity and novelty, namely the depth buffer descriptor, spin images, salient spectral geometric features and heat kernel signatures.

1.7.3 Part III: 3D Imaging Applications

Chapter 8 gives an overview of 3D face recognition and discusses both wellestablished and more recent state-of-the-art 3D face recognition techniques in terms of their implementation and expected performance on benchmark datasets. In contrast to 2D face recognition methods that have difficulties when handling changes in illumination and pose, 3D face recognition algorithms have been more successful in dealing with these challenges. 3D face shape data is used as an independent cue for face recognition and has also been combined with texture to facilitate multimodal face recognition.

The next two chapters discuss related topics in remote sensing and segmentation. Chapter 9 presents techniques used for generation of 3D Digital Elevation Models (DEMs) from remotely sensed data. Three methods, and their accuracy evaluation, are presented in the discussion: stereoscopic imagery, Interferometric Synthetic Aperture Radar (InSAR) and LIght Detection and Ranging (LIDAR).

Chapter 10 discusses 3D remote sensing for forest measurement and applies 3D DEM data to 3D forest and biomass analysis. Several techniques are described that can be used to detect and measure individual tree crowns using high-resolution remote sensing data, including aerial photogrammetry and airborne laser scanning (LIDAR). In addition, the chapter presents approaches that can be used to infer aggregate biomass levels within areas of forest (plots and grid cells), using LIDAR information on the 3D forest structure.

Finally, Chap. 11 describes imaging methods that aim to reconstruct the inside of the human body in 3D. This is in contrast to optical methods that try to reconstruct the surface of viewed objects, though there are similarities in some of the geometries and techniques used. The first section gives an overview of the physics of data acquisition, where images come from and why they look the way they do. The next section illustrates how this raw data is processed into surface and volume data for viewing and analysis. This is followed by a description of how to put images in a common coordinate frame, and a more specific case study illustrating higher dimensional data manipulation. Finally some clinical applications are described to show how these methods can be used to affect the treatment of patients.

References

1.Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision. In: Landy, M., Movshon, J.A. (eds.) Computational Models of Visual Processing (1991)

2.Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3d point sets. IEEE Trans. Pattern Anal. Mach. Intell. 9(5), 698–700 (1987)

3.Bartczak, B., Vandewalle, P., Grau, O., Briand, G., Fournier, J., Kerbiriou, P., Murdoch, M., Mller, M., Goris, R., Koch, R., van der Vleuten, R.: Display-independent 3d-TV production and delivery using the layered depth video format. IEEE Trans. Broadcast. 57(2), 477–490 (2011)

4.Bennet, R.: Representation and Analysis of Signals. Part xxi: The Intrinsic Dimensionality of Signal Collections, Rep. 163. The Johns Hopkins University, Baltimore (1965)

5.Besl, P., McKay, N.D.: A method for registration of 3D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

6.Bigun, J., Granlund, G.: Optimal orientation detection of linear symmetry. In: First International Conference on Computer Vision, pp. 433–438. IEEE Computer Society, New York (1987)

7.Blundell, B., Schwarz, A.: The classification of volumetric display systems: characteristics and predictability of the image space. IEEE Trans. Vis. Comput. Graph. 8, 66–75 (2002)

8.Boyer, K., Kak, A.: Color-encoded structured light for rapid active ranging. IEEE Trans. Pattern Anal. Mach. Intell. 9(1) (1987)

9.Brewster, S.D.: The Stereoscope: Its History, Theory, and Construction with Applications to the fine and useful Arts and to Education. John Murray, Albemarle Street, London (1856)

10.Brownson, C.D.: Euclid’s optics and its compatibility with linear perspective. Arch. Hist. Exact Sci. 24, 165–194 (1981). doi:10.1007/BF00357417

11.Creusot, C.: Automatic landmarking for non-cooperative 3d face recognition. Ph.D. thesis, Department of Computer Science, University of York, UK (2011)

12.Curtis, G.: The Cave Painters. Knopf, New York (2006)

13.Faugeras, O.: What can be seen in three dimensions with an uncalibrated stereorig? In: Sandini, G. (ed.) Computer Vision: ECCV’92. Lecture Notes in Computer Science, vol. 588,

pp.563–578. Springer, Berlin (1992)

14.Faugeras, O., Luong, Q., Maybank, S.: Camera self-calibration: theory and experiments. In: Sandini, G. (ed.) Computer Vision: ECCV’92. Lecture Notes in Computer Science, vol. 588,

pp.321–334. Springer, Berlin (1992)

15.Faugeras, O.D., Hebert, M.: The representation, recognition and locating of 3-d objects. Int. J. Robot. Res. 5(3), 27–52 (1986)

16.Forsyth, D., Ponce, J.: Computer Vision: A Modern Approach. Prentice Hall, Upper Saddle River (2003)

17.Fusiello, A.: Visione computazionale. Appunti delle lezioni. Pubblicato a cura dell’autore (2008)

18.Gennery, D.B.: A stereo vision system for an autonomous vehicle. In: Proc. 5th Int. Joint Conf. Artificial Intell (IJCAI), pp. 576–582 (1977)

19.Gernsheim, H., Gernsheim, A.: The History of Photography. Mc Graw-Hill, New York (1969)

20.Harris, C., Stephens, M.J.: A combined corner and edge detector. In: Alvey Vision Conference (1988)

21.Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University, Cambridge (2003). ISBN 0-521-54051-8

22.Hartley, R.I.: In defence of the 8-point algorithm. In: Proceedings of the Fifth International Conference on Computer Vision, ICCV’95, p. 1064. IEEE Computer Society, Washington (1995)

23.Hartley, R.I.: In defence of the 8-point algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 19(6), 580–593 (1997)

24.Horn, B.K.P.: Shape from shading: a method for obtaining the shape of a smooth opaque object from one view. Ph.D. thesis, MIT, Cambridge, MA, USA (1970)

25.Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)

26.Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1997)

27.Jordt, A., Koch, R.: Fast tracking of deformable objects in depth and colour video. In: Proceedings of the British Machine Vision Conference (BMVC) (2011)

28.King, H.: The History of Telescope. Griffin, London (1955)

29.Koch, R.: Depth estimation. In: Ikeuchi, K. (ed.) Encyclopedia of Computer Vision. Springer, New York (2013)

30.Koch, R., Schiller, I., Bartczak, B., Kellner, F., Koeser, K.: Mixin3d: 3d mixed reality with ToF-camera. In: Dynamic 3D Imaging DAGM 2009 Workshop, Dyn3D, Jena, Germany. Lecture Notes in Computer Science, vol. 5742, pp. 126–141 (2009)

31.Kolb, A., Barth, E., Koch, R., Larsen, R.: Time-of-flight cameras in computer graphics. Comput. Graph. Forum 29(1), 141–159 (2010)

32.Kolb, A., Koch, R.: Dynamic 3D Imaging. Lecture Notes in Computer Science, vol. 5742. Springer, Berlin (2009)

33.Kriss, T.C., Kriss, V.M.: History of the operating microscope: from magnifying glass to microneurosurgery. Neurosurgery 42(4), 899–907 (1998)

34.Lippmann, G.: La photographie integrale (English translation Fredo Durant, MIT-csail). In: Academy Francaise: Photography-Reversible Prints. Integral Photographs (1908)

35.Longuet-Higgins, H.C.: A computer algorithm for re-constructing a scene from two projections. Nature 293, 133–135 (1981)

36.Ma, Y., Soatto, S., Kosecka, J., Sastry, S.: An Invitation to 3D Vision: From Images to Geometric Models. Springer, Berlin (2003)

37.Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)

38.Marr, D.: Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. Freeman, New York (1982)

39.Nayar, S.K., Watanabe, M., Noguchi, M.: Real-time focus range sensor. IEEE Trans. Pattern Anal. Mach. Intell. 18(12), 1186–1198 (1996)

40.Rioux, M.: Laser range finder based on synchronized scanners. Appl. Opt. 23(21), 3837–3844 (1984)

41.Savran, A., Alyuz, N., Dibeklioglu, H., Celiktutan, O., Gokberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3d face analysis. In: Biometrics and Identity Management. Lecture Notes in Computer Science, vol. 5372, pp. 47–56 (2008)

42.Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)

43.Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge (2002)

44.Schwarte, R., Xu, Z., Heinol, H.G., Olk, J., Klein, R., Buxbaum, B., Fischer, H., Schulte, J.: New electro-optical mixing and correlating sensor: facilities and applications of the photonic mixer device (PMD). In: Proc. SPIE, vol. 3100 (1997)

45.Shirai, Y.: Recognition of polyhedrons with a range finder. Pattern Recognit. 4, 243–250 (1972)

46.Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: CVPR (2011)

47.Smith, A.M.: Alhacen’s theory of visual perception: a critical edition, with English translation and commentary, of the first three books of Alhacen’s de aspectibus, the medieval Latin version of Ibn al-Haytham’s Kitab al-Manazir. Trans. Am. Philos. Soc. 91 (2001)

48.Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. Comput. Graph. Forum 28(5), 1383–1392 (2009)

49.Szeliski, R.: Computer Vision, Algorithms and Applications. Springer, Berlin (2010)

50.Tanimoto, S., Pavlidis, T.: A hierarchal data structure for picture processing. Comput. Graph. Image Process. 4, 104–113 (1975)

51. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment— a modern synthesis. In: Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, ICCV’99, pp. 298–372. Springer, London (2000). http://portal.acm.org/citation.cfm?id=646271.685629

52.Trucco, E., Verri, A.: Introductory Techniques for 3-D Computer Vision. Prentice Hall, New York (1998)

53.Tsai, R.Y.: A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 3(4), 323–344 (1987)

54.Wheatstone, C.: Contributions to the physiology of vision. Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision. In: Philosophical Transactions of the Royal Society of London, pp. 371–394 (1838)

55.Witkin, A.P.: Scale-space filtering. In: Proceedings of the Eighth International Joint Conference on Artificial Intelligence, vol. 2, pp. 1019–1022. Morgan Kaufmann, San Francisco (1983). http://portal.acm.org/citation.cfm?id=1623516.1623607

56.Yang, R., Pollefeys, M.: A versatile stereo implementation on commodity graphics hardware. Real-Time Imaging 11, 7–18 (2005)

57.Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)