The extension of visual content to 3D as well as dynamically capturing scenes in 3D to generate an image on a remote site in real time has long been considered merely a part of science fiction. Today they are a reality, collectively referred to by terms such as 3D television (3DTV), free viewpoint TV (FTV) and, more generally, 3D video. This new type of image creates the illusion of a real environment, resulting from continually improving efforts in research and development over a number of years.
Numerous experts believe that 3D represents the future of media, such as television and the Internet, and will in turn improve the quality of visual experiences for the end user. The whole chain of content production must be reconsidered, beginning with recording techniques, since those designed specifically for 3D are far more numerous and varied than those used normally in conventional 2D context. The same can also be said of other aspects, such as, for example:
– the description and representation of scenes according to more or less informative structures, ranging from multiview or multiview-plus-depth videos to 3D digital reconstructed models;
– 3D reconstruction which extracts 3D models in various forms from videos acquired from multiple viewpoints, such as static or animated meshes;
– the compression of representations of scenes created by capture (stereoscopic or multiview videos) or reconstruction (3D models);
– 3D display, with or without adaptation/enhancement of content and/or intermediate view synthesis.
The democratization of these technologies needs specifically designed display devices. Stereoscopic or autostereoscopic screens show a heavy tendency toward this while their use for displaying 3D content today still poses a number of problems, showing that all these techniques must yet be perfected to avoid being rejected by the end user due to reasons of poor quality and/or eyestrain.
3D videos therefore cover a multitude of aspects, collectively linking a series of recorded videos to full depth 3D visualizations, potentially using estimations of depth in video sources. The developments examined here are therefore based on methods and tools from highly varied fields, such as applied mathematics, computer imaging, computer graphics, virtual reality, signal processing as well as psychophysics and the psychology of human vision.
In this highly multidisciplinary context, the objective of this book focused on 3D video is twofold since it aims, in addition to summarizing current information about the subject, to provide:
– for students: a solid base enabling readers to carry out activities relating to this topic and to learn the underlying concepts overall;
– for researchers: as complete a reference for this subject as possible which precisely indicates current research and understanding in this field as well as future trends and perspectives.
Its organization into four parts is due to a desire to cover all phases of 3D video by bringing together formal presentations of theoretical tools and developments of more technical or technological aspects. It should be noted that all figures are also available in color at http://www.iste.co.uk/lucas/3D.zip.
The first part of this book runs through the basics of 3D video and the recording of its characterizing multiview videos. This begins with, in Chapter 1, the different fundamental aspects of this technology. Historical and mathematical aspects relating to 3D computer vision and physiology of human vision are thus presented. Chapters 2 and 3 look at technological and methodological problems in relation to capturing images, more specifically in Chapter 3 within a multiview context that characterizes 3D video. The specification of geometric elements relating to the recording and display of 3D media is then examined in Chapter 4. Chapter 5 concludes Part 1 of this book, focusing on the problems of geometric and colorimetric camera calibration.
Part 2 focuses on the description and reconstruction of 3D scenes. Chapters 6 and 7 analyze the problems of local feature detection, stereo-matching and stereo-correlation through dense depth estimation. Chapter 8 then presents different scene reconstruction methods, notably using silhouettes, providing an overview of the technical principles used to structure previously constructed 3D information. Finally, Chapter 9 provides an outline of intermediate view synthesis using images with depth information. Direct and inverse projection approaches are examined alongside a description of uncovered areas filling methods.
In Part 3, the field of compression and transmission norms for 3D content is covered. Chapter 10 in particular introduces 3D video formats as well as specific techniques for stereoscopic and multiview stream coding. In Chapter 11, the multiresolution compression of meshes and mesh sequences is examined in terms of standardization and visual perception. Chapter 12 then focuses on depth video coding while Chapter 13 presents the problem of protecting stereoscopic videos by watermarking the 3D stream.
In Part 4, aspects relating to 3D rendering and display are covered. This begins, in Chapter 14 with the implementation and use of autostereoscopy and is followed, in Chapter 15, by techniques relating to augmented reality. In Chapter 16, psychophysical effects relating to problems of eyestrain and visual discomfort are discussed with a specific examination of flaws in 3D content and technologies which generate these unusual stimuli. Chapter 17 focuses on the delicate problem of 2D-to-3D conversion which remains in between technology and the arts where human intervention remains indispensable.
The practical implementation of all these technologies and their applications are considered in Part 5, the final part of this book. Aspects relating to data mining (Chapter 18), high dynamic range videos (Chapter 19), biomedical visualization (chapter 20) and sport scene reconstruction (Chapter 21) are covered. This final part is concluded by an overview of experiments in live recording and transmitting of 3D stereoscopic videos (Chapter 22).