5

Acquisition

Any digital intermediate process begins with the acquisition of media (sometimes referred to as the “ingest” of media). In many cases, the digital intermediate process is the final stage of a production, following shooting and editing. In these cases, the type of media required can be worked out before the digital intermediate process begins. Typically, there is footage for the final cut and there can also be audio and textual elements, such as titles and dialog. The focus of the majority of this book will be on the visual component of the digital intermediate, because the audio is almost always mastered independently and added separately at the end.

5.1 Crunching Numbers

Computers don’t understand analog information (which includes printed images, reels of film, people talking, and so on). This means you can’t just show a film, edited or otherwise, to a computer system and get it to create a color-graded digital master (not yet anyway).

It is important to remember throughout the intermediate process that from the moment you begin, all you’re doing is crunching numbers. Even though a skilled colorist can turn a blue sweater red, or selectively add or remove colors throughout a scene; to the computer systems involved in the process all that is happening is that one long number is converted into another long number. The system works because of the innovative software involved and the experienced operators who can predict how the system is likely to react to changes and can understand the limitations and shortcuts.

Computers can do very complicated mathematics very, very quickly compared to humans. Computers are able to take a digital image with the same amount of detail as a piece of film and almost instantly produce a perfect copy. However, all they are doing is performing some repetitive calculations at very high speed. This strength can also be a weakness. Because the computer has no knowledge of what the image contains and can associate no meaning to it, it cannot tell which parts are important. For this reason, defects such as noise, dust, and scratches are treated with the same care as desirable parts of an image. For this reason, you must always assume the golden rule: what you get out of a digital system will only ever be as good as what you put into it.

Actually, that rule isn’t strictly true. Some methods for automated processes can perform feature analysis on images and sequences of images, can estimate which parts of an image or sequence of images are important and use that information to enhance the images—“estimate” being the keyword. Also, at every stage of the digital intermediate process are operators who are able to make subjective decisions about the processing and quality of the images, ensuring the highest possible standard throughout.

5.2 Digital Image Quality

The fundamental advantage that digital media has over analog materials is that digital information can be transferred across large distances with no loss of quality. A digital image looks exactly the same whether you’re accessing a digital camera directly, or from the other side of the world, or accessing the image via the Internet, bounced off of satellites, or through some other transfer method. As long as the data remains unaltered, it’s as good as the original. Sadly, the same cannot be said for video and film, which are both analog formats and therefore subject to many forms of degradation, the most significant being “generation loss.”

5.2.1 Generation Loss

Every time you copy an analog source, such as a video tape or a reel of film, it degrades quality. In fact, every time you even view certain analog sources (including both film and video), it suffers a loss of quality. This is because most analog-viewing devices are mechanical and have moving parts that can damage the source. Viewing (or even handling) film prints or negatives can cause scratches or dust and other damage to them.¹ Video tapes risk decay or becoming demagnetized every time they’re moved across a video head.

Each copy of an analog source introduces additional, lasting damage to the copy. A copy of a copy retains all the errors so far and then adds more errors. Errors are caused by effects such as noise (in the case of video) and chemical stains (in the case of film), among others.² Each successive copy is called a “generation.” A fifth-generation copy is therefore typically of lower quality than a second-generation copy. To maximize the quality, it’s preferable to work from original material whenever possible.

5.2.2 What is Quality?

This book includes discussions of various factors that influence the somewhat esoteric criteria of “quality.” As we know, quality, especially picture quality, can be a very subjective thing. For example, a well-lit scene recorded on DVCAM video might be considered of superior quality to a poorly lit scene filmed on 35mm negative. There are ways to measure image quality objectively though; if you assume all other things (such as lighting, set design, and so on) to be equal, then “quality” can be defined as “how closely an image matches the original content,” which is how the term is used throughout this book. There is another issue at stake though. What if you don’t want the final project to closely resemble the original? After all, part of the “magic” of filmmaking is in creating imagery that looks far removed from or even better than real life. However, even in those situations, you want to produce images that are of high quality—with as much detail as possible—so that you selectively control what stays and what doesn’t later on.

In the digital intermediate pipeline, the aim is to maintain the highest possible quality throughout, with the material prepared for supervised sessions with the filmmakers, who then make decisions about how to affect the images and control the subjective level of quality. For example, if you want a shot to look blurry in the final production, it can be argued that it doesn’t matter if the scene is transferred at low quality, making it visibly blurry. However, in this case, there is no way to “unblur” the image later on (even digital sharpening techniques won’t create as good an image as the original footage), and the level of blurriness can’t be adjusted interactively, in context with the scene. Chapter 14 contains information about using digital processes to help define the aesthetics and style of a production.

5.2.3 Resolving Power

Resolving power is one objective method for measuring the quality of an image. It is found by measuring how much detail is crammed into a given area. The simplest way to measure is to capture an image of a chart of lines that alternate between black and white and get increasingly thinner. When the lines are too thin for the recorder, they are recorded as gray. The point where this occurs is used to calculate the “resolution” of the recorder.

With film, the resolution depends upon the average grain size, and the resolution of video tape depends upon the properties of the type of signal received. Other factors can also play a part in influencing the resolving power, such as the optical system (i.e., lenses). In reality, the derived resolving power is actually a measurement of the entire imaging system.³

Ultimately though, to get the most relevant result, the resolving power, and thus the maximum resolution of any system, should be measured by looking at the final output.

5.2.4 Spread Function

Further evaluation of any optical system can be made by determining its spread function. The spread function is essentially a measurement of how different a single point of light (or a line or a hard edge) appears when recorded and demonstrates many different characteristics of degradation in a system.

5.2.5 Modulation Transfer Function

An even more useful result is the modulation transfer function (MTF). In simple terms, MTF is a measurement of the response (or accuracy) of any system across varying frequencies (which amounts to the level of detail). You would expect that the accuracy of reproduction for most systems would be directly proportional to the level of detail, so that, for example, an image of large squares would be more faithfully reproduced than an image of very thin lines. However in practice this often isn’t the case. MTF measurements and graphs serve to illustrate exactly how the systems respond to different frequencies, thus highlighting potential problems, as well as serving as a useful method for making objective comparisons of systems and components.

One of the reasons the MTF measurement is useful is because certain systems respond better than others under certain conditions. For example, some types of photographic film exhibit greater granularity in the mid-tone regions than in shadow or highlight areas. An MTF can provide an objective measurement of these types of effects.

MTFs are measured by imaging a special chart (which displays a series of sine wave patterns) and performing various measurements and calculations on it.

5.2.6 Signal-to-Noise Ratios

Another useful measurement of the quality of a system is its signalto-noise ratio. This is calculated by dividing the signal strength of a system by the inherent noise. Every system, or signal (such as a video signal) has some degree of noise inherent, which comes from a variety of sources. Every time the signal is amplified, the noise is amplified along with it. A higher signal-to-noise ratio indicates a higher quality system than a lower ratio, because it means the effects of noise are less pronounced.

5.2.7 Detective Quantum Efficiency

Modulation transfer functions don’t take into account the effects of noise in a system, and signal-to-noise ratios don’t tell us anything about the accuracy of a system’s response to light (i.e., its contrast performance). A method that combines these two measurements is the detective quantum efficiency (DQE). A low signal-to-noise ratio, combined with high-contrast performance results in a high DQE (and a superior image). As with the other methods, a system’s DQE is typically measured across a range of frequencies input (such as by using a standard resolution chart), and the results allow direct comparison between different systems.

5.2.8 Compression Ratios

Certain digital and analog compression methods (such as video compression or JPEG digital compression) discard a set amount of information (and by extension, reduce quality by a proportional amount). While some of these methods are designed to discard information that is imperceptible to the human eye, the image is still considered degraded. Digital systems tend to measure compression as a percentage of the available information, so that a lower percentage is more degraded than a higher compression. The same is also true of video compression, which describes compression ratios in terms of YIQ or YUV levels. Video compression is discussed in Chapter 2 and digital compression is discussed in Chapter 4.

5.2.9 The Eye of the Beholder

With all this discussion about image degradation and quality, it’s important to remember that the human eye itself isn’t a perfect optical system. Many of the shortcuts used by video-and digital-imaging devices are based upon inaccuracies in the human eye. For example, the human eye is much more sensitive to red and green light than blue.⁴ Also, due to the large number of rod (luminance-sensitive) cells compared to cone (chroma-sensitive) cells, humans can more readily detect changes in luminance than in color. For this reason, many devices (in particular, video systems) are optimized for these parameters. Because of this, it’s important to remember that even if some degradation occurs, it may be within the parameters of degradation experienced by the human eye anyway and is thus undetectable to end users.

There are two main reasons why you might want to keep information that is imperceptible to the human eye. The first is machine vision (which is less relevant to the entertainment industry) and the second is image manipulation, such as digital color grading (which is covered later in Chapter 8). Ultimately, it’s desirable that a pipeline not suffer any degradation at all, whether or not it’s perceptible. In fact, selective information loss can be more of a problem than overall degradation—first, because it means an image may not look as degraded as it is, and second, because it degrades the affected parts of the image more (which then means that those parts of the image don’t respond well to manipulation).

5.3 Media Acquisition

There are two distinct ways to acquire media for inclusion within a digital intermediate: by electronic transfer or by analog-to-digital conversion. These methods vary in efficiency, speed, and quality.

If the source material is already in a digital form, it simply needs to be transferred from the original storage device into the storage devices used for the digital intermediate pipeline. If the material exists in some other form, then it must first be digitized (or converted to a digital format).

5.4 Data Transfer

A data transfer involves taking a set of images that are already in digital form (or taking some other data such as audio tracks or timecode information) and copying or moving the associated files from one device to another. For example, if you’ve shot something on a DV (digital video) camera, you can just pop the tape in a DV tape deck connected to a digital-editing system via a firewire cable, and all the data can be copied digitally from the tape, ready for editing.⁵

This process becomes complicated because there are many different ways to store, or encode digital information. In the same way that different languages can be used to describe the same things, different images can be recorded in a variety of different ways. It’s often necessary to perform additional data management, such as file format or color-space conversion during or after transfer. The transfer method might use a transcoding process, which effectively reinterprets the images from scratch, and thereby subjects them to further degradation.

Data conversion notwithstanding, digital transmission allows data, such as images or video streams, to be copied or moved over long distances, with no loss in quality at all. For this reason, data transfer is preferable to every other acquisition method. However, everything is ultimately sourced from analog media, so the ideal scenario is to keep everything digital beginning with the step of analog-to-digital conversion and continuing onward. Whether the analog-to-digital conversion happens in the camera, as in the case of digital cameras, or after shooting, as when scanning film negatives, everything should be digital from that point onward.

An interesting factor affecting data transfer is speed. Video is always copied in real time (i.e., it takes one minute to copy one minute of footage, and so on). Video also has an advantage because it can be copied to many tapes at the same time (to as many recorders as available) through a process of signal amplification. Film can be duplicated at a rate of hundreds of feet per minute.

Data, however, is transferred at varying speeds. The transfer speed completely depends on a number of factors, such as bandwidth, network traffic, type of cabling, speed of the host computer, and so on. In some cases, it’s possible that a digital video sequence can be copied or moved faster than real time, and in others, it may be much slower than real time. It’s also possible that a transfer may begin rapidly and then slow to a crawl. The important thing is that each copy is identical to the original. Further, a single piece of data can be accessed by multiple devices at the same time, which is a (theoretically) more efficient way of working. Given a reel of film or a video tape, only one person can modify the contents at a time. But the digital equivalent allows each frame to be treated and modified independently, and in some highly specialized applications, even a single frame can be modified by different people at the same time. There are some pitfalls though. Certain applications “lock” files so that they cannot be accessed by others, to avoid reducing the speed of throughput. Locking files is especially common on softwarebased grading systems, which prevent frames that are in use from being copied or modified from outside the grading system. Although file locking is sensible, it can cause problems (e.g., when data transfers overrun their allotted time, as when the process of copying a number of frames overnight hasn’t been completed by the time the colorist starts work the next day), or when the system contains software errors, requiring it to be rebooted to “unlock” certain files.

Some digital media formats (e.g., DV or Internet streaming) are specified to work at a preset speed. In the case of DV video, for example, transfer from a DV tape to a capture system always occurs in real time. If the transfer speed drops during the capture process, the process might halt, normally reporting “dropped frames” or some other error, and have to be restarted. Internet streaming is designed to work at a given transfer speed (normally significantly lower than the maximum transfer speed) and will continue to transfer video at speeds above that rate—although the end user may experience viewing problems (e.g., slow downs or dropped frames) if the average transfer speed falls below the target rate.

A computer accessing its own files uses data-transfer operations. For example, displaying a digital image on a screen typically involves several processes: the data is transferred from the source storage device to the computer system through a number of cables and junctions, into the computer’s RAM. From there, it’s transferred to the computer’s graphics subsystem, which in turn transmits the data to the monitor through another cable. This description somewhat oversimplifies what exactly happens; in practice, even more factors play a part—for instance, displaying an image from a network location rather than a local disk drive necessitates even more transfers, through more cables and numerous network routers.

Data-transfer operations are normally used only for computergenerated or digital video productions, because most footage is generated using analog means, such as shooting on film, which is unfortunate because it’s the most convenient method for acquiring images. As digital filmmaking matures and gains popularity, the data-transfer method of acquisition will become more common. Until that time, the majority of footage will be fed into the digital intermediate pipeline using digitization methods.

5.5 Digitization

Digitization is the first step in converting an analog source to a digital format. It is currently the most common method for image acquisition in filmmaking. Almost everything is sourced from an analog format at some stage, with the notable exception of computer-generated imagery.⁶

To produce a digital version of anything analog, an analog-to-digital conversion must be made. Without question, this event ultimately has the most impact on quality in any digital pipeline (coupled, of course, with whatever was shot in the first place). From this moment on, you can only reduce quality—purposefully in many cases, but irreversibly nonetheless. How you go from an analog source to a digital source has a bearing on every subsequent stage, and therefore this process must be planned carefully in advance based upon the requirements of the film.

5.5.1 Sampling

The process of digitization is a statistical reduction of information. Analog sources effectively have an unlimited level of detail. The closer you examine any analog source, the more detail you find. Some of that is “useful” detail, providing more of the image content, and some of the detail is just a factor of defects, such as noise. The first step in digitization is to decide how much of this detail you actually need. It isn’t possible to capture absolutely everything, but in most cases, it isn’t necessary. In the same way that the attitudes of an entire population can be determined by interviewing a small, carefully selected “sample” of individuals, an analog source can be sampled to build a complete picture. An entire branch of mathematics, sampling theory, is devoted to methods for doing this, but the basic idea is that if you make enough samples of an image at regular intervals, you’ll be able to re-create the original faithfully.⁷ This means that you start with an analog source, which has an infinite level of detail, and you split it into parts.

Imagine, for instance, that you’re painting a scene that includes a tree. Let’s say you use a large paintbrush, one that can cover the entire canvas in a few strokes, and a pot of black paint. After maybe four strokes, you might have a vague impression of the tree. Now let’s say you use a smaller brush (or a bigger canvas). It takes a lot more strokes to paint a picture of the tree, but now you can paint more of its subtle details. But it’s still not an exact image of the tree. Let’s now assume that you start the painting again, but this time with a few different colors of paint. You can even mix these colors to create new colors. This provides an even more accurate image of the tree.

What you’ve done, in a very loose sense, is digitized the tree onto the canvas. Even with a very thin brush, you can’t replicate all the detail of the scene. The more carefully you inspect the scene, the more details become revealed. But at some point, you decide that the level of detail is close enough to make no visible difference. For example, make the canvas as big as a cinema screen, and the level of detail high enough that your eyes can’t actually perceive all the detail from a few feet away. Then say that you don’t want to be able to perceive specific points where one color of paint begins and another ends. Maybe now you have something that approximates a projected photograph.

Anything analog has to be digitized somehow so it can be replicated in another form. Take photographic film, for instance. The detail level of a scene imaged on a photograph is determined by the grain size (among other things) of the film. You can think of taking a photograph as an analog-to-analog process, which digitizes a scene at the level of the grain size.⁸

As another example, consider the way that the moving picture actually works—by projecting a series of stills at a rate of around 24 frames a second. What has happened is that time itself has been digitized by this process, breaking down a continuous (analog) motion into discrete (digital) parts. But when a moving picture is played back, we perceive the motion as smooth, because it occurs at a slightly higher rate than our eyes can detect.

You can use many different ways to digitize an image, but the most common method is to break down an image into regularly arranged, repeating squares. There are many reasons for doing it this way, but the main reason is that it allows the data collected to be identical to how digital images are stored and displayed, resulting in no necessary further degradation of the image.

In the simplest case, a single point of a source is sampled, which then corresponds to a single pixel of the digital image. This process is repeated at regularly spaced intervals, horizontally and vertically, until the entire image is digitized.

5.5.2 Interpolation

Digital images can change resolution as required, through a process called “resampling.”⁹ For example, a 200 × 200 image can be resampled to a 100 × 100 image or to a 400 × 400 image. Increasing the number of pixels is referred to as “upsampling” (or “uprez’ing”), while decreasing the pixel count is “downsampling” (or “downrez’ing”). Resampling is used for a variety of reasons, often so that different footage is processed at the same time (so that video footage can be recorded onto film, it must first be upsampled to match the film image resolution). However, increasing the pixel resolution in this way won’t increase the image’s level of quality (though decreasing the resolution will cause a reduction of quality). This is because you can never get additional quality from a digital image that wasn’t captured in the first place.

To resample an image, a calculation must be made to work out what the new pixels look like. This process is known as “interpolation.” There are different interpolation methods (or algorithms) for different situations, which are explored in Chapter 10. Interpolators merely estimate how additional detail would look (in the case of upsampling) based upon existing pixels. When downsampling, estimation is less an issue, because a reduction in quality doesn’t have to estimate detail that doesn’t exist. However, upsampling should be avoided whenever possible.

5.5.3 Color Space

So far, we’ve mainly looked at the RGB model of digital images, because the most common types of digital image are of this model and are ideally suited to being viewed on an RGB display device, such as a computer monitor. However, there are many different ways of using digital images, other than displaying them on a monitor. Digital images can be printed on paper, put on video, recorded by lasers onto photographic film, or even just analyzed by machines without ever being looked at.

Each of these different methods understands color in a different way. For example, many colors that can be seen on projected film can’t be displayed on video (and vice versa). The range of colors that a system is able to reproduce is known as its “color space” (or “gamut”). Most of the time, the color spaces of different systems overlap, and the systems can show the same color. So, for example, an image displayed on a (properly calibrated) monitor will look exactly like the color print. However, where areas of color don’t match, the colors are said to be “out of gamut” and may look wrong.

Even the human eye has its own gamut. For example, some video cameras can detect infrared or ultraviolet, which is outside of the gamut of the human eye. However, the color space of the human eye is much larger than most color-reproduction systems, which is why we can easily detect differences between colors viewed in different mediums.

Digital images are often optimized for a specific color space. This means that the images may be interpreted a certain way, depending on their application. For example, the common three-channel RGB color image discussed earlier, aims to be suitable for viewing on most computer monitors. Printing houses may adopt the four-channel CYMK color space, which is interpreted in a similar way to the cyan, yellow, magenta, and black inks used for printing on paper. Images for photographic film typically use a three-channel RGB color space but by using a logarithmic scale. This is done for efficiency because film responds to light differently at the extremes of color than in the mid-tones. This topic is covered in Chapter 8.

5.5.4 Nonlinear Color Space

Certain materials, including photographic film, don’t respond to light in a linear fashion. That is, adding twice as much light to a photographic material won’t necessarily result in an image that is twice as bright. For this reason, digital images can be stored in a nonlinear fashion. With linear-color-space images, pixels are encoded with brightness values on each channel. Nonlinear color spaces work this way too, but the difference is that the values are mapped onto another scale, so that values of 0, 1, 2, 3, etc. might be mapped to brightness values of 0, 1, 4, 9, respectively (in this case, an exponential color space is used) instead. This method makes it possible for very high values and very low values to be stored together in the same image, resulting in more efficient use of disk space.

Where problems occur with nonlinear-color-space images is in trying to convert data between different spaces, because it can be difficult to correlate colors from one color space to another. Color-space conversion is covered further in Chapter 8.

5.5.5 Gamma

Although RGB provides a suitable model for displaying images on a number of devices, such as televisions and computer monitors, there is no direct correlation between the brightness of a pixel of an RGB digital image and the corresponding brightness of a point on a monitor. Part of the problem is that monitors have a luminance range that is nonlinear, but another problem is that there is so much variance between monitors.

The first problem is resolved by applying gamma correction to the image. In gamma-correcting an image, the luma signal sent to the monitor for display is adjusted so that it’s represented on a nonlinear scale, compensating for the response of the monitor (so that the image looks as it should, at least in theory). Gamma is a specific scale that combines both contrast and brightness. The amount of gamma to be applied is variable, with a value of 1.0 making no difference to the input and output of the pixel values, and higher and lower values increasing or decreasing the gamma correction, respectively. This is complicated somewhat by the use of different standards for different applications. For example, older Macintosh computers use a gamma of 1.8, whereas Windows computers typically use a value of 2.2.

Because of the variance between different monitors (and lighting conditions), the gamma correction must be adjusted for each monitor for accurate results. The most common way of doing this is using a gamma-calibration system. Using such a system, a series of readings of the response of the viewing device to different images and luma values is made, and the optimum gamma correction is determined. Many digital images may also have additional gamma information encoded into them, so that they can be displayed more accurately on other calibrated systems (although such embedded information is useful only if the originator of the material was working with a calibrated system). During acquisition, gamma correction may be applied automatically to certain types of media, such as video or digital images from elsewhere, and can affect the images’ colors. Many digital intermediate pipelines try to avoid altering color content in any way prior to the color-grading stage because such alterations may degrade the images. Color-grading and calibration are covered further in Chapter 8.

5.5.6 Dynamic Range

The intensity range of an image is the ratio of its brightest point to the darkest point. The density range of a photographic image is the ratio of the highest density on the film (the D-max) to the lowest (the D-min). Either of these terms can be taken as the dynamic range of an image. In terms of digital image formats, the dynamic range of the image can be derived from the bit depth: an 8-bit image, having 256 values per pixel, has a dynamic range of 256:1. A (linear) 10-bit image has a dynamic range of 1024:1. The dynamic range of the human eye is roughly 10,000:1, whereas most CRT displays are approximately 100:1. Televisions are around 30:1 or 40:1. Where this becomes important is in trying to capture the available intensity range from another source (either analog or digital). If the dynamic range of the capture device (such as a film scanner or digital camera) is lower than the dynamic range of the scene being imaged, some tonal information will be lost. If the dynamic range of the image format being used is lower than the information being captured, some information will be lost. Similarly, using a file format with a high dynamic range doesn’t necessarily mean that the image itself will have a high dynamic range.

There’s a subtle difference between dynamic range and color precision. Color precision is a purely mathematical concept, determined by the bit depth of an image. A 10-bit file has greater precision (and is therefore less prone to color artifacts) than an 8-bit file. On the other hand, a nonlinear 8-bit image might have a greater dynamic range than a linear 10-bit image, meaning it could reconstruct a greater range of luminance but at a lower degree of accuracy.

5.5.7 Acquisition for Digital Intermediates

The aim of acquisition for the digital intermediate pipeline is to obtain, whenever possible, digital images whose quality isn’t compromised in any way. Current technology dictates a trade-off between speed and quality, particularly in analog-to-digital conversions. However, most digital intermediate facilities believe that quality must take precedence over speed at this stage, especially when dealing with the high-quality demands of cinema or (to a slightly lesser extent) high-definition video formats. The quality level set at the stage of image acquisition will have a profound effect on the quality through the pipeline, regardless of the target quality of the final product. This is true of many mediums—for instance, a VHS video originated from a HD video source will look superior to a VHS video originated from a VHS source, even though the content might be the same. There are other considerations—for example, the quality level shouldn’t be so high that it impacts the performance of the playback system. Most facilities require that all screenings for the production team, or any interactive sessions (such as color grading) be done at the maximum level of quality in real time.

5.6 Acquisition from Video Sources

Acquisition of digital images from video sources is actually a fairly straightforward process. All that is required is a video playback device, an analog-to-digital converter, and a data storage device. The quality of the conversion depends entirely upon the analog-to-digital converter (ADC).

Video signals are formatted in a way that translates to digital images very easily. Depending upon the video format, the necessary resolution can be derived that corresponds exactly to the number of lines in the image. In this case, no additional benefit is derived from digitizing the video to a higher resolution (because there will be no additional information to record, unlike for photographic sources). Both YIQ- and YUV-encoded colors can be converted to RGBencoded ones, although some RGB color spaces have different gamuts than the video ones.¹⁰ A higher bit depth may provide a quality advantage in terms of color precision, because a wider variety of colors can be sampled, although this will inevitably depend on the precision of the source video. However, many video formats inherently use color compression to reduce the transmission overhead, and so the benefit may be slight. In most cases, it’s unlikely that any benefit will be gained from using more than 10 bits per channel RGB images.

In any case, provided the file format is appropriate, any reduction of quality will largely be due to noise added by the ADC and its subsystems. With most ADCs, the noise introduced will be negligible; otherwise, a noise-reduction technique, such as image averaging, can be used (see Chapter 9 for more on this technique). A table of digital resolutions for various video formats is included in the Appendix.

A simple, effective system for digitizing video sources is a dedicated computer workstation with a video capture card. The source deck (or video matrix, in facilities with a large number of video decks) connects to the capture card, and software within the computer saves the incoming video as a series of still images onto a local disk or network. Video is normally transferred in real time, meaning it takes one hour to digitize an hour’s worth of video.

5.6.1 Digital Video

With digital video formats (such as DV, DVCAM, and HD video), the video is already stored in a digital format. Therefore, acquiring video of this type requires no analog-to-digital conversion and is usually facilitated by directly transferring the data from the tape deck to the host computer (such as with a firewire connection). When transferring digital video this way; there will be a much lower degree of quality loss compared to transferring video using analog means.

The process isn’t exactly the same as copying a set of digital images from one disk to another, however. Digital video is always transferred in real time, and thus, any read errors can’t be corrected by attempting to re-read the data from the tape, which is why some errors may be introduced during capture. However, this disadvantage is the only real weakness of digital video capture, and in general, very highquality transfers can be achieved. In the event that the errors are significant, problems may be corrected by redigitizing the tape (assuming the tape itself is not damaged).

5.6.2 Timecode

Video timecode is often used to determine the parts of a video tape that have to be captured. For example, a video tape may contain 90 minutes of footage, but only 2 minutes are actually needed. In this case, the capture system might allow for the capture of video starting at a timecode of 01:00:00:00 (the “in point”) and ending at 01:02:00:00 (the “out point”).

To enable video footage to be referenced correctly later on, it’s often desirable to capture this timecode information along with the image content for every frame. Capturing this information is usually done by using a simple cable between the capture system and the VCR, but the information somehow must be tied with the image data. Certain digital file formats, such as MXF, allow timecode to be embedded within the file, but for less-sophisticated formats, a common method is to convert the timecode to a frame number, as covered in Chapter 6. Even for systems that have no capability for capturing timecode, the frame-numbering method can be used to retrospectively attach timecode to captured footage by renumbering the data.

In some cases, it may be necessary to include additional information, such as the tape reel number or the date of shooting.

5.6.3 Frames Versus Fields

One stumbling block in video acquisition is that many standard definition video formats are interlaced, which means that each frame of video actually consists of two interlaced fields. Typically, each field is recorded with a slight delay in between. This means that a frame of a fast-moving object might be at a slightly different position on each field of a frame, and that visually, the object will appear to be shifted horizontally on alternate lines (which can look similar to aliasing artifacts). If the final output is a video format anyway, this doesn’t necessarily matter (because it would be split back into fields anyway). But often, especially when mastering to film or other formats, it’s desirable to remove the interlacing motion artifacts on each frame. The methods for doing this are covered in Chapter 9.

5.7 Acquisition from Photographic Sources

At present, an undeveloped piece of film can’t be digitized. For this reason, the initial step in the digital intermediate chain is to develop the photographic material. This is an inherently chemical process and should be done as soon after exposure as possible, to prevent the risk of “fogging” or otherwise spoiling the film.

Once the film has been processed, the bulk of image acquisition involves using specialized film scanners to digitize reels of film, creating a series of stills that correspond to each frame on a reel. Depending upon the particular digital intermediate pipeline you’re using, you may wish to scan everything that has been shot as soon as the negative is developed, or (if the film has already been completed in a chemical lab environment) to scan a theatrical release print (e.g., for transfer to video or digital master). A common method at the moment is to scan “master negative reels,” which are reels that comprise only the selected takes that are to be used in the final edit.

5.7.1 Film Scanners

Regardless of the paradigm, the process of digitizing film is the same. The basic methodology of a scanning process is that a reel of film is loaded onto the film scanner (in a way similar to using a projector or an ultrasonic cleaner).¹¹ The film is scanned by illuminating the surface, usually with a very intense light source (such as a metal halide lamp). The light is then measured in terms of transmittance (i.e., how much light passes through to the other side), typically a line at a time by a trilinear CCD array (a single line of photosensitive CCD elements for each channel of red, green, and blue light) or by some other method, depending upon the scanner configuration. Each point measured, or sampled, along the line corresponds to a single pixel in the image, with separate measurements for the red, green, and blue light. Motors in the scanner advance the film to the next line, and the process is repeated. Finally, when the entire frame has been scanned, the image is saved, and the scanner advances to the next frame. It can take anywhere between 4 and 320 seconds to scan a foot (i.e., 16 frames) of film, depending upon the resolution and type of scanner.¹² It is interesting to note that the maximum color precision (or bit depth) of the scans doesn’t affect the scanning time at all but is instead dependent upon the sensitivity of the scanner.

Most high-end film scanners are made to exceptionally high optical standards and don’t introduce any significant artifacts, such as noise. In addition, scanners don’t capture motion; they merely record a sequence of stills. The distinction is important because it means that they don’t introduce (or remove) any additional temporal artifacts (such as motion blur) that weren’t originally recorded. However, spatial and chromatic artifacts may still be introduced as a result of the analog-to-digital conversion. If the sensitivity of the scanner to color or if the scanned pixel resolution is too low for the type of film, artifacts such as banding or aliasing may occur.

With current systems, the color information contained on a frame of film is higher than can be captured by most scanners. Most film scanners capture 10-bit (linear or nonlinear) Cineon or DPX files (see the sidebar, “Cineon/DPX Images,” later in this chapter). Ideally, nonlinear log (or logarithmic) files are preferable because they provide a better representation of film color space. If the color-correction, conforming, and finishing systems aren’t designed to handle log files, it may be better to scan to linear color space files, avoiding problems later on. Even with log files, it isn’t possible to capture the entire dynamic range of film with most image formats.

Either way, the scanner process should be set up to maximize the contrast range of the images captured, so that the brightest point on the film (i.e., the white point) is set to a value just under the maximum brightness value of the file, and the darkest point on the image (i.e., the black point) is just above a value of pure black. Note that values that are outside the picture area (and therefore irrelevant) can safely be ignored. Maximizing the images’ color ranges ensures that no clipping or crushing of colors will occur and affords the maximum possible flexibility and quality levels later on in the process. Most scanners have this capability, and settings can usually be made on a shot-by-shot basis.

The usual work flow for the scanner operator, having loaded the film onto the scanner, is to create black-and-white point settings for each shot on the film, while playing through the reel at high speed. (These point settings are typically stored as a small “grading list” that can then be saved if needed later—e.g., for rescans. Most scanners are able to display the film on a monitor at a faster rate than the time required to digitize the film to data files.) While it’s possible to set these levels visually (in a properly calibrated environment and with a highly experienced operator, at least), it’s preferable to use more objective means to determine these settings. The most common method is to use a vectorscope, which can measure the intensity of the video signal generated by the scanner and match the peak white level to the white point of the output image (and do the same for the black point). Some scanners provide this information automatically, by taking density readings across an image.

Once the levels have been set, the scanner returns to the start of the reel and digitizes each frame, applying the point settings, typically generating a series of digital images to a local disk drive. With certain scanners, such as Filmlight’s Northlight scanner (www.filmlight.ltd.uk), shots don’t have to be set up this way. Instead, such scanners employ an “unattended scanning” paradigm, which means that once the film is put onto the scanner, the scanner identifies the base density of the film (either by matching stock presets from the key numbers or by taking a reading in between frames), calibrates itself, and digitizes the film without having to set up grading parameters for each shot. This capability is ideal because it guarantees that the film is captured quickly, with the maximum possible color range for the film stock. This system may fail, however, when scanning a reel made up of different film stocks spliced together.¹³

5.7.2 Scan Dimensions

Unlike video, there is no perfect translation between the dimensions of photographic film and digital images. Photographic grains are randomly shaped and randomly arranged, unlike the precisely structured video image. It is therefore reasonable to suggest thatscanned images can be any resolution. It can also be suggested that the ideal image resolution should be as high as possible, while remaining practical to work with (i.e., not taking too long to process, copy, or display). Other considerations include the possibility that the film scanner color-correction and conforming system may be limited to working with images of a specific or optimum size.

Of the several accepted standards, the most common is 2k and 4k resolution for 35mm negatives. This value refers to the number of pixels along the horizontal edge. A typical, full frame of a 35mm negative scans at 2048 pixels (the width) by 1556 pixels (the height), making it a 2k image (i.e., having approximately 2000 pixels along the width). A 4k image is 4096 by 3112 pixels in size (resulting in a file size that is four times larger for each frame). Different specifications apply to different formats (e.g., when you want to scan only the Academy area). These specifications are listed in the Appendix.

5.7.3 Nyquist Frequency

The Nyquist frequency (also called the “Shannon sampling frequency”) is a simple mathematical equation used to determine the amount of information required to successfully reproduce a signal (i.e., an analog source). It is used to determine the point where all the relevant information is captured (and so recording any more will have no benefit).

This equation—Nyquist frequency = 1 / (2 × sampling interval)—can be used when digitizing an image to determine the maximum useful resolution, using a measurement of the resolving power of the image. For example, for a typical 35mm negative, the resolving power is around 150 lines per millimeter.¹⁵ The Nyquist frequency, in this case, is equivalent to the resolving power (because no more information can be successfully recorded beyond this amount). Putting this value into the equation gives us a sampling interval of 0.0033, meaning that a pixel must be sampled every 0.0033mm (or that there must be 300 pixels for each millimeter of film).¹⁶ In practical terms, given that the height of a full frame of 35mm negative is 18.67mm, a completely accurate representation of a frame of 35mm film requires approximately 5600 lines of resolution. Assuming the resolving power of film is the same horizontally as it is vertically, our desired image size is approximately 5600 × 7500 pixels per frame, which is required to completely capture all of the detail in a frame of film. At 10 bits per channel, it would make each frame about 150MB.

Unfortunately, current technology simply cannot store or process files that large in a practical manner. However, this limit is theoretical; in reality the difference in quality between an image that large and a 4k image, for example, may be negligible. It is questionable whether that much detail really can be perceived, given factors such as film grain; and in any case, the ultimate question is whether an audience in a cinema is able to perceive any difference. The bottom line, at the moment at least, is to aim to try to capture as high a resolution as possible.

5.7.4 Punch Holes

Many scanner systems require that each reel of film includes a “punch hole” (or a marker frame at the beginning of each reel). The easiest way to make a punch hole is to literally punch out a hole in the negative at the beginning of the reel (before any useful images!). The reason for doing this is to provide an arbitrary sync point that will allow each frame of data to be matched back to a specific frame on the reel. In simple terms, each frame scanned is given a frame number, so that the punch hole is frame 0, and every frame after goes up by one. For example, the thirty-first frame after the punch hole is simply frame 31.

5.7.5 Splices

When scanning film, you may encounter a problem if the reel contains splices. When a film splice runs through certain scanners, it can cause the frame to “bounce,” meaning that the image shifts vertically. This problem is less likely to happen with pin-registered scanners. Even when using scanners that are not pin-registered, the problem of bounce can be corrected digitally, but doing so is usually an operator-controlled procedure (rather than an automatic one) and as such can add to the cost and duration of the project. Perception of bounce is much greater on monitors than on projected images; therefore, bouncing shots may not be noticeable on cinema releases.

Additionally, the materials used to make a splice may intrude into the image area, damaging the image and requiring its restoration. For all of these reasons, it is preferable for productions to minimize the number of splices made to film material intended for scanning.

5.7.6 Datacines

In addition to film scanners, telecines are used to convert film reels to other media (typically to video). Telecines, particularly older machines, may perform an analog-to-analog conversion rather than an analog-to-digital one and work in real time. Film scanners traditionally differ from telecines in a number of ways. An important advantage of using scanners is that they have the capability to output to a digital format. On the other hand, telecines traditionally enable color grading and the viewing of film material in real time. However, in recent years, the differences between the two have become more subtle. Some telecines, such as Thomson’s Spirit 4K (www.thomsongrassvalley.com), can directly output high-quality data and are referred to as “datacines,” whereas some scanners provide rudimentary color-grading and display options. At the present time, the main difference between scanners and telecines seems to be that telecines are operated by experienced colorists, while scanners operate almost automatically.

Color grading performed at the acquisition stage may have less of a destructive effect on the quality of the images because most high-endscanners perform color grading prior to actually capturing the image, effectively recalibrating the scan parameters to maximize the captured dynamic range within the new color parameters. In fact, this paradigm is a perfectly viable one for the digital intermediate pipeline. Even so, most pipelines prefer to leave color correction to a later stage because doing so allows more flexibility and makes it possible to view changes in a more useful context, with all the material available at the same time.

Telecines are routinely used by the film industry to allow the viewing of rushes (or dailies) of film that has just been developed, although this practice eventually may be replaced by the advent of digital dailies.

5.7.7 Digital Dailies

One of the advantages of digital media is that it can be viewed simultaneously from a number of places over a great distance. When shooting film, filmmakers typically send the day’s filmed material to be developed and then printed or telecined, or both (depending on their preference and the budget). The developed negative is stored, and the filmmakers watch the telecine or print to assess their work. However, as more filmmakers turn to the digital intermediate pipeline, the idea of digital dailies may become more attractive. As soon as the film negative is developed, it can be sent to the digital intermediate facility to be scanned. The scanned data can then be made available for viewing and copying. The negative can be stored safely (and theoretically is never needed again), and the data can be archived for post-production. The copies can be manipulated (e.g., compressed or encrypted) in many ways that don’t affect the original data, and they can be transmitted across the Internet, providing them to filmmakers, editors, and any number of people who may require an instant copy.

Digital dailies aren’t presently a popular option, mainly because the speed of the technology is much slower than a traditional telecine process, and the space requirements for storing a high volume of scanned film (especially when most of it will never be used) are too large to be practical. (There is certainly a positive element to limiting the number of people who have access to production material as well). However, as the technology becomes faster, the need for digital dailies will increase. Digital dailies may even be generated using more automation to produce footage that is displayed by a single piece of equipment, and may also include the means to apply some basic color grading to the footage.

5.7.8 Scanner Gauges

Although 35mm film is by far the most common gauge of film that is scanned, some film scanners accept a variety of other gauges, such as 16mm or 70mm. In addition, many scanners can capture specific regions of each frame, such as the “Academy aperture” region (which has a 1.37 aspect ratio). Capturing film of different gauges normally involves replacing the scanner gate to allow the different size of film to run smoothly through the machinery. It may also be necessary to change scanning parameters, such as the scanned image dimensions, to accommodate differences between formats.

5.7.9 Anamorphic Film

Anamorphic images have a ratio of 2.35 to 1, which is much wider than the 1.33 to 1 ratio of full-aperture 35mm film. Rather than waste film area, anamorphic images use a special lens to “squeeze” the picture horizontally to make it fit much more comfortably on the film area. When projected, the film is “unsqueezed” to give it the proper dimensions.

In the digital intermediate pipeline, anamorphic images can either be unsqueezed at the time of scanning and then saved as images with a wide aspect ratio, or be saved as narrower, squeezed images (with a wide pixel aspect ratio). The latter method has the advantage of saving disk space (and removing the need to resqueeze the images if out-puttingback to film) but requires that each device used to view the images (e.g., the conforming and color-correction systems) be able to automatically unsqueeze the images for display purposes. However, for images that have been squeezed in this way, the effective horizontal resolution (and by extension, the quality level) is halved compared to the vertical resolution. Superior quality images may be obtained by unsqueezing the images optically at the scanning stage (or by scanning twice the number of pixels horizontally than normal) and saving the images with square pixels.

5.7.10 Pin Registration

Pin registration ensures accurate positioning of each frame of film. Essentially, precisely positioned pins or “teeth” position film using the perforations along its edge, in the same way that a film is loaded into a camera. Using pin registration is necessary for two reasons. First, it reduces the possibility of the images “weaving” (i.e., moving up and down or from side to side in the frame) when played back in a sequence.¹⁷ Second, if any frames have to be rescanned at a later date, they’re more likely to match the originals exactly in terms of positioning and can therefore transparently replace the original scans. These effects become more pronounced at higher image resolutions.

The disadvantage of using pin registration is that it somewhat increases the scanning time because pin-registered scanners have to pause between each frame to ensure the frame’s correct positioning, whereas those scanners that aren’t pin-registered can continuously run the film over the scanning laser without stopping.

For nonpin-registered scanners (which includes most telecines), it’s possible to compensate for the lack of pin registration by using a digital process known as “motion stabilization” (or image stabilization), which is covered in Chapter 9. However, having to run this process on every scanned frame may be considered an extravagant waste of computer resources (and probably justifies the additional cost of a pin-registered scanner).

5.7.11 Key Numbers and Timecodes

A useful component of motion picture film is the key numbers. Every few frames of film negative manufactured is pre-exposed with a unique serial number, so that every frame can be independently referenced. These numbers were originally introduced to allow the use of cut lists, which specifically refer to individual or ranges of frames. The cut lists were useful because they enabled film editors to cut together sequences from prints of negatives (making it possible to avoid handling the original negative and subjecting it to possible damage) and then easily match each frame in the cut back to the corresponding negative frame.

More recently, a machine-readable code (or “bar code”) is included alongside each number. Some film scanners incorporate a key number reader that reads the bar code as the film is scanned and then embeds the corresponding number into the image file. This procedure is necessary for productions where the data will be conformed digitally, using a key-numbered cut list (see Chapter 7 for a discussion of conforming projects using key numbers).

Similar to key numbers is the idea of timecodes. Most video sources contain a timecode track that makes it possible to reference a particular frame. In some cases, it’s possible to record timecode information onto film for the same purpose. While timecodes aren’t unique like key numbers (i.e., two reels of film might contain the same timecodes but never the same key numbers), they’re useful for synchronizing film with other devices. For instance, when filming a music promotion, the timecodes might be linked to the soundtrack, so that the audio can be quickly matched to the picture. Sometimes this linking is performed with a “timecode slate” or a clapperboard containing timecodes, which is filmed at the beginning of each take, or alternatively, the timecode information can be embedded on the outside edge of the film in a machine-readable format. When the latter method is used, film scanners equipped with a timecode reader embed the timecode information into the scanned image.

5.8 Summary

Before any material can be used in the digital intermediate environment, it must first be ingested. With digital source material, this ingestion may be as simple as transferring the source data by copying files between devices.

With analog material, such as film and video, analog-to-digital conversion, a largely mathematical process, must be used. Film material may be digitized using a digital film scanner, or a suitably equipped telecine, while video sources require a video capture device. In general, analog-to-digital conversion should be performed at the highest quality possible, working from original material whenever feasible, capturing as much useful information as practicable, to benefit later stages in the pipeline.

In theory, data transfer can occur at much higher speeds than video or film, but in practice, data transfer can be a very slow process. For this reason, a balance must be achieved between image quality and file-size considerations, which in turn impact the pipeline’s performance and relative cost.

The image quality of acquired footage depends on a number of factors, such as the color space, the precision, and the resolution, and there are methods for measuring each factor throughout the pipeline. The digital acquisition process typically generates a large volume of data, and the organization of this data is the subject of the next chapter