11

Output

The previous chapter looked at the uses of digital effects and titles for putting the finishing touches to conformed and graded productions. The digital intermediate process has a significant advantage over other paradigms for creating productions because it can easily and simultaneously output the same data to a number of different mediums, such as video tape, DVD, 35mm photographic film, and streaming web servers. This chapter discusses the different options for outputting the finished production to several different mediums.

11.1 Rendering

With many of the different systems and processes in the digital intermediate pipeline, changes, such as color grading, made to the acquired footage are saved as metadata and don’t permanently affect the original images; they are just used to modify the way the images are presented to the viewer. Keeping the original image data separate from the modifications allows greater flexibility in making revisions, without having to compromise quality. If changes were integrated with the images at the time they were created, later revisions would have to be based upon the revised material. Therefore, the images would suffer from a generation-loss effect similar to the effect that photographic film and video tapes are prone to.

Every single frame in the production therefore can require several processes be run to generate the images for the final production. This procedure is usually known as “rendering.” Depending on the specifics of the digital intermediate pipeline, each frame may require several rendering “passes”—for example, one to resize an image, another to remove defects such as dust or dropout, another to apply the color grading, and several others to add optical, text, and other effects.¹

Rendering consumes an incredible amount of computer processing. Calculations are required for each of the millions of pixels on each frame. Because the total frames per production usually number several hundred thousand, the total number of calculations required to fully render a production are in the trillions. For the production to be rendered within a reasonable time frame, either the rendering system must process each calculation very quickly or the task must be divided among several systems.

11.1.1 Dedicated Rendering

Some digital intermediate facilities use specially configured computer systems solely for rendering purposes. Others may harness the processing power of the playback or grading system to render the files once the final version is “locked.” Either way, such configurations rely on these systems’ speed to render the required files.

The advantage of the dedicated rendering setup is that it can be very cost effective, particularly for medium-sized facilities that render only one production at a time. On top of this, it’s easy to make predictions as to the time frame of each rendering process because few factors affect the processing speed, which is fairly consistent. Furthermore, such systems tend to be fairly robust, in that they are less prone to compatibility issues and other render-related problems. The downside is that dedicated rendering is a case of putting all the proverbial eggs into one basket: if the rendering system fails, deadlines might be missed. Similarly, while the system is rendering, it can’t be utilized for other tasks, such as grading another production.

11.1.2 Distributed Rendering

An alternative method is to distribute the frames to be rendered among a number of systems. For example, in a rendering “pool” (or render “farm”) consisting of 10 rendering systems, every 100 frames having to be rendered may be divided equally among the rendering systems, so that each renders 10 frames out of every 100. This approach can be even more sophisticated, with render “wranglers” assigning more frames to faster machines and less to slower ones.

This system can be extremely efficient, particularly for large pipelines, because each rendering system can be used as necessary or removed from the pool when needed for other tasks or other productions. Additional expense is required to provide enough rendering systems (although most of the time, plenty of existing systems used for other purposes can be adapted), as well as to provide the required infrastructure to facilitate the rendering and management. Such configurations must be continually monitored for problems, because they’re prone to network errors, disk failures, and other sporadic glitches. Two separate systems occasionally produce different results. This problem occurs with very complex images, when mathematical errors are made or when part of the rendering process utilizes a system’s GPU (graphical processing unit) rather than the CPU; any of these situations can lead to continuity problems or grading flashes.

11.1.3 Background Rendering

A third approach is to harness other computer systems’ idle time. Most modern computer systems rarely use their maximum processing power at any given time. For example, digitizing an hour of video footage might use a fraction of the available memory and no network bandwidth. Thus, these resources aren’t being used (although the display resources may be used to provide feedback about those frames being digitized). Rendering processes, on the other hand, typically use all the available memory, processing power, and network bandwidth, while not having to provide a display of the frames being processed. Likewise, many systems are sometimes left completely unused while waiting for the next job to be processed. Rather than monopolizing the resources of a single dedicated rendering system, or of a pool of systems, it’s sometimes possible to harness the idle systems’ “spare” resources and put them to work rendering some of the frames. As soon as the resources are needed again by the host system, the rendering is suspended until the resources become available again, a process known as “background rendering.”

The benefits of background rendering are that it’s an extremely economical method of rendering frames and enables the use of the systems for other processes. Thus, background rendering is suitable for smaller facilities, which may not have access to a large pool of computers or be able to afford a dedicated system. However, this method is the slowest of the three, because it must share rendering resources with other processes. Furthermore, this method can be prone to render errors, caused by miscalculations or software failures.

In practice, most digital intermediate facilities don’t stick to a single rendering paradigm and usually turn to different approaches for different rendering requirements. For example, background rendering may be used throughout the production, while distributed rendering is used toward the end to ensure maximum speed, with dedicated systems used for specific processes, such as reformatting images for specific output mediums.

The required amount of rendering varies by the production, digital intermediate pipeline, and the pipeline subsystems. Some configurations, particularly those for video productions, are able to process and render all frames completely in real time, meaning that rendering time doesn’t even have to be considered.

11.2 Video Output

Output of digital footage to video-based media is a fairly straightforward process. After all, most of the viewing devices used throughout the digital intermediate process require some form of analog video signal as an input. So, in theory at least, video output requires merely attaching a VCR to the output signal, providing that the system can play back the production in real time. This process is commonly referred to as “crash” recording (or “live” recording) and effectively dumps the incoming signal onto tape as it’s received, with no particular concern for adhering to specific timecodes.

However, most pipelines require a much greater degree of control over this process. First of all, the output has to be synced to very specific timecodes, particularly when using “insert” editing to precisely overlay particular shots and sequences into exact positions on the tape. For this to work, the output or playback system must be given the ability to directly control the VCR, which is usually facilitated by adding a control cable to suitably equipped systems.

Because different video formats have different requirements (e.g., a 1080p24 HD video records at a different picture size and frame rate than a 525 SD video), separate renders of the production are necessary for each required format. Some playback systems, such as Thomson’s Specter FS (www.thomsongrassvalley.com), can perform the necessary calculations in real time while outputting the video, so no extra rendering time (or storage space) is required for video output. However, for many systems, time must be allocated between the completion of the production and the output to video to accommodate this additional rendering.

The only factors affecting the quality of the recording (provided the output system is of maximum quality) are the VCR deck, the type of video tape, and the cables connecting the deck to the output systems (i.e., gold-plated and low-capacitance cables minimize the introduction of noise into the signal). So a digital Betacam can provide a better recording than a VHS, for example, despite the fact that they record an identical signal.

Digital video systems (e.g., HDV or DV) usually directly record a digital signal, such as through a Firewire cable. This results in the maximum possible quality, although the footage may first have to be “transcoded” to the proper digital format, which requires additional time and storage space. Most digital video decks also have analog video inputs, and so for the sake of simplicity and speed considerations, many digital intermediate facilities choose to output video by analog means.

VCRs usually require some “pre-roll” time (usually several seconds) to get the tape up to speed before beginning the recording. At the end of the recording some “post-roll” time allows the tape to speed down. Many distributors have specific requirements for how the tape should be formatted, with specific timecodes for slates and test patterns. Audio for production is usually created separately; the video and audio components are on different “tracks” of the video, and thus can be recorded at different times in different places if need be.

Video tapes vary in length, and productions either are recorded in their entirety onto one tape, or they’re split into several reels of around 20 minutes each. The material is output in real time so that an hour’s worth of footage takes just over an hour to output to tape (allowing for setup time and other necessary operations). Every tape the system outputs usually undergoes a lengthy “quality control” check for a number of defects and issues (covered in Chapter 12). Because of the QC process, a digital intermediate facility usually outputs only one tape per format, which becomes the “master” tape; the facility outputs an additional tape as a backup, which is usually stored within the facility. If corrections must be made, the corrected shots can be inserted onto the master tapes.

Because each tape’s output can be a lengthy and expensive process overall, especially for film or HD-based projects, it’s often preferable to generate a single tape on a high-quality format and then just “dub” this tape across to the other formats. The typical method is to output an HD master at 50i, which is then used to generate all other 50 Hz formats (such as 625 PAL SD tapes), and an HD master at 59.94i or 23.98p to generate all other NTSC formats. Progressive-format-based productions (such as film or 24p HD projects) normally also output an HD master at 24p for posterity. Providing that the frame rate remains the same, “downconverting” (i.e., copying a video from a higher-quality or larger video format to a lower-quality, smaller one) is a viable alternative to re-rendering and reoutputting the finished production to a number of formats.

11.2.1 Frame-Rate Conversion

One characteristic of video formats is that each has different framerate requirements. Most productions stick to a single frame rate throughout. At the time of output, therefore, a decision must be made as to how to convert the program to match the output frame-rate requirements.

One of the simplest, and most common methods is to play out each sequence “frame for frame,” meaning that one frame of footage in the digital intermediate equals one frame of video. Of course, this approach results in different playback speeds, because if images are shot at 24fps and played back at 25fps, for example (e.g., images shot on film and played back on PAL video), then the footage will playback slightly faster than it was shot. This also means that the audio must be sped up or slowed down accordingly so that it doesn’t become unsynchronized. For small adjustments, such as from 24fps to 25fps (i.e., a difference of 4%), the frame-for-frame method usually suffices. However, for larger differences, such as film to NTSC, some other method must be used. The common solution is to apply a pulldown (as described in Chapter 7), which repeats some frames at predetermined intervals. This results in footage that plays back at the same speed as the original, although motion won’t appear as smooth as with other methods. An additional solution is to apply a motion effect (as described in Chapter 10) to the entire production, to generate a new set of frames for playback at the desired speed. This approach usually results in smoother motion, but it requires significant time and storage space to generate the additional frames. Depending on the method, some visible motion artifacts may be produced.

11.2.2 Video-Safe Colors

Video has a slightly different color space from the gamma-corrected RGB spaces used by computers. Thus, some colors that appear on a computer monitor won’t show up correctly when broadcast; and such colors are termed “illegal” colors. Therefore, it’s usually a requirement that all video footage output from the digital intermediate process be “legal” (or “video-safe”) within the respective video color space.

Most digital intermediate video output systems perform this check automatically and adjust the color space as needed. In some cases, it may be impossible to output video that contains illegal colors. Where this isn’t the case, video can usually be output using a LUT to convert the colors so they fall within the correct color space, or software or hardware may be available to re-render all the footage so it’s safe. Sometimes this process can be done in real time on the output system; otherwise, additional time and storage space must be factored in.

11.2.3 Aspect Ratio

Different formats have different aspect ratios or picture shapes. For example, some high-definition video formats have an aspect ratio of 1.78:1, whereas the majority of standard definition video formats have an aspect ratio of 1.33:1, which is much narrower. It’s possible to convert between different aspect ratios fairly easily, using any of a number of methods.

When converting a wide image to a narrower one, the wider image can be “cropped” vertically, trimming the image’s left and right edges to achieve the desired shape.

Alternatively, the image can be “squeezed,” squashing the image to fit the narrower shape, although doing so distorts the image and isn’t generally recommended.

Finally, the image can be “letterboxed,” resizing the picture so that it maintains its original shape and filling the top and bottom of the adjusted image with black.

The same options are also available when converting a narrow image into a wider one, cropping the top and bottom of the image, squeezing or letterboxing it, or filling the sides of the image with black.

Cropping an image affords the greatest flexibility (although part of the original image inevitably is lost) because the picture can also be resized to enable the shot to be recomposed as part of the process. Where the image is cropped, it may be desirable to perform a “pan and scan,” to best select how to crop the image on a shot-by-shot basis.

11.2.4 Reinterlacing Video

Although it isn’t a particularly common request, it may be necessary in some circumstances to “reinterlace” progressive footage to reconstruct fields, which is usually the case for projects that were originally shot on interlaced video systems and subsequently deinterlaced as part of the digital intermediate process. When frames are output to video normally, the odd lines of each frame are used to construct one field, and the even lines are used for the other. Thus, no intervention usually is required.²

The options for subsequently reinterlacing footage are much the same as those for deinterlacing footage (as described in Chapter 9)—that is, a frame may simply be split in half and duplicated so that it appears once for every frame, or you may perform sophisticated motion analysis on the frame to separately reconstruct each field. A rule of thumb is to simply reverse the method used to deinterlace the footage in the first place.

11.3 Film Output

Digital images can be output to film using a straightforward process. Unexposed film (usually a negative stock with very fine grain) is exposed a pixel at a time, firing three lasers at the surface of the film (one for each of red, green, and blue) which are at varying intensities depending upon the RGB values of the image’s source pixel. This process is repeated for each pixel in the digital image until the entire frame has been exposed, and the film is advanced to the next frame. When the entire reel has been exposed, it’s sent to a lab for development and printing.

Film recording is currently a fairly slow process. Arri’s popular Arrilaser Speed Performance recorder (www.arri.com) takes approximately 2 seconds to record a frame from a 2k image or around 4 seconds to record a frame from a 4k image. Productions are usually recorded using lasers (or “filmed out” or “shot”) in reels of up to 2000 feet (up to 22 minutes of footage). A 2000-foot reel of film, comprising up to 32,000 frames, can take around 18 hours to record at 2k or 36 hours at 4k resolution, depending upon the recorder. This means that a typical film might take several weeks just to complete the recording.

Most facilities get around this bottleneck by using multiple recorders, simultaneously recording several separate reels on each recorder. Another option is to stagger the film-out of reels throughout the postproduction process, so that as soon as a single reel is approved, it’s sent to the recorder, while the facility completes other reels. Reels must be processed separately, and each reel should end on a scene change, to minimize continuity errors caused by differences in each reel’s development. Other recorders, such as the Celco Fury (www.celco.com), can output to other formats, such as IMAX.

To ensure the graded digital version matches the final film output, correct calibration of the film recorder is crucial to the digital intermediate pipeline. The recorder is usually calibrated to a standard, and the calibration of the digital-grading system is manipulated to match the output of the calibrated recorder. That way, the recorder can be calibrated quickly and independently from the grading system (of course, this step must be done before grading begins).

Each digital image to be recorded must be “mapped” to each frame of film to match a particular format (or “aperture”). For example, Super-35 images usually fill the entire frame, while Academy aperture occupies a specific region of each frame. Predefined standards specify the size of digital images (in pixels) depending on the output aperture, and many digital intermediate facilities use these standards to determine the working file size used throughout the pipeline. In other cases, the digital images have to be resized to match the standard. Film recorders also output each frame individually, so footage sequences have to be output as individual images (i.e., one for each frame), which again is standard for most facilities but may require separate rendering for the others.

Once the recording process has started, it can’t be interrupted without loading a new reel of film. If the system breaks down, it may be possible to recover the frames recorded until that point and start a new reel from the start of the last shot. However, this requires that the two reels be spliced together later to create the complete reel.

At the end of the film-out process, a “digital internegative” or a “digital interpositive” is created. These are equivalent to the internegative or interpositive produced by a traditional (nondigital) chemical lab process to finish a film, which would then be duplicated to make the distribution prints. Because generating each reel digitally requires such a long time, productions normally generate a single digital interpositive, which is then used to create several internegatives. The latter, in turn, are used to create the thousands of distribution prints for cinemas. Clearly this approach leads to more image degradation, because it introduces an additional generation into the process. Some digital intermediate facilities have sufficient film recorders to be able to offer multiple digital internegatives, which can then be used to create the distribution prints directly, resulting in superior quality pictures. As the recording technology improves and becomes less expensive, more facilities will offer this approach.

It’s interesting to note that, of all the output formats, photographic film is widely considered to be the most suitable for long-term storage. Digital file formats and storage devices change so frequently that anything used today probably will be unreadable in 50 years. Video tapes don’t have the necessary level of quality or the longevity of film. On the other hand, film originally shot almost a century ago is still viewable today. For this reason, even productions that aren’t likely to have a cinema release might still benefit from digital internegative output.

11.4 Digital Mastering

Without a doubt, the most convenient and highest-quality output can be obtained by generating a digital master. This can be done for archival purposes, to allow conversion to other output digital formats, such as DVD or streaming Internet video, or for digital cinema distribution.

A “digital source master” is a finished production in digital form from which all other formats can be created. No standards to speak of currently exist for creating digital source masters, and thus the final rendered output usually serves this purpose. The digital source master can be a number of individual digital images stored inside one or more folders (such as one image per frame, and one folder per reel). It can be a single digital video-encoded file, such as for DV-based productions, or it may be a more complex, proprietary format devised by the digital intermediate facility.

Regardless of the specifics of the digital source master, outputting to other digital formats includes three stages: transcoding, compression, and encryption.

11.4.1 Transcoding

A digital image file consists of two components: the image data (the “essence”) and data concerning the organization of the image data (the “metadata”). Each image format organizes the image data in a different way, even when the image itself is the same. For example, a TIFF file and an SGI file might display an identical image, right down to the individual pixels, but the image would appear completely different when comparing the raw data saved on disk. For this reason, converting between different formats, even those that use the same color space and so on, involves reorganizing the data. For formats that aren’t directly compatible (such as converting a 10-bit format to an 8-bit one), not only must the data be reorganized, but changes must also be made to the image data to make it compatible with the new format.

This process of transcoding can be very time consuming, particularly for most productions with large number of frames. Transcoding is also used when converting between still images and digital video streams (such as a QuickTime file). In addition to reorganizing the data, it may be necessary to resize, retime, and alter the colors of the footage.³ For this reason, the transcoding process may degrade the images somewhat, depending upon the specific differences between the formats, but the degradation is usually a factor of the new format’s limitations of and can’t be avoided. All transcoding should be done from the original digital source master to maximize the quality of the transcoded footage.

11.4.2 Compression

As discussed previously, the two types of compression methods are lossy compression and lossless compression. At this stage in the digital intermediate pipeline, lossless compression might be utilized more because storage space may take precedence over other considerations. Lossy compression probably will be used too, especially when transcoding to other formats (such as for DVD versions).

Using lossy compression, in particular the so-called “visually lossless” compression methods, at this stage is acceptable because even though the image quality is degraded, most such compression methods are designed so that the quality differences are imperceptible. Because lossy compression is performed at the very end of the pipeline, the images won’t usually undergo any operations that rely on the imperceptible information and that would otherwise introduce compression artifacts (such as color grading).

Lossy compression techniques fall into two main categories: intraframe and interframe methods. Intraframe compression methods, such as M-JPEG compression, analyze each frame separately, compressing the image by discarding data that’s imperceptible (which is the same as compressing a single digital image to a JPEG file). This process is repeated for each frame. The result is a single frame of footage, when viewed, appears no different than the original frame. However, it’s possible that motion artifacts may be visible when the compressed footage is played back at speed.

Interframe compression methods, such as MPEG compression, compare each frame to the previous one and discard parts of the image where no change is visible between frames. This can result in much smaller files, particularly as the vast majority of footage doesn’t change very much between each frame. However, sometimes the compression method (or “codec,” short for “compressor decompressor”) can be fooled by subtle changes in the scene, which can lead to smearing and other artifacts. The amount of file-size reduction is determined by the compression method in conjunction with the threshold, which determines the level of minimum quality (i.e., deciding how much of the footage can be thrown away).

One problem with compression is that it can be difficult to estimate the amount of file-size reduction that will be achieved for a given program. Compression is often used to ensure that a certain amount of footage, such as a 90-minute production, will fit on a specific storage media (such as a DVD). The most common way around this difficulty is to specify a target bit rate for the compression method. For example, for 100 seconds of footage that must fit on a 150MB device, a bit rate of 1.5 megabytes per second might be specified to ensure that the final compressed version will fit on the device. The codec analyzes the footage a second at a time and aims to provide sufficient compression to meet the target level of compression. This technique is often used for DVD, videos, and streaming video on the Internet. However, it results in scenes containing a lot of motion (thus requiring less compression to be perceived at a high level of quality) looking inferior to those with less motion, and it may even produce visible artifacts. One solution to this problem is to make use of variable bit-rate codecs, which adapt to the requirements of the footage, increasing the bit rate for scenes with a lot of motion and reducing it for scenes with much less motion, while aiming to maintain an overall target bit rate. Though this approach can produce superior results, it may be impractical for certain applications, such as streaming video, which must be delivered at a constant rate to ensure smooth playback. Another option is to perform multiple analysis passes on the footage, to more accurately interpret the footage content.

Even for a specific codec with a standard bit-rate requirement, the quality of the final, compressed footage is determined in part by the software that performed the compression. For example, DVD video requires footage that has been compressed using an MPEG-2 codec at a specific bit rate, but several different algorithms and compression systems can be used to process a digital source master into the MPEG-2 file format. Each method can produce a different result, according to the image evaluation, in terms of factors such as sharpness, color, motion smoothness, and artifacts. For this reason, many compression systems provide the capability of tailoring many of the detection parameters affecting these factors to the specific requirements of each production, although this process can require lots of time and expertise.

11.4.3 Encryption

Data encryption methods can restrict access to the contents of a digital file or storage system. It’s normally used to prevent piracy or leaked copies of images, or even the entire film, prior to release. It’s commonly used in a digital cinema environment, where data is prone to interception, but in theory, it could be used throughout the entire digital intermediate process as an added security measure. However, it can increase the processing time and make the files more vulnerable to corruption. Thus, it’s typically used at the end for transmission or long-term storage.

Methods for encrypting data range from password-protected file storage, to quantum cryptography, each of which varies in terms of security, expense, and practicality.

Data encryption works by scrambling the contents of a file (or an entire storage device) in a manner similar to randomly rearranging all the letters in a book. When the correct credentials are supplied, the contents are put back in the correct order—that is, the file is decrypted. These credentials can be provided in the form of a password, a digital “key,” or even something like a swipe card used at the workstation, which then “unlocks” the file, allowing access to its contents.

11.4.4 Digital Cinema Distribution Masters

Digital cinema works by projecting images directly from a digital file. A projection system’s specific workings vary between cinemas, and they’re determined in part by each distributor. In general, the various projection systems share some similarities in terms of requirements.

From the digital source master created at the end of the digital intermediate process, a digital cinema distribution master (DCDM) must be generated. A DCDM is a set of data that conforms to a specific standard in terms of formatting and storage. The DCDM is then duplicated to several central servers within the digital cinema network, each of which is then distributed to each cinema for playback.

The specifications for the DCDM, as outlined by the Digital Cinema Initiative, lean toward an uncompressed, color-space-independent format, which should guarantee that the creation of the DCDM doesn’t result in any degradation of the digital source master. Individual cinemas interpret the DCDM based on the characteristics of their projection systems, so that, for example, a projector’s specific color space will determine the color calibration or corrections (if any) that must be applied to the DCDM during playback.

The current DCDM specifications are covered in detail in the Appendix.

11.4.5 DVDs

One of the most important output media, particularly for commercial productions, is DVD (Digital Versatile Disc) output. DVD is currently the best-selling consumer-grade medium for film and television productions. It offers several significant advantages over video-tape-based media, such as VHS, including higher-quality pictures and audio, the absence of degradation or generation loss, the ability to instantly seek to any part of the program and switch between audio and caption formats and languages, all in a more compact package.

Because DVD is an inherently digital format, the digital source master can create the DVD master directly. DVD is a relatively new format and, very simply, is, like CD-ROM, a set of digital files stored on a write once read many (WORM) optical disc.

Each of the several different DVD specifications is for a different purpose, which can lead to some confusion. For example, a DVD-ROM disc can be used as a digital storage medium (under the “Book A” specification), similar to a flash drive or a floppy disk, by recording the desired files directly onto it, which perfectly preserves the data but makes it accessible only to other computer systems. For set-top DVD players to display the DVD content, a DVD video (or “Book B” specification) disc must be created. This format has an explicit file structure, requiring that audio and video content conform to a detailed specification. Video must be compressed to the MPEG-2 standard and split into separate files, with a typical single disc capable holding an hour of high-quality audio and video.⁴

The DVD video-mastering (or “authoring”) process involves transcoding and compressing the digital source master to an MPEG-2 format file before assembling the other elements (such as audio and caption streams and menu elements) into a DVD master. For large distributions, the master is then usually sent to a replication center that creates a glass master for stamping thousands of discs to be sold. More limited, low-budget distributions can simply use DVD writers to copy the master to writable DVD discs (such as DVD-R or DVD+R discs), which can then be played on most set-top players.⁵

11.4.6 Other Digital Formats

New formats are introduced on a regular basis. In addition to DVD video, there’s CD-ROM-based video formats, such as Video-CD (VCD) and Super Video-CD (SVCD), each of which has variants. Shorter productions can be mastered in the same way as DVD video, but they require the storage capacity of only a CD-ROM, giving rise to mini-DVDs and other hybrid formats.

New display devices, such as cellphones and handheld digital video viewers (e.g., Sony’s PlayStation Portable (www.us.playstation.com), will no doubt give rise to other new formats; the introduction of new storage media such as HD-DVD, HVD, and Blu-ray discs will enable the distribution of content of higher quality and increased quantity. And new compression methods and codecs such as MPEG-4 (also called “H.264”) and Windows Media High Definition Video (WMVHD) will make it possible for higher-quality footage to be transmitted across limited bandwidths.

For now, at least, digital formats have no widely adopted standards (other than DVD video), and so most digital formats are targeted for playback on specific systems, rather than for wide-scale distribution.

11.4.7 Streaming Video

Digital files can be transmitted across the Internet using several different methods; one such method is to place them on a website, and another is to establish a secure network connection between two connected systems. Any of these methods can send digital footage of any type, whether as a series of still images, a compressed MPEG file, or even an entire DVD video structure.

Such methods, however, are not generally useful for viewing footage “live”—that is, as soon as it’s received. Transmission of data across the Internet occurs at a variable rate. The same file, transmitted several times across the Internet, requires a different length of time to complete the process each time it’s sent.

Compounding this issue further, it can be very difficult to calculate with any degree of accuracy the length of time required to transmit a given file from one location to another. Transmission durations depend on a number of factors, including the quality of the route between the two systems, Internet traffic, and the network bandwidth of all points along the route. Trying to view footage as it’s received can result in dropped frames or stalling, or the playback can suffer from lag.

A couple of ways can be used to get around this issue, employing a process known as “streaming.” Streaming a video means transmitting it from the host system to one or more client systems, at a rate that enables it to be viewed while it’s being transmitted.

The simplest method for streaming video is “buffering.” With buffered video, the host system transmits all data to the client as fast as possible, usually through simple HTTP (web-based) or FTP protocols. The client playback system stores the locally received material and analyzes the average rate that data is received. It uses this information to estimate how long the transfer will take to complete. Once a point is reached where the estimated time remaining is less than the duration of the video, playback begins. Provided that the analysis was correct, the video will finish playing just after the last of it has been transferred. This process relies on several factors, however; first, the transmission rate can’t drop below the average once playback begins, and the video is encoded in such a way that each second uses the same amount of storage space. This method requires client systems with appropriate playback software to handle the buffer timing, although the host system doesn’t require any specific software.

A more advanced paradigm is the dedicated streaming server that controls the rate at which data is transmitted to the client. Such a system enables the host to adaptively vary the bit rate to compensate for discrepancies in transmission speed, so the client experiences a much smoother playback (although the image quality will vary). Installing and configuring such a server requires a significantly higher cost than the buffered method and is therefore unsuitable for limited distribution of material.

At the present time, streaming video output is considered low quality, even with the widespread availability of broadband Internet connections, particularly for long feeds that take a long time to buffer. Streaming video is usually reserved for short promotional videos, commercials, or trailers. In the future, it may be possible to transmit full-length, DVD-quality video using streaming techniques without experiencing much delay, which will serve a new distribution market.

11.4.8 Archiving

The digital source master has different requirements for long-term storage than output targeted for playback on specific systems. The aim of output in the long term is to create files that can be used to generate more output at some point in the future. The simplest and most common form of archiving is to simply copy the digital source master onto a long-term storage device, such as a tape or optical system.

Many digital intermediate facilities also retain digital copies of each different output format, such as a standard-definition PAL video, and a high-definition 1080i59.94 video, in addition to the digital source master. Other facilities may choose to archive the original elements, such as the scanned footage, the grading lists, restoration layers, and so on. These elements will usually require a lot of storage.

The problem with archiving is that it can be difficult to predict under which circumstances the archive will be needed. In the short-term, an archive may be needed to generate duplicates of certain output formats, which means that it would be much more convenient to just archive the digital versions of each format. The archive may be needed to allow output to other, newer formats at some later date, and in that case, it would be useful to keep the digital source master to generate new formats. In the long term, the archive might be required to regrade or re-edit elements, in which case, saving all the original elements is much more useful.

The ultimate aim is to create a single format that is stored at the maximum resolution (or is, in some way, resolution independent), is color-space independent, and contains information to relate the data back to the original offline EDL, perhaps even including handle frames. Although this can be achieved to some degree through specific file formats and the use of metadata, the prevailing problem remains: ensuring that anything archived today is useful in the long term.

Digital image file formats that were widely used 10 years ago, such as the TrueVision Targa format, are less popular now and, in some cases, are not viewable using modern systems. This trend doesn’t seem to be going to change. Even the popular Cineon format used throughout the industry already seems to be beginning to lose out to the next generation of high-dynamic range formats such as JPEG 2000 and Extended Dynamic Range (OpenEXR).

This is compounded by the inevitable changes that will be made to the file structure of the host operating systems and to hardware interfaces. Chances are that the hardware used to archive files today won’t be compatible with future systems, rendering their contents unreadable. This is, in turn, made worse by physical decay of the storage media—for example, many consumer-grade recordable CDs that were created ten years ago are now unreadable, despite the fact that the hardware used to read them hasn’t changed very much. Tapebased archiving systems have a life of around 5 to 10 years, which means that either the tapes have to be regularly regenerated (i.e., the data restored and rearchived to new tapes) to increase the data’s longevity or online storage must be used.

Use of online storage simply means that the data is stored on systems that are operational and powered. For example, a digital intermediate facility may have 100TB of online storage, of which 90TB is devoted to ongoing projects and 10TB to archiving previous projects. The archived files can be easily monitored for disk errors, and individual disks replaced as needed. Having instant access to the images for viewing or transfer is an additional benefit; in contrast, tape-based archiving systems have linear access, meaning that the tape must be spooled along to find the desired file, and the file must be restored to an online system before it can be accessed. Using online storage is expensive, particularly because the overall capacity must be increased with each new project.

Standard definition video-based pipelines may resort to using a DV25 or DV50 format for archive because these archive formats store the footage in a digital format, and the DV codecs are the most likely to be around for another decade or so (particularly because of all the consumer investment in them). DVD is also suitable, although the quality is inferior to DV.

The individual components of the digital intermediate pipeline are constantly updated. New versions of grading systems, for example, tend to be released on a yearly basis, and ten years from now, many of the most prominent ones won’t exist, meaning that reading the project files created by the original system will be impossible. So the chances are that many project files created for use within a specific pipeline won’t be usable in a few years’ time and might even become outdated within a year or two, especially if major changes are made to the host system.

All of this amounts to a great deal of uncertainty as to how best to ensure archives are readable. Some standardization is certainly required—standardization of image file formats in the long term but also of various components within the pipeline. Most conforming systems rely on the CMX 3600 EDL format, which in turn is supported by all major editing systems. Thus, it should be possible to correctly interpret these EDLs on different systems, even in a few years’ time. Color-grading systems, on the other hand, tend to have proprietary project files, formatted in such a way as to make them unreadable by other systems. So, although it’s possible to archive an EDL to preserve the details of a film’s final cut, it’s unlikely that the settings for the color grading applied to each shot can be preserved in a useful way.

The only way to completely guarantee perfect preservation of the digital intermediate is to store, not only the associated data, but also the physical systems, network cabling, display devices, and so on. Clearly, this approach is preposterously impractical (not to mention that physical decay of the components probably could cause the failure of the systems when they’re reactivated anyway), but it’s indicative of the process’s volatility. A much more feasible, though time-consuming and costly, method is to archive the digital source master, periodically updating it by transcoding and rearchiving whenever changes are made to the pipeline—changes that threaten to make the archived data redundant.

11.5 Multiple Versions

It’s rare that a production will require a single version of the final project to be output, especially for larger productions. Often a feature is re-cut to satisfy the needs of different distribution outlets—broadcast television and airline versions of films are often created which censor certain scenes (such as airplane crashes in the cases of airline versions), or cut down to fit a particular time slot. DVD may be output in several versions, such as a theatrical version (to match the version on film) and one or more extended or recut versions (e.g., for a special edition release). These versions can be output in addition to those output to different media, such as film.

Although additional versions such as these share much of the same footage, it’s common practice to output each version separately, in its entirety. This greatly simplifies project management, although it requires additional storage space and version checking. In some instances, the changed scenes are output on their own as separate elements (usually with a guide frame before and after to show exactly where in the picture the new scene belongs), although this approach means that the output version must be re-edited separately to incorporate the changes.

Multiple versions may be output for additional reasons. For example, it’s common practice to output a “widescreen” version (i.e., formatted to fit a screen with a 16:9 aspect ratio) and a “full-screen” version (formatted to fit a screen with a 4:3 aspect ratio). A separate 2.35 aspect version can also be output (although it would normally be letterboxed to fit within a 4:3 or 16:9 aspect display) to match a theatrical release. Finally, versions may be output for different locations—for example, replacing titles and other text with other languages. Creating each of the separate versions can be time consuming, particularly from an organizational point of view because each shot must be cross-referenced against the particular version to be output, with changes added as necessary.

Systems such as Thomson’s Bones software (www.thomsongrassvalley.com) streamline this process somewhat, enabling different operators to be linked to different outputs, so that, for example, a title may be added to the rendered film output but not to the video output.

11.5.1 Panning and Scanning

In some instances, outputting a finished production to different formats may destroy the original composition. For example, outputting a 4:3 formatted picture to a 16:9 medium (or vice versa) results in having to crop part of the image (assuming that the picture isn’t letterboxed). In such situations, it may be necessary to “pan and scan” the new output.

With panning and scanning, the picture is resized to fit within the new format (usually ensuring that the aspect ratio of the original image remains unchanged) and then repositioning the shot to create a new composition.

You must avoid some pitfalls during this process. One of the most frequent problems is with shots of two or more people talking. The original composition may have a person at each edge of the frame, which means that inevitably one of the people will be cropped out of the panned-and-scanned shot. One solution is to pan the image, moving the image within the frame dynamically throughout the shot. This process can be difficult to achieve because poorly timed panning can be distracting to the viewer or can impair the visual style created by the original camera work.

Sometimes it might be desirable to omit the person who’s talking from the recomposed frame and focus instead on another element in the scene. Having to make these kind of decisions is why it’s prudent to perform the panning-and-scanning phase with a synced audio mix of the final cut, to determine which parts of a shot have to be viewed and which can just be heard.

11.5.2 Captions

Captions, subtitles, and other “timed text” elements are words (typically a transcript of spoken and audio effects) that appear onscreen for the hard of hearing or in foreign-language films.⁶ The captions are normally stored in a simple text file, with a timecode (usually either a frame count or a SMPTE timecode) to set the temporal positioning and the duration followed by the text to be displayed. In the case of “closed captions” (or subtitled DVDs), the text isn’t printed over the image, but instead is displayed when requested, superimposed over the image (in the case of TV, video, DVD, and various digital formats), or transmitted to a portable receiver (in the case of cinema releases). Subtitles for cinema releases, on the other hand, are usually embedded into the image during the release printing process, appearing onscreen as part of the image.

With digital cinema distribution, it is possible to keep all caption and subtitle material separate from the image, with the text rendered over the image during display, negating the need for making prints for specific regions, as different languages can be selected on a perscreening basis. In theory, some provision would allow more control over the display of subtitles, such as associating different typefaces or colors with different characters or events, and the ability to reposition and animate text, such as having the words crawl along the bottom of the screen.

11.6 Copy Protection

Digital imaging allows each copy of an image to be identical to the original, thus preserving the overall quality. Throughout the digital intermediate pipeline, this fact is a boon for maximizing the visual quality of the finished production. However, it also means that it can be very easy for pirates to run off a copy that is in every way identical to the original. This is good news for the pirates, who would have previously been selling inferior quality video dubs, or material re-photographed from a cinema screen, but bad news for distributors selling genuine, licensed material. Piracy may take the form of nonprofit file-sharing, where users connect to a large online database and transfer films to each other digitally, or professional duplication, where pirates aim to sell replicas of commercial DVDs and videos, even those which may not yet have been released.

There are several methods for dealing with piracy, such as litigation, although this approach has a tendency to target the wrong people and create a sense of disdain with consumers, as has happened to some degree with the music industry.

Other methods include embedding visible or invisible watermarks in the images or utilizing rights management and image scrambling or digital locks. The use of watermarks, discussed in chapter 10, doesn’t prevent copies of a production being made, but it does allow copies to be traced back to a source, information which can then be used to prevent future piracy.

Digital rights management systems assign ownership or licenses of material to each file. When a consumer attempts to copy a file, the system verifies the user’s credentials to ensure that they permitted to make the copy before proceeding. The problem with such systems is that they are dependent upon the hardware and software running on the display system being set up correctly for digital rights management. So far at least, no standard has been established for digital rights management, meaning that it’s usually limited to proprietary devices, such as Apple’s online music store iTunes (www.itunes.com), which allows purchased music to be played on a single system with Apple’s own iTunes software or on a single iPod music player. Clearly such a situation can cause problems: if the same type of system was used to handle DVD playback, for example, DVD rentals would be impossible, and different DVD releases would be required for each digital rights management system, causing unnecessary confusion.⁷ In fact, Sony recently abandoned a new copy-prevention system for music CDs amid claims that the system made the CDs incompatible with certain players and prevented legitimate copies being made.

Similarly, the media might be protected by digital locks, whereby the content is encrypted and can be decoded only by the correct digital “key” supplied by the playback system. This is the reason that commercial DVDs can’t be copied digitally using normal DVD writers.⁸ An extension to this solution is to incorporate a pay-per-view type scheme into each production. The file can be encrypted, and therefore copied freely, but is unwatchable or scrambled until the digital key is supplied to temporarily unlock it. This functionality is somewhat more standardized within future digital formats such as MPEG-7 and MPEG-13. Although digital locks can ultimately be broken, the process can be a difficult one, sufficiently complicated to deter the average consumer.

Finally, it’s also possible to create distribution media that “selfdestruct,” such as Flexplay’s ez-D discs (www.flexplay.com) that expire after 48 hours. Although it’s not an ideal solution for consumers wishing to buy DVDs of a production, they’re suitable for generating preview or screening copies or commercial DVD rental copies. However, the process can be very expensive and is unpopular with the majority of consumers.

11.7 Summary

The digital intermediate paradigm offers a wide range of options for producing finished productions in a form appropriate for viewing by consumers. With many pipelines, all the footage has to undergo a rendering process, to make all the changes to the images permanent, although this process can require significant time. The images can then be output to a number of different media, including both film and video, although the prerequisites for each are slightly different. It’s also possible to output to other digital formats, such as for digital cinema distribution, which generally maintains a high level of quality.

A digital intermediate pipeline may also offer other advantages for output, such as the capability of adding watermarks or encrypting footage. In the long term, no clear solution exists that specifies which storage method is most suitable for archival purposes, although a suitable method will likely evolve.

The next chapter focuses on ensuring the output has attained an acceptable level of quality, covering the types of problems that can arise, as well as methods that can be used to correct them.