1
The Digital Intermediate Paradigm
The phrases “never been done before,” “pushing the envelope,” and “attempting the impossible” crop up rather often in the film industry, particularly in post-production. But the methodology used to finish HBO’s “Band of Brothers” mini-series in 2000 involved a completely new work flow, justifying the frequent use of these comments to some extent. Every scene of the production was shot on 35mm photographic film (with the exception of a number of computer-generated visual effects and a few seconds of 16mm footage), but by the time of completion, the entire, edited series existed only as a bunch of data stored on a number of hard disks and digital tapes. This wasn’t anything new in itself. Disney’s Toy Story had been an entirely digital production, and the film Pleasantville involved scanning and digitally color-grading a cut film to a set of digital files, before recording back onto film.
What was unique was the unprecedented level of control made available to the filmmakers. The series’ editors, who were themselves working with separate video references of all the film footage, were able to make adjustments to their cuts, and see the changes implemented into the final version almost instantly, the existing sequences rearranged, and any new material added. At the same time, the cinematographers and directors would oversee the color correction of the production interactively, at the highest possible level of quality, complete with visual effects and titles, and synchronized to the latest audio mix. They had the ability to jump to any particular scene and even to compare the look of multiple scenes to ensure continuity. Further, the level of control of color grading enabled a unique look that would be extremely difficult to obtain (some might say impossible) using traditional photochemical processes. Meanwhile, other departments composited titles, dissolves, and other optical effects with a similar degree of interactivity and control, and others used digital paint tools to remove defects such as dust and scratches from the scanned images, resulting in images of extremely high quality.
This process wasn’t completely smooth-running: a lot of things didn’t work as theorized, a great many things broke completely, and some days it seemed that we were attempting the impossible. But many lessons are learned the hard way, and envisioned ideals soon gave way to practical fixes and workarounds. And to some degree, that philosophy still prevails throughout this rapidly growing digital intermediate industry.
For this reason, the aim of this book is not just to present a technical discussion of the theoretical possibilities, the “wouldn’t it be good if everyone did approach”; the aim is also to cover current working practices, and explain why certain things have to be done the way they are—even though it can seem inefficient or irrational at times—and, of course, to list possible methods to improve them.
There is also a website that accompanies the book, which details new advances and changes in the industry, as well as points to other helpful resources. Find it online at www.digitalintermediates.org.
1.1 What is a Digital Intermediate?
If you have ever worked with an image-editing system, such as Adobe’s ubiquitous Photoshop (www.adobe.com), then you are already familiar with the concept of a digital intermediate. With digital image editing, you take an image from some source, such as a piece of film, a paper printout, or even an existing digital image (if using a digital camera for instance), bring it into your digital system (by using a scanner or by copying the file), make changes to it as required, and then output it (e.g., by printing onto photographic paper or putting it on the Internet). Your image has just undergone a digital intermediate process, one that is in very many ways analogous to the process used for high-budget feature films (albeit on a much smaller scale).
The digital intermediate is often defined as a digital replacement for a photochemical “intermediate”—a stage in processing in which a strip of film (either an “interpositive” or an “internegative”) is used to reorganize and make changes to the original, source footage prior to output—and is often regarded as only applicable to Hollywoodbudget film productions. It is true that the digital intermediate fulfills this function; however, it potentially fulfills many others too. First, any material that can be digitized (i.e., made digital) can be used as source material, and the same is also true of output format. For this reason, it is somewhat arrogant to presume that digital intermediates are only used for film production, as the digital intermediate paradigm has already been used with video production for many years. Second, the budget is something of a nonissue. More is possible with a larger budget, less compromises need to be made, and a higher level of quality can be maintained. But footage that is captured using consumer-grade DV camcorders as opposed to 35mm film cameras is still a candidate for a digital intermediate process. Possibly one of the most interesting features of the digital intermediate process is that it can be used for almost any budget or scale. So, perhaps for a better definition of what a digital intermediate is, it’s a paradigm for completing a production by digital means, whether it’s the latest Hollywood epic or an amateur wedding video.
1.2 Digital Intermediates for Video
Video editing used to be a fairly straightforward process, requiring two video cassette recorders (VCRs), one acting as a player, the other as a recorder. Shots are assembled onto the recorded tape by playing them in the desired sequence. This simple, yet efficient “tape-to-tape” system was excellent for putting programs together quickly, but it limited creativity and experimentation because you couldn’t go back and change something at the beginning without re-recording everything again. Video editing was a linear process (unlike film editing, where editors could happily chop up bits of film at the beginning of a reel without having to rework the whole film).
Figure 1–1 With a digital intermediate pipeline, it’s possible to work with a number of different media
The nature of video also meant that creating effects, such as splitscreens or dissolves, was possible but required dedicated hardware for each function. Suddenly a fully equipped video-editing suite was crammed with various boxes, each of which required specific training to use, not to mention cost to install and maintain.
With the advent of nonlinear editing systems, computer technology brought a digital intermediate of sorts to the video world. Rather than record video from one tape to another, it was captured onto a computer system, which then was used for editing in a nonlinear fashion, allowing video editors to work with far greater flexibility. Once the virtual editing (or “offline” editing) had been completed, there were two possible routes for output. In some cases, the details of the edit could be sent to a linear system, and the original tapes were used to recreate the desired cuts in what was termed the “online” edit. Alternatively, the offline version could be output directly to tape from the digital system, complete with dissolves and other effects. Over time, the possibilities grew, such that it became possible to perform sophisticated color-grading of footage, along with numerous other effects. Now, much of the same technology is available to the amateur videographer as to the professional.
1.3 Digital Intermediates for Film
Back in the “golden era” of film production, perhaps thousands of feet of 35mm negative (the same stuff you put in a nondigital stills camera) would be generated every day of a shoot. All this film would have to be soaked in numerous chemical baths, and then it would all get printed onto more film—this time 35mm “positive” (or “reversal”) film. This process allowed a light to be shone through, projecting the image onto something (e.g., a cinema screen) in the correct color for viewing.
So, at the end of shooting a production, you would have maybe a million feet of celluloid, at which point would come the editors to sort through it all, which had to be done by hand. But because the negative is so fragile and yet so valuable (each bit of original negative retains the highest level of quality and represents all the set design, acting, camerawork, and lighting invested in it), there was constant risk of damage to it, particularly during editing but also during duplication and printing.
More copies of the negative were created for the editors to work with, and once they had decided how it was going to cut together, they would dig out the original and match all the cuts and joins they made (and they really used scissors and glue to do it) to the original.
So now there would be a cut-together film, which consists of hundreds of valuable strips of film, held together by little more than tape. Woe unto any filmmaker who decided they wanted to make changes to it at this point.
Other problems still had to be dealt with. First of all, you don’t want to keep running a priceless reel of film through a duplicator every time you want to make a cinema print (especially bearing in mind that you might need to make some 10,000 or more prints for a distribution run). Second, different scenes might have been shot on different days, so the color would need adjusting (or would require being “timed”) so that the different colors matched better.
Hence the internegative and interpositive copies of the original, cuttogether film. Creating these additional reels allows for color timing to be done too, and so the result is a single piece of film with no joins. So now everyone is happy, even after running it through the copying machine 10,000 times.
Well, almost. See, the internegative is of significantly lower quality than the original, and all the color timing is done using a combination of beams of light, colored filters, and chemicals, which is a bit like trying to copy a Rembrandt using finger paints. But until the digital intermediate process gained a degree of authority, filmmakers (and the audience) just lived with the limitations of the photochemical process.
Things improved a little with the advent of nonlinear video editing. It basically meant that rather than having to physically wade through reels of film to look at shots, copies of all the film could be “telecined” (i.e., transferred) to videotapes and edited as with any other video production, finally matching the edit back to the individual strips of film. But many of the quality issues remained, and filmmakers were still unable to match the degree of stylization achieved in video production without resorting to visual effects (and at significant cost).
At least, not until the advent of the digital intermediate process for film came into use, offering significant advantages over the traditional, optical process.
1.4 The Advantages
A digital intermediate work flow provides lots of wonderful advantages to a production. The most immediately obvious is that in addition to its nonlinear nature, it also offers “instant seek”—the capability to jump to any point in a sequence without having to spool through the footage to get there (this was one of the features that drew many consumers to adopt DVD technology and dump VHS videos). As well as saving time, digital intermediate work flow has practical implications: two shots at different points in a program can be checked easily for continuity.
It offers a more flexible approach. Changes made to the production can be automatically synchronized with the digital version, so that if, for example, changes to the edit are made, the digital version reorganizes (i.e., reconforms) itself to match the new edit. Rather than editors having to manually search through reels of film or hours of video, they can match digital files to offline edited versions using information such as timecode or keycode numbers. In addition to the nonlinear editing capability, almost every aspect of the digital intermediate process is nonlinear, and so productions can be completed in a more convenient order, rather than having to be completed chronologically.
Digital systems enable a degree of intelligence to be imparted to the images used, meaning that images can be tagged, sorted, and indexed as needed. Specific digital processes can provide a form of image analysis, allowing objective quality measurement, filtering to increase sharpness or reduce noise, and enabling options such as feature tracking.
Digital images are just numbers, and so may be copied accurately and easily. Unlike analog formats, digital media doesn’t suffer from “generational loss,” meaning that each copy is a perfect duplicate of the original. This single fact means that a superior level of image quality can be maintained throughout the digital intermediate process, allowing for greater experimentation and disaster recovery (as backups of the original images can be made without risking damaging them). With film and video processes, even with a tightly controlled work flow, the final version of a production would already have undergone a number of copies (or generations) and be inferior in quality to the original.
In many situations, digital image processing can be much faster than its analog (nondigital) counterparts. Many operations, such as resizing or repositioning images happen almost instantly, compared to analog systems that may require setting up complex optical systems to accomplish the same thing. This speed allows for a great deal of interactive feedback, particularly with color grading, where changes to the color of an image can be adjusted by turning dials and the like, and seeing the changes instantly reflected in the full-quality version. This affords the filmmaker a much greater level of creative control and more freedom to experiment with different options (or to make tweaks).
Finally, digital systems have the potential to add increased security and protection to stored data. Use of encryption and digital rights management technology can ensure that only authorized people have access to the images in the first place. This can be used to help combat piracy and to prevent the making of unauthorized copies, as well as to allow a greater number of (authorized) people to access the production with ease.
Is the digital intermediate system a perfect one? Not by a long shot. Anyone who has used a computer for five minutes instantly becomes an expert in all the things that can go wrong—accidental file deletion, hardware failures, software crashes, and so on. Now, take into account that a typical digital intermediate needs around 150,000 individual frames, which typically amounts to 1.5 terabytes (which is 1.5 million megabytes) for film material, and you don’t have to have be a professor to realize that things can and do go wrong. Having said that, as time goes by, the associated equipment is getting faster and more reliable, and more people are using it, which means that it’s also getting easier to use. There are many possibilities for exploiting this technology that haven’t properly been explored, and the potential is certainly there to create productions far exceeding even the quality of film.
In addition, parts of the process tend to be very slow. Data transport in particular can take significantly longer than video systems, and many digital systems require time-consuming rendering and checking processes to be factored into the overall completion schedule.
There are also issues that arise from a general lack of standardization throughout the industry, such as the problem of the best method of long-term storage. Many of the potential advantages of a digital work flow are dependent on the system they run on, meaning full-quality playback, or any of the other bells and whistles of a rival system may not be available due to performance and other factors.
Finally, data is not an inherently visual medium, and so it’s not as easy to look at a disk to work out the contents as it is to, say, play a tape in a VCR or put a strip of film on a lightbox.
1.5 The Flexible Approach
A digital intermediate is not just for feature films. It is a paradigm that can be used on any number of source and destination media, be it, film, video, or digital. There are already a number of established digital intermediate pipelines that aim to solve different problems.
1.5.1 Telecine
Getting film transferred to video has been necessary for many applications for a number of years. For example, creating video releases to coincide with theatrical releases of films (for rental or sale) previously required converting the film to a video format. This is accomplished by using a “telecine” process.
A typical telecine process involves using a dedicated machine to convert photographic film directly to video. Although analog telecines exist, they have recently all but been phased out by digital ones. Nevertheless, the basic idea is the same: in goes the film, out comes video. The limitation is that this is something of a linear process, in that the film must be output in the same order as it is fed in.1 Moreover, although it’s possible to modify the output to change the color components of the images, these modifications must be done in real time and may therefore be more limited and more expensive to implement than those provided by a digital intermediate process.
Using a digital intermediate process, digitizing the film to data, such as by using a scanner instead of a telecine, the scanned film may be processed in whatever way is necessary, and then the modified data may be output to video. This makes all the features of the digital intermediate available, such as the ability to remove dust and scratches from the film image, to reorganize scenes or shots, and to apply color grading in a nonlinear fashion. Moreover, multiple reels may be processed together, in any order. The downside is that the digital intermediate process may ultimately take longer than a direct telecine approach.
Figure 1–2 A typical telecine work flow
Figure 1–3 Adapting the telecine process to a digital intermediate pipeline
The usual method for finishing a feature film is to telecine all the “dailies” (film footage produced during shooting, sometimes called “rushes”) and then import the resulting video footage into a nonlinear editing system until a final cut is produced. This final cut is then matched back to the original film negative. It is then cut together (a process referred to as a “negative cut,” or “neg-cut”) to match the final edit before being color-timed and duplicated for distribution. More recently, an additional telecine process of the edited, colortimed film is used at the end to generate video deliverables for television, airline, and consumer release.
A digital intermediate work flow streamlines this process somewhat, but there are numerous ways of doing it. Ideally, the rushes could be scanned rather than telecined, with digital copies sent to the editing team (or videos output from the digital scans). The edited version could then be sent for digital color timing, digital restoration (e.g., to repair dust and scratches on the scanned images), and effects. Finally, the finished, digital production can be output back onto film and simultaneously to other formats, such as video or streaming Internet media.
Practicality (not to mention expense) places some limitations on this idealized pipeline, however, and the reality is that the production is normally edited before any scanning is done, using telecined rushes (as with a nondigital pipeline). The edited version is used to determine which footage needs to be scanned (rather than scan everything), and to help automatically conform the scanned material to create the desired program in a digital form.
Figure 1–4 A typical film post-production pipeline
Figure 1–5 An ideal digital intermediate work flow for film post-production
Much of the problem with a digital intermediate approach is that the volume of data required to complete a single film is enormous, which in turn means it can be slow to transport and process and can require a lot of storage space. In addition, color-management issues have to be addressed: film is very different from video in terms of color and dynamic range, and so accurate color management is essential, particularly when outputting to both film and video, and especially for ensuring that digital color timing looks as it was intended.
Figure 1–6 A typical digital intermediate work flow for film post-production
1.5.3 Video Production
Video production already makes use of a digital intermediate pipeline almost exclusively, with the exception of linear (tape-to-tape) video production. The simplest (and ideal) method is to digitize all video footage, edit and process it digitally, and then output it directly back to video, as used by many broadcast news organizations. This is not only fast, but it’s also inexpensive and allows the production to enjoy all the benefits of the digital intermediate work flow, such as effects and color grading.
In some instances, a slightly different approach is needed, such as when working with the slightly unwieldy high definition (HD) video formats. Under these circumstances, it may be more efficient (and less expensive) to use the offline/online approach, such that the editing process is done “offline,” typically at a lower quality than the original source, focusing on cutting individual shots and sequences, rather than on the specific effects to be applied. The final cut is then saved as a list of edits (or EDL), which is used to assemble the original, full-quality material during the “online” editing stage, where the necessary effects and image processing is applied, and the sequence is output.
Figure 1–7 A linear video-editing work flow involves making edits sequentially
Figure 1–8 A digital intermediate work flow allows edits to be made in any order
Figure 1–9 An online/offline digital intermediate paradigm
The only real downside to using a digital intermediate pipeline in this way, as opposed to a tape-to-tape system, is that additional time may be needed to digitize video material before it can be used. Even so, many productions are able to even eliminate this disadvantage by performing the digitization during the initial recording of the source material.
Figure 1–10 A video-to-film work flow
1.5.4 Video-to-Film Conversion
A digital intermediate process is essential to taking video footage and outputting it to film. Generally speaking, video footage is of inferior quality to film.2 However, it’s sometimes necessary to print video images onto film—for example, to show televised commercials in cinemas or for short films that may have originally been shot on a video format. By far the best way to do this is to digitize the video footage, process it, and then record it onto film. Several digital processes may be necessary for this to happen successfully, including resizing and repositioning the images (as video is usually too small to fill a frame of film), as well as color-space conversion (to correct for color differences between video and film), and possibly de-interlacing them (to correct for video motion artifacts).
1.5.5 Live Production
Provided that the data can be processed fast enough, it may be possible, or even desirable to use a digital work flow for live broadcasts. In the simplest instance, this may mean a “webcam” that processes and outputs footage directly to the Internet, potentially without even keeping a recorded copy of the footage. At the other extreme, such as for sports event coverage, video footage can be captured, processed, and output as quickly as it is received, allowing the use of digital effects to be applied, such as a digital zoom, text, or some form of filtering. However, such a system is prone to digital defects such as image artifacts or corruption because there is not enough time to correct or repair such problems. In addition, the entire system is limited to digital-processing techniques that can be applied faster than the images are received.
Figure 1–11 A digital intermediate work flow can be used for live productions
1.5.6 Digital Production
It is entirely feasible to work with footage that originated in a digital form, created on a computer system. In fact, this process is used to create completely digital productions such as Disney’s The Incredibles and Dreamworks’s Shrek. Scenes are generated within a virtual environment and are rendered to produce digital footage. This digital footage can be referenced within the digital intermediate pipeline exactly as if it were digitized video or scanned film, with a multitude of processing options available.3
As new shooting methods gain prominence, such as recording video footage directly to digital media, the available processing options will increase, as will the ease of use and integration of a digital pipeline. Note that audio does not factor into any of these pipelines, though it will represent a significant part of most productions. This is because audio content tends to be treated separately from visual content and is only combined with visual elements at the output stage.
Figure 1–12 A completely digital production provides many possibilities
A digital pipeline enables the filmmaker to have an unprecedented degree of creative control over the visual content of a production. The most obvious benefit is the significant amount of control with respect to color-grading digital images. Whereas nondigital grading is usually limited to adjustments that affect the overall content, digital grading allows precise control over the color of individual elements within a scene. It’s a lot like the difference between a chain saw and a scalpel.
Creative options are not just limited to altering the color content however. Numerous digital filters are available to stylize footage further, as well as correct problems or enhance camerawork. All of these effects can be controlled as precisely as the color-grading process, using masking and keying techniques to isolate specific regions within the images, and tracking and animation techniques to adjust each effect over time.
Use of a digital intermediate pipeline is not necessarily a guarantee of quality, as there are many factors to consider, which will be covered throughout this book. Done properly however, a digital pipeline can ensure that the production is processed at the highest possible level of quality, and that this level is maintained through to final output.
Copying a digital file creates a duplicate that is mathematically identical to the original. It doesn’t matter if the file in question is a webpage accessed from the Internet or a complete digital feature film copied between file servers; the content, and by extension the quality, remains the same.
This is different than copying other media, such as film and video (and to some degree, digital video), which inevitably undergo some degradation (both physically and in terms of the diminishment of signal strength) with each copy.
Figure 1–13 With film formats, each new copy degrades the image further
Figure 1–14 With video formats, each new copy degrades the image further
Figure 1–15 With digital formats, each new copy is identical to the last
1.8 The Path to Digital Cinema
Perhaps what will prove to be the biggest selling point of the digital intermediate pipeline is that it offers the easiest transition to digital cinema. Digital cinema (or D-Cinema) involves the projection of images and audio in a theatrical context from a digital source. The aim is to provide a cinema experience that maintains a level of quality and flexibility that at least matches traditional cinema without increasing the associated cost. At the present time, the vast majority of cinemas do not have digital projection capability, and so output for digital cinema is rare. But as this changes (which is expected within the next few years), the need to output to a digital format will grow, and this need will be fulfilled easily through use of the digital intermediate process.
1.9 Summary
The digital intermediate is a paradigm for completing productions, able to suit a number of different work flows and requirements, and universally conferring a greater degree of creative control and flexibility, maintaining a higher level of image quality, and allowing simultaneous output of many different formats.
Regardless of the specific work flow required, all digital intermediate pipelines share various key components that are covered in subsequent chapters. Chapter 5 covers the requirements for bringing image and other components into the digital realm. Chapter 6 describes how to manage the volume of data used by a typical pipeline, while Chapter 7 deals with ways to assemble the data into meaningful shots, scenes, and programs. Chapters 8, 9, and 10 examine the creative options available to a digital pipeline, in terms of color grading, retouching, and effects, respectively. Chapter 11 looks at the options available for outputting the finished, all-digital production, while Chapter 12 covers ways to ensure a high level of quality. Chapters 13 and 14 look at the future of the digital intermediate and ways to better take advantage of the available features.
The next few chapters will focus on the three main media involved in production: video, film, and digital images.
1 There are methods for getting around this limitation, but they involve video editing to some degree.
2 Many estimate that HD video is of equivalent quality to super-16mm film however.
3 In fact, such a pipeline affords even more options, particularly as computergenerated images can be encoded with additional image information, such as relative depth and separation of different elements into layers.