When a creature is very close or small, then it may be useful to place ping-pong or tennis balls on rods. These would be placed at the eye position. This has the advantage of supplying not just the eye line for the actor but also something to motion track in postproduction. The small size means the rendered CG creature will cover the reference and not require painting out of the rig.

For The Mask (1994) a real clock was hand moved on the set to represent the CG animation to be added later. This was not only a reference for the actor and director but also for the technical directors and digital lighting artists who could use the real clock as a lighting and material guide and for the animators to use the motion at least as a rough guide.

Shooting of reference materials is useful for the digital lighting artist to check the lighting and the material response on the actual set or location. This might be a square of fur, a piece of plastic, or a piece of cloth.

Tips

1.  A C-stand¹⁹ can be used to hold a reference or monster stick if it’s not moving. A spud²⁰ may be taped to the back of a foam core cutout to allow mounting to a C-stand without damage. Heavier references (such as made out of plywood) may require a movable light stand.²¹

2.  Cut holes in any large flat reference if it’s to be used outside. The wind turns any large flat object into a kite that may be blown down.

3.  For plywood references it may be good to have hinges to make it more compact when packed. A latch or lock mechanism would be needed to keep it from folding while in use.

4.  Foam core can be easily damaged when moving it around from location to location in different weather conditions. Make two of any foam core references and handle them with care. A large box that can hold the reference is recommended.

5.  By slotting the base of a full-size foam core cutout, a cross piece of foam core can be used to hold up the cutout when it’s placed at 90 degrees in the slot.

6.  A foam core cutout can be made of both the profile and straight-on views of the creature (head or full body). Each of these is slotted with one slot going halfway up and the other going halfway down. Once the two pieces are slid together at the slots, the cutout now represents both the side and the front view of the character. This provides a simple physical 3D representation of the character.

7.  Remove the reference from actual shots when possible. When that is not practical, then keep an eye on any action behind the reference. Moving actors or other motion behind the reference will make painting out the rig even more difficult.

8.  Try to keep the mounting system as small as possible since it will likely have to be painted out in some shots. Actors frequently give better performances when a reference is in the frame. This should be discussed with the director depending on the reference required and the specific actors involved. The budget should incorporate the rig removal in the shots.

9.  Mark the monster stick for common heights (standing, bending over, etc.) to make it quick to change.

10. Mark the monster stick with alternating 1-foot bands of black and white or place bright tapelines every foot. This makes it quick to check the height on the set and in the filmed reference.

11. Record the monster stick or reference height information along with the other matchmove set data. If a surveyor’s transit is used, then it’s worth recording a point on the monster stick for reference.

12. The image of the reference can be outlined with a grease pencil on the on-set video monitor so it can be checked when shooting even if the reference has been removed. It’s also possible for the video operator to do an overlay of the reference to check for problems with eye lines or placements. This can be useful when laying the running reference footage over the current shot to determine if there were timing issues.

13. Do not dress the crew member holding the reference in a blue or green outfit unless the shoot is against a colored screen. A colored suit is likely to have dark shadows and highlights that make it difficult to extract a matte so the reference will likely need to be painted or rotoed²² anyway. The other problem is the colored material will likely cause colored bounce onto the actor or set in the scene that will require some work to remove in post. When possible the crew member and the reference should be dressed or colored to match the final creature (i.e., avoid a crew member dressed in white next to an actor if the final CG creature will be very dark).

14. The eye convergence of the actors will change with their focus. Normally this isn’t a problem if the creature is large or more than 10 feet away. If the actor is supposed to be interacting with a creature 2 feet away but in real life the actor is focused on an object 30 feet away, then this can be a problem if both eyes of the actor are clearly visible. The actor may appear to be talking to something beyond the CG creature. Having a small reference in frame or having the actors memorize their exact eye angles will help.

15. Work out a consistent order of shooting references and be ready to do them as soon as the shot is set up.

ON-SET ANIMATION CAPTURE: WITNESS CAM

Joel Hynek

A witness camera setup generally refers to the use of one or more video, high-definition, or motion digital cameras on a live-action set for the purpose of capturing the motion of one or more of the actors while the production cameras are shooting. The preferred method is to use two to four cameras to capture true 3D information. After the shoot the action from the various camera perspectives is tracked and converged into one set of 3D motion data.

Capturing animation by tracking only the view from the primary production camera is, of course, possible but it is not as accurate, and an animation with conflicting information can result. For instance, because it’s not possible to capture distance from only one point of view, the lower part of a body may resolve as being farther away than the upper part of the body when in fact it is the same distance.

It is appropriate to use one or more witness cameras when there is a need to capture more motion information than can be obtained from the production’s primary or secondary taking cameras.

Generally witness cameras are used to gather 3D information. However, they can also be used as an aid to 2D tracking when the view of the subject from the taking camera is partially obscured or in and out of focus. In these cases a sharp unobstructed view of the subject from a nearby witness camera can provide the track (this assumes that either the taking camera is static or that it can be stabilized).

Wireless Nonvideo Motion Capture

Systems are available that can motion capture on set without video cameras but they require that the actor wear inertial motion sensors or optical motion sensors and usually wire tethers to return the information. This kind of on-set motion capture is beyond the scope of this section. Please refer to the discussion of motion capture in Chapter 4.

Factors Affecting Witness Cameras

Position and Setup

Witness cameras should be placed about 60 degrees apart. The height of the cameras is not critical but it is good to place at least one of the cameras at a different height (at least a foot) than the rest so as to get a greater range of view. Obviously, the most important thing is to place the cameras so that they get a good view of the subject. Placing cameras so they get an unobstructed view and do not interfere with other set equipment is often not easy.

Generally speaking, the witness cameras should be fitted with as tight a lens as possible to keep the action contained within the frame. However, panning with the action is also possible as long as the witness camera move can be tracked afterward.

It’s important to align/register the cameras to each other. Some common objects on the set itself can be used for this as long as the dimensions of the common objects or features on the set are known. It will make tracking and converging the data later much easier.

Camera Synchronization and Phasing

Keeping the witness cameras in sync is a must in order to capture coherent data. This is best accomplished by using a common time code and common sticks. It is not necessary for the witness camera sticks to be the same as those of the production cameras but it is good if at least one of the witness cameras also captures the production sticks and slate for each take. This is so an “offset” can be captured, making the syncing of the witness camera information to the actual filmed plate easy.

Time Code

Time code can be driven from the production’s time code generator or an independent time code generator employed by the witness camera team. This is accomplished with either a hard wire going to each camera or a remote device such as a “Lockit Synchronizer” that is jammed from the time code generator at the start of the day and which in turn genlocks each of the cameras so that they all have the exact same time code.

Keeping the cameras in phase is also a must if extreme precision is required. Letting the cameras run out of phase is not the end of the world but it does require the imagery or tracking data to be time shifted back into phase if higher accuracy is needed. If the nonprimary witness cameras are being used to just occasionally confirm a 3D position rather than locking every frame, then letting the cameras run out of phase and not time shifting back into phase will probably be acceptable.

Camera Resolution

The higher the better is the rule. Generally high-definition (HD) cameras are required. Production-quality cameras like the Viper or Red Camera are great as well as some prosumer-type HD cameras. Certainly a test should be performed to confirm camera quality, functionality, and result. In the case of the Red Camera or other complex cameras, it should be pointed out that they will require more training than a prosumer-type camera and cost more.

Actor Preparation

Putting markers on the actors wherever necessary is important. Markers on body joints work better than markers placed in the middle of a limb. Also, using Velcro with different colored patches is good. If the filmed image of the actor, either part or all, is to be used in the final composite, then it may be necessary to minimize the markers or choose a color that is close to flesh tone (if the markers are on flesh) so as to make it easy to remove in post. At some point it is good to take stills of the actor with a ruler or calibration structure to record the dimensional placement of the markers.

Crew Preparation

To keep the set running smoothly it is important to brief all concerned, especially the ADs, that there will be cameras and operators all around the set and that VFX will need to shoot their own sticks.

Generally, members of the visual effects crew operate the witness cameras. They need to be able to keep the action in frame, in focus, and properly exposed and to stop and start the camera. Sometimes the responsibility of running the witness cameras can be taken on by the assistant cameraman. This will reduce the size of the overall on-set crew and relieve the data acquisition team of the task. However, it is best to dedicate one person per camera. This also ensures that the camera doesn’t get bumped during the process.

Dealing with the Data in Post-Production

The first step in post-production is to edit and line up all the takes from the witness cameras. An editing system like Final Cut Pro works very well because it has a feature that will automatically line up all the takes based on their time code. It also affords a four-way split screen so that up to four cameras can be viewed and confirmed at once.

The witness camera’s images, once tracked, are then converged using a 3D package such as Maya. Either custom software or a plug-in to Maya are required to semiautomatically converge the tracking data into one coherent animating 3D rig.

Conclusion

Witness cameras are a great way to capture the 3D motion data of actors or moving objects while they are being filmed on a live-action set. It is like having a motion capture system on set except that the result of the capture cannot be viewed simultaneously in real time as on a motion capture stage. On-set discipline, adequate preparation, and time in post are required to successfully realize captured 3D motion data from witness cameras but the results can be totally satisfying.

REAL-TIME MATCHMOVING AND CAMERA-TRACKING DATA

Charlie Clavadetscher

Real-time camera tracking refers to a process in which the motion of the stage camera, and possibly other key objects involved with the recorded image, is invisibly followed or tracked through some technical process. This information is sent to a computer that has 3D data about the shooting location as well as computer graphics data for the intended effect for the shot.

The computer, or other technology, is able to combine these elements and generate a simplified video version of the intended final visual effect. This generated visual effect correctly follows along with the camera motions as the shot is photographed. This is roughly similar to the way some video games and cellular phones are able to take the user’s motion and generate different imagery based on that motion. Although this comparison helps explain the concept, the comparison is limited because real-time visual effects for production require far greater precision, preparation, participants, and many other considerations and advanced technologies.

During production, the real-time visual effect (combined with the live-action camera image) may be viewed on video monitors, potentially through the camera eyepiece, and may also be viewed by stage personnel who trigger specific events such as a door opening, a window shattering, or other specialized action.

Seeing synchronized, generated effects at the time of photography may be of significant help to the camera operator, the director, the DP and possibly to the actors, among others. Real-time visual effects may provide a better understanding of a scene and guide it to a more completely realized, fine-tuned, and approved version of the final visual effect shot instead of shooting a scene that is potentially blank and may otherwise be difficult to visualize.

At this time, such technology is still in the early stages and has not yet been developed for truly invisible use or use that does not conflict with other production priorities, such as lighting.

However, the benefits of seeing correctly scaled and tracked visual effects as the scene is photographed should be apparent.

To avoid confusion with virtual cameras, please note that the real-time visual effects described here only pertain to traditional stage situations, such as dressed sets, and traditional filmmaking locations, such as a forest, beach, street, etc. In these situations, a motion picture camera or equivalent works in the real-world environment, and real-time visual effects are combined with those real-world scenes. In contrast, virtual cameras, such as those found on motion capture stages, typically involve a very different physical situation. While some aspects of camera tracking may be similar, many of the conflicts and shooting considerations are very different.

Real-time camera tracking data and real-time visual effects have two main applications:

1.  To provide a simplified but representative rendered composite scene that can be viewed by cast and crew members as the scene is shot (real time). Ideally, the camera is run as on any set; however, the camera operator and others are able to see the simplified version of the visual effects, such as sets and creatures, rendered on the fly and shown in sync with the live-action background.

2.  To provide data and reference to be used later in the visual effects pipeline to speed up CG processes and to assist CG object and camera tracking.

Of the two uses, the first is by far the most attractive. The ability to preview visual effects on stage and in real time has enormous appeal to directors, actors, and crew. It may be very helpful in finding the most effective framing and blocking of the shot, particularly when important elements in the shot are imaginary. Being able to watch a camera rehearsal with a reasonable approximation of the effects in place helps everyone better understand the scene and react accordingly.

Real-time tracking records all physical camera motion and lens information: xyz motion, pan, tilt, roll, plus zoom and focus. With this data, the camera motion can be recreated in a CG environment which is then seen through monitors. While the final visual effects shot typically includes the original actors and any live set combined with finalized and realistic CG effects, real-time visual effects, for the foreseeable future, must be simpler and limited to the most important CG elements in the scene.

Some technical issues must be dealt with to get a precise and consistent match and provide acceptable and successful realtime visual effects:

•   Noise in capture systems leads to random motion jitter. Large amounts of noise can make real-time visual effects swim or vibrate compared to the background. This type of problem can change the real-time visual effects from something attractive to something incomprehensible.

•   Lens magnification can change with focus: for example, when focused close, a 50mm lens may become 54mm.

•   Lens distortion can vary greatly across the frame.

•   The roll axis of the camera system must be accurately located;

bad roll data can invalidate all other motion.

Good tracking data is only the beginning. Having the correct path is at best only about one-third of the problem. Tracking the camera motion in relation to existing objects in places on the set is the other critical part of the process necessary to lock visual effects into the set. In a fully decorated set, it’s often impractical to locate every object, but some key measurements will be invaluable to providing the foundation for this process. Fortunately, greenscreen and bluescreen sets are usually highly simplified and well suited to this process.

For example, imagine a large empty stage with a green screen. Assume the stage has a green sloping floor, stairs, platforms, and other green features on which the actors will walk, stand, or interact. With the dimensions and location of those features known plus known camera data, simplified visual effects can be related to the space and inserted into the camera’s image as seen on a monitor or other device. Example technology for real-time tracking was used in the film, A.I. For specialized shots that used real-time visual effects, a multitude of special tracking markers were attached to the stage ceiling. A device on the production camera used these markers to track camera motions.

A.I. only used one process and this was a serious problem. The methods described next were not used on the film and are examples only. Other proposed real-time technical processes may use special markers on the camera itself, similar to the way a motion capture system operates. Both of these solutions, markers on the ceiling for A.I. and proposed markers on the camera alone, require additions to the production camera and changes in the stage that may have a direct impact on what is possible for lighting, stage design, and other considerations.

Other technical solutions and proposals involve using transmitters or sensors on the camera, with subsequently less equipment in the stage area. Such systems may use a technology similar to a GPS system or a technology that uses sensors on the camera to detect the camera’s motion. Other proposals include ways to track the actual image coming from the camera video tap²³ and are therefore more passive in nature.

These are just examples of the types of technology that may be considered, though each may require additional equipment on stage or impose limitations on other aspects of the production process, such as lighting. Also, as productions change from indoor stage to outdoor locations, it may be necessary to use more than one system to lessen negative impact on production while still achieving the desired real-time visual effects.

Some practical considerations include the following:

•   It’s vitally important to have the production crew, and especially the camera crew, accept the technology which is used to track the camera and other parameters. Crews have a lot to deal with when performing normal photography, and there is plenty of equipment attached to the camera already, so any additional cables and technology must be minimal in order to be accepted and successful in a production environment.

•   Motion tracking equipment should be low profile and low impact. Likewise, even if on-camera tracking equipment is lightweight, antennas or emitters sticking out in several directions can interfere with the crew’s work. Likewise, LEDs, both visible and invisible to the naked eye, can create problems.

•   It’s also important to have equipment that can be quickly (as in “instantaneously”) transferred from one camera to another, and once attached, won’t come loose. It’s ideal to have several sets of instruments already attached to any cameras and provide a means to electronically switch from one camera to another, rather than physically transfer connections from one to another. Any switch and initialization from one camera to the other must also be quick if not completely invisible.

•   Visual effects personnel should be prepared for the possibility that multiple cameras may need to be tracked on certain shots, or a decision must be made if or when only one camera can have real-time visual effects.

Other on-set equipment may not be friendly to the tracking electronics, and it’s critical that any real-time visual effects system likewise not interfere with existing production equipment and processes. The dense thicket of crew radio and video assist broadcasts, magnetic fields, RF interference from computers, stage grid work and building materials, hundreds of amps of lighting equipment, and even truck ignitions can pull down an untested system. It should go without saying: Test any system extensively in the real world before using it for real work!

The stakes and pressure can increase dramatically if the entire day or week of shooting is dependent on unfailing synchronization of computer graphics images with human-driven cameras for some critical or climactic scene or sequence. The systems are complex and the possibility of failure is high. Halting production for an hour to work out some technical glitch would likely be seen as a disaster.

Just a few examples of problems would be systems based on line of sight or ultrasonic or RF transmission, which suddenly becomes confused or unusable if blocked with flags, cast or crew, or other equipment. (This is why these systems require redundant information capture.)

Also, appropriate computer graphics models and computer graphics animated characters must be developed, approved, tested, and ready to go into existing real-time live-action scenes. This can require a substantial real-time visual effects crew before shooting begins as well as on set, and it may also demand flexibility and control over animated CG elements that may not match the camera or actor’s expected timing. The amount of preparation and support crew may become much more demanding, and the magic of real-time visual effects may become cumbersome, unexpectedly expensive, and complicated compared to original expectations.

Also, keep in mind that real-time visual effects yield two possible recorded variations for every shot, and both may be required for immediate playback and archiving. These two recordings would be:

1.  the normal video recording of the set without any real-time visual effects, which duplicates what is photographed by the production camera, and

2.  the background with the real-time rendered visual effects as a separate recording.

This puts a heavier burden on stage video personnel to record and play back both versions of each take and likely later in production editorial. There are a number of situations where the director or others will want to review the take without the visual effects, for instance, to focus more on the actor’s performance or anything else that might have been obscured or unclear by the insertion of the real-time visual effects. Similarly, on stage at the time of photography as well as later, people will want to view the recorded shot with the real-time visual effects.

For instance, during the editorial process, it may be advantageous to have either the clean plate or to choose the recording of the same take with simplified real-time visual effects. Normally editorial would be working from the images transferred from film or the original digital images. However since the on-set composites only exist in video form it would have to be integrated into the editorial process as needed. This possibility and other variations should be discussed with stage and production personnel before shooting begins, including editorial and other members of the production process who may be affected by having multiple video versions of the exact same take.

Certain technical difficulties and potential added complexity can arise when using real-time camera tracking and real-time visual effects. However, as technical problems are overcome and reliable, low-impact systems become commonplace, realtime visual effects and previewing could have an important positive effect on set and for the overall production process, and may even play a role in test screenings. Like storyboards and the camera video tap, it may become an important aid in understanding what will be seen in the final shot, and realtime tracking systems may become one more tool to expand options and give more freedom to the creative process and imagination.

TRIANGULATION AS A METHOD OF RECORDING CAMERA DATA

Stuart Robertson

Photorealistic visual effects require all elements within an image to share common image and lighting characteristics and, most importantly, a common perspective. Since the photographic perspective is entirely dependent on the spatial relationship between the camera and the subject being photographed, it is essential that this relationship be recorded and applied systematically to the photography or computer generation of each element that goes into the finished composite. When shooting the image that establishes the perspective for the shot (usually the background plate), the visual effects crew has the responsibility for recording the camera position data and then applying that data to the additional elements (greenscreen plates, miniatures, CG animation, etc.) to be combined in the final composite image.

Although much of this positional data can now be derived after the fact from software tracking and other computer applications, on some occasions this work is still most easily accomplished manually, for instance, when the appropriate computer application is not available. In particular, when multiple physical elements are to be laid out (green screen, miniature, etc.) the specifications for such a layout will have to be created, either manually as described in this section or specifically from software. Knowledge of the manual method is especially important if multiple elements will need to be shot during live action with no time to scan in the film, matchmove it on a computer, and send back all the data in a usable form. Therefore, it is worthwhile to review the traditional methods of manual camera data recording, and to offer a supplementary procedure that increases the accuracy of such data. This approach provides a simple method of re-creating the camera/subject relationship for additional photographic or CG element capture.

Camera/Subject Positional Information

This is the data that guarantees the matching of perspective for all elements in a shot. Positional data can be derived using a very simple tool kit, and should be recorded relative to the visual effects nodal point of the camera lens. The triangulation method discussed here provides an accurate and convenient approach for recording and applying this positional data.

Basics: The Tool Kit

The tools needed for manual camera data recording are simple ones, most of which have the great advantage of being powered by gravity: a universally available and unvarying reference for vertical measurement—regardless of scale. The tools shown in Figures 3.26 through 3.29 are the minimum necessary. Most of them are available at any local hardware store.

Figure 3.26 Measuring tapes (two are required, three are preferred). (All images in this section are courtesy of Stuart Robertson.)

Figure 3.27 Inclinometer (aka angle finder, angle dangle, etc.) A professional sighting inclinometer is shown at right.

Figure 3.28 Plumb bob and line (one required, two preferred).

Figure 3.29 Laser level (not essential, but very handy).

Basics: Nodal Point

Camera data should be recorded relative to the camera’s visual effects nodal point²⁴, not from the film plane. Although current practice tends to favor data collection from the camera’s film plane, measurement from the nodal point is more immediately accurate and will translate more accurately between differing lenses or image formats or between photographic and CG environments. It should be clearly noted in the data record whether the measurements are taken from the nodal point or from the film plane.

Figure 3.30 Visual effects nodal point.

The visual effects nodal point is considered to be the center of rotation and the perspective center for the image-forming function of a lens. Therefore, the position of the nodal point of any given lens will yield an equivalent perspective for any other camera lens, of whatever focal length, whose nodal point is placed in that same position.

And, because the infinitely small lens of a CG camera is by definition nodal to know the relative position of the nodal point of a camera lens is to know the relative position at which to place a CG camera in a CG environment.

The degree of accuracy needed in determining the nodal point for data acquisition depends on the circumstances of each shot, varying inversely with the distance between the camera and the subject. For extreme close-ups (miniature or table-top work, for instance) the distance between the camera and the subject is very small and accuracy of all measurements is critical, so the nodal point needs to be determined very accurately, using techniques discussed elsewhere in this handbook.

At greater distances—6 to 8 feet or beyond—an approximation is usually sufficient (although this must always be a considered judgment based on the nature of the shot). A rule of thumb that has proven reasonably accurate takes advantage of the fact that the entrance pupil of a complex lens can be found at the apparent depth of the image of the aperture stop as seen looking into the front element of the lens. A visual estimate of the apparent depth of the aperture stop within the lens barrel will give a working location of the approximate visual effects nodal point, when an exact determination is unavailable or less than entirely necessary.

Tutorial: Camera/Subject Positional Information

The following tutorial demonstrates the procedure for recording and applying manual measurements of camera/subject relationships for a typical visual effects shot. In this typical case, the tutorial will deal with greenscreen characters walking along a path in an interior location but will also mention general considerations that would apply to other types of shots, such as the placement of a miniature building element within an exterior location. The tutorial will follow the desired shooting order for visual effects plates, first shooting the background plate and then shooting the elements (in this case, the greenscreen characters) that will fit into this background.

Figure 3.31 Background and greenscreen plates.

Step 1: Establish a Baseline

Once the camera has been placed to give the desired framing for the background plate, the visual effects crew will need to determine a baseline from which measurements will be taken. Determining an appropriate baseline requires careful judgment. A baseline must be long enough to provide accurate triangulation and must be chosen so that it will have a meaningful and measurable spatial relationship to common elements within both the background subject and the element plates that will be photographed to fit into it.

For instance, if one wanted to replace the front face of the building shown in Figure 3.32 with a pyrotechnic model, a sensible location for a baseline would be the lower front edge of the building face. This would be a common measurement for both the actual building and, if scaled, for the miniature.

For the tutorial example, the chosen baseline is derived from the path that the character will take as he walks through a hallway. A 20-foot tape measure is placed into the set to mark this baseline, and temporary tape markers are placed at either end of the measured baseline (these markers will be removed before shooting begins).

Figure 3.32 Baseline location on typical building.

Figure 3.33 Baseline established in background plate on set.

Step 2: Mark the Station Point

Drop a plumb line from directly underneath the nodal point of the lens to the floor below the camera, and place a square of tape to mark that spot. Designate this point (marking the camera’s plan position on the set or location) as station point (X).

Figure 3.34 Station point—plumb line from nodal point.

Step 3: Create the Data Plan and Triangulate the Camera Position

Draw an overhead view floor plan of the setup, including the station point (X) and the baseline (this plan view does not need to be drawn to scale).

Figure 3.35 Data plan with baseline.

Figure 3.36 Data plan with baseline and triangulation.

Measure the length of a straight line from the left edge (L) of the baseline to the station point (X). Record this distance as distance LX. Record the distance from the right edge (R) of the baseline to station point (X) as distance RX.

Because of the rigid nature of a triangle, these measurements establish the only possible plan position that the camera could occupy relative to the baseline—and therefore relative to the subject.

Step 4: Record the Camera Height, Tilt, and Dutch Angle (If Any)

Using an inclinometer placed on an appropriate surface of the camera or camera mount, record the tilt and dutch (roll) angles of the camera. Assuming that the surface on which the camera rests is level with the height of the baseline, obtain the camera height by taking a direct measurement of the vertical distance from the camera’s nodal point to the floor.

Figure 3.37 Camera height, tilt, and dutch.

If the baseline is not at floor level (in table-top photography, for example), one will need to offset the camera height to reflect the camera’s height relative to the baseline instead of relative to the floor.

Figure 3.38 Camera height offset for baseline height.

Step 5: Determine and Record the Camera Pan Angle Relative to the Baseline

The measurements so far have recorded the camera’s height, tilt, and Dutch angles, and the camera station point relative to the chosen baseline. This information is enough to re-create the position of the camera’s nodal point and, therefore, the perspective of the shot (see following diagrams).

Figure 3.39 Station point established relative o baseline.

However, even the widest lens is sampling imagery from only a small portion of the panorama potentially visible from the position of the nodal point.

The data collected so far gives no information about the horizontal direction in which the camera is actually pointing. In fact, one could point the camera in the exact opposite direction from the original pan heading, and the data we’ve collected would still be valid. The only wrong thing would be the image.

Figure 3.40 Camera field of view (one of an infinite number of views).

Figure 3.41 Right station point, wrong image ....

Figure 3.42 Camera crosshair establishes the center of field of view.

To deal with this, one must have a way to record the horizontal angle by which the camera was pointed toward the subject—the azimuth or horizontal pan angle relative to the subject. Magnetic com-pass direction is notoriously unreliable due to local magnetic influences especially on set or in interior locations. A better strategy is to use the features of a professional camera itself as a survey instrument to establish the pan angle relative to the baseline.

Looking through the camera at the triangulated set, notice that a typical professional camera viewfinder has crosshairs that delineate the center of the image, as shown in Figure 3.42.

By dropping a plumb line from the center of the crosshairs one can establish the center of the camera frame as a point along the baseline. Mark this point with a temporary piece of tape or a chalk mark as and delineate this point as viewfinder center (V).

Measure the distance from the right end of the baseline (R) to the viewfinder center (V), and record this on the site plan as distance (RV).

Figure 3.43 Plumb line to mark viewfinder center (V) on baseline.

Figure 3.44 Record viewfinder center (V) on baseline.

To re-create this configuration on the greenscreen set, pan the camera so that the image center as shown by the viewer crosshairs will be directly above (V), thereby ensuring that the camera pan angle is correct. The camera data plan is now complete.

Figure 3.45 Data plan—completed.

Step 6: Apply the Positional Data to the Greenscreen Element

The first step upon beginning element photography is to reestablish the baseline—in this case, a 20-foot tape placed on the green floor. Viewfinder center (V) should be marked on the baseline at this stage.

Locate the camera station point by swinging two tape measures, set to the distances (LX) and (RX) and pivoted from the left and right edges of the baseline. The point at which the two tapes meet marks the station point (X ).

Having established the station point, preset the camera to the heights and tilts previously recorded on the data plan. Placing the camera so that the nodal point is directly above station point X (the plumb line comes in handy here), pan the camera so that the center image crosshairs are directly above viewer center (V), and make adjustments as needed to reset the height and tilt to match the data plan.

Figure 3.46 Baseline reestablished on greenscreen stage.

Figure 3.47 Station point located from baseline.

Figure 3.48 Data plan applied to greenscreen stage.

Proceed to shoot the greenscreen action, confident that the greenscreen elements will fit into the space of the background plate with little or no adjustment in post. (Don’t forget to remove any temporary markers before shooting.)

Figure 3.49 Rough comp without adjustment.

Additional Notes 1: Scaling

Direct scaling of the data plan provides simple and accurate measurements for scaling of elements—in this case, for 2:1 scaling of the greenscreen characters (who will be positioned along a 10-foot baseline).

Figure 3.50 1:1 and 2:1 data plans.

Figure 3.51 2:1 greenscreen plate and rough comp.

Additional Notes 2: Uneven Ground Level

More often than not, the visual effects crew won’t be lucky enough to be working from a level floor while shooting background plates—especially on outdoor locations. A more typical case might involve shooting with a low camera tilted upward, on ground that slopes unevenly downward away from the camera. Because the intention in this instance is to place a greenscreen character on the first stairway landing, a baseline has been established at that height—above the camera.

The problem is to determine exactly how far the camera is below this baseline—the negative camera height.

Figure 3.52 Baseline above the camera.

Figure 3.53 Side view—baseline above the camera.

This is a situation in which the laser level provides the simplest solution to the problem. Direct measurement from the visible line provided by the leveled laser gives the relative heights of both the camera’s nodal point (Height LLC) and the baseline (Height LLB). Subtracting LLC from LLB gives the measurement of how far the camera is below the baseline—the measurement needed when setting the camera height below a greenscreen platform for the element photography.

Figure 3.54 Heights measured from laser level line.

However, if the crew does not have access to a laser level, or if the distances and lighting conditions make it impossible to use one, one can obtain the information needed by using the camera itself as a surveying instrument.

The visual horizon is defined by the height of the observer, so the line of sight of a level camera is also a reference for the height of objects above and below the horizon and therefore the height of an object above or below the camera.

If one levels the camera and sight through it, one can obtain a visual horizontal reference shown by the viewfinder crosshairs. This of course would be done after completing the shot.

Figure 3.55 Camera leveled to determine camera height—view from camera viewfinder.

Direct measurement along the front edge of the subject, from the baseline to the spot on the subject on which the viewfinder center rests, gives the precise measurement of the distance by which the camera is below the baseline.

Figure 3.56 C camera leveled to determine camera height—side view.

PHOTOGRAPHIC REFERENCE

Charles Clavadetscher

Oftentimes the individuals or teams capturing data will want to shoot photographs or video that visually document some aspect of the shooting situation, which may then be easier to understand later in the post-production process. These reference images provide a method to capture a very visual, active situation on set where written notes and diagrams won’t or cannot convey clearly what can be seen in a few pictures.

This type of generalized reference photography is primarily helpful when specialized image and data gathering, such as for photogrammetry, textures, or lighting, fail to show some significant or unique factor. It can provide a better human-oriented angle of the set, or it may better emphasize specific components that other processes miss.

First, whenever possible, shoot a slate indicating that these images are for general or specific reference, and not for established areas, such as for lighting or matchmoving. Include the shot, sequence name, set name, and other relevant data (which should always be on a slate). When possible, make some note on the slate about the purpose of the reference shots. For instance, “flame rig and actor’s position reference.”

When shooting reference photographs, one way to highlight or direct the post-production artist’s attention to a specific issue or detail is to shoot two nearly identical photographs in a row of the same scene or setup. The first picture is clean, showing the set without any special marker, while the second has a visual aid to direct the viewer’s attention to a particular area in the image. Switching between the two images should make it clear what is being pointed out or noted.

For example, the first image may be the set and actor’s position as an overview. The second image, shot with the same view, may have the photographer’s finger pointing at the production film camera, which may be in a unique position, such as in a pit in the floor or under a table and therefore hard to see. The slate for these two photographs should say “camera location and actor’s position reference.”

Or the photographer’s pointing finger may indicate the location of a special effects pipe that will emit water, but is otherwise hard to see and identify on a complex set with lots of other equipment. By literally pointing out the location of the pipe, others who were not on the set may find this reference image useful when trying to identify and work with the source of water in the production image.

Although it may seem somewhat crude, the technique of shooting one clean picture and a second picture with a pointing finger can be quickly accomplished by a lone photographer, and nothing fancy is required in terms of crew cooperation or the photographer’s time or equipment, so the process has no impact on the production set itself, nor does it create obstructions. Plus, having only 2 photographs keeps the number of images low compared to having 5 or 10 images designed to highlight something from different vantage points and hoping the end viewer will figure out what’s important.

The same technique can be used to help point out a person or object of special interest, a particular rig, unusual camera position, camera start and stop positions, a hidden camera, or a handheld camera that may be difficult to find in a single overview image. The almost limitless number of possibilities that can occur when shooting on a set, particularly for visual effects, can easily be pointed out using this simple technique.

As an alternative to the photographer’s finger, someone else in the image may do the pointing, or a special marker could be placed as required. The problem with both a second person or a marker is that both require additional personnel or preparation. It may pose a problem for the production crew to have additional people or markers on the set, even temporarily. In contrast, a lone photographer can stand away from the set and use the pointing finger technique to passively and quickly record the item of interest and its location in relation to the rest of the set and then move on to more traditional data gathering.

When shooting references, keep in mind that on a busy set with many people and tasks, drawing the attention of a later viewer to one point of interest might be harder than expected. Whatever means is used to gain the attention of the post-production artist needs to significantly stand out and clearly indicate the exact detail of interest.

In some circumstances, a third photograph may be needed to act as a record of general changes or details. For instance, on a set that has wild walls, it may be hard for someone coming along later to understand the setup. In this case, a third photograph can help tie this together. The third photograph might be the set with and without the wild wall, taken from the exact same position, essentially showing a before and after situation. Or the third photograph may be of a set blueprint or diagram, and a finger points at exactly the same spot in the diagram as the finger in the second image. The common element of the pointing finger makes the purpose clear by guiding viewers to exactly what’s important and ignoring what’s not.

Don’t forget to get feedback and do tests for this type of reference image. On the spur of the moment, it may seem like a good idea to shoot overview photographs of the sets. However, the artists themselves, looking at the images months later with no frame of reference, may only see a jumble of people and equipment, and the image is no help whatsoever. Worse, additional but useless images and data just add confusion to the other reference data and overall process.

Shooting Video as a Reference

Shooting video for reference sounds like a great idea for tying loose threads together, putting a helpful perspective on the production shot, or recording some minute detail in real time as the camera rolls. However, video reference can quickly transform into hours and hours of unusable and incomprehensible material.

This is not to say that video is without its use, but many factors need to be kept in mind in order to make video valuable:

1.  First and foremost, get a slate shot for the video, preferably at the head. If this video pertains to a particular scene and/or sequence, make sure that is on the slate. Also it doesn’t hurt to verbally read the slate as the video is slated since the video camera also captures audio, and it might make a hastily written slate easier to understand. Slates should always have the current date, time of day (to coordinate with other time-of-day information), and production shot number matching the production slate used to film/record the scene. Additionally, this slate should contain unique identifying information, such as the set name or location name. Because video is being shot for a particular purpose, it is always a good idea to put this on the slate as well as record that verbally.

2.  The second priority is to use video precisely. This suggestion cannot be stressed enough. The most common reason video becomes useless is that by the end of the production there are hundreds of hours of video that someone must search through, log, copy, edit, organize, and/or perform other arduous and repetitive tasks on to make it usable. Plus, because the person organizing the video may not be the one who shot the video, the purpose and end usage may not be clear and may even be outright impossible to understand later. As a result, a lot of time spent shooting video has no benefit because it lies unused by the artists, misfiled, or thrown away because of the time-consuming, frustrating, and often impossible task of organizing or deciphering the video.

Another issue is the willingness and ability of the artists to roll through the video themselves. In many cases, an alternate to video, such as photogrammetry, or other standard data gathering, would provide a more precise method of capturing the same information and would prove to be more useful and easier to access.

In this sense, shooting precisely is the key to successful video, or in other words, shoot just enough to get the job done and keep it organized

The idea of putting a video camera up in the rafters and having it run for the whole day sounds enticing—especially to people who never had to deal with the end results before. Now imagine 90 days of stage video that need to be sorted through. It could take 90 days just to watch it!

No wonder it gets tossed or deemed a waste of time compared to whatever benefit there might be. It would be cheaper to hire someone to operate the video camera on set, turning it on during the appropriate times and off as soon as its use is ended—all in order to reduce the overall mass of video to as little as possible. This reduces or prevents hiring someone later to sit and sort through video for days at some later date.

Another problem with video arises when it is intended to be in sync with production motion picture footage. One of the biggest problems in this scenario is the difference in standard video frame rate compared to film. Standard North American video is 29.97 fps and film is 24 fps. Even if video is somehow brought in sync with the film, there is a perceptible mismatch between the video image and the film and interlacing issues to consider. To correct this, if possible, use a video camera that shoots true 24fps, and test it under shooting conditions to make sure there is a solid, reliable system of sync throughout the production process, including editorial. The reference video and film images should be known, accessible, and usable to the post-production artist.

The other critical synchronization problem occurs when the visual effects artists get a small section of the photographed scene as the final cut, while the original video from stage contains the entire take from beginning to end. If the video needs to be synchronized correctly with the production footage, a solid and absolutely reliable system must be developed to get the video in sync with the much smaller cut shot.

Without this type of information, trying to synchronize a long video with a short take turns into an error-prone and expensive guessing game. While it may seem obvious on set as to which portion of the scene is connected to which motion, someone else who wasn’t on set and is trying to figure this out later can face a frustrating, hopeless endeavor.

From all this, video may sound like a losing proposition. Yet, in some cases, and in support of some visual effects, it may be the ideal solution. For example, having video that is shot 90 degrees off the main production camera, or above it, may resolve some complex motion and timing issue that is otherwise open to guesswork or time-consuming tests and approval.

Those types of specific examples are covered in the earlier sections on witness cameras and matchmoving. However, other types of reference videos may be useful, such as walking through a set to explain some unusual setup, how a specialized rig was set up or used off camera, or how a wild wall moves in and out.

Using video for this type of reference is something that needs careful thought, preparation, and testing (when possible), before shooting, in order to get results that can be used as intended, and to make the effort worthwhile. It would be a mistake to have hours of video walking through sets. Make it concise; make sure it is both needed and useful and delivers significant information for later use. Given that visual effects data gathering has limited personnel, time, and equipment, choosing video for support reference, over or in combination with some other method, should be carefully considered and tested before production begins and before other methods are discarded in favor of video. In some situations, survey, scanning, or still images may be a better way to gather reference data or images. It bears repeating that shooting video which is never watched or can’t be put in proper sync is not a good use of time and resources.

However, if captured correctly and in a way that is easy to manage, reference video may be just what is needed. But always be aware that video is unlike other data capture because it is a real-time process, generates many potential post-production processes and organization issues, and can overwhelm users and artists if it becomes confusing or cumbersome or doesn’t seem to connect to the specific artist’s process.

Overall, general photographic reference, either still image or video, provides a means to visually capture and organize production setups, data, and conditions that may fall outside the realm of other processes, such as lighting data capture. Recognizing this before production begins, discussing alternatives with acquisition personnel, and setting up procedures and guidelines can make this a flexible alternative to support the CG post-production process.

RULES, SETUP, AND TESTING

Charles Clavadetscher

As in many jobs, preparation is the key to success. Preparation helps prevent delays or interference with other tasks of the production/shooting process and helps all work proceed smoothly, working with the other departments to the completion of each shot.

As such, the job of acquiring visual effects data from stage or location begins well before the production starts shooting, beginning in pre-production, so that the first production shot is not a testing ground, which could waste time, cause confusion, or cause other problems on stage.

This is similar to nearly all other departments, which begin their work before the first shoot so they are fully prepared when shooting begins. For instance, the costume department goes through extensive fittings, visual tests, alterations, and fine adjustments of the actor’s costumes, which are then ready and fitting properly when shooting begins.

In a similar vein, visual effects processes should be tested, rehearsed, and evaluated before the first day of shooting so that the data collection process proceeds as smoothly as possible, with the least impact or drag on production. The visual effects process should present a professional, nonconfused appearance to the rest of the production crew, which also ensures that the best results are obtained. Preparation for data collection helps avoid errors and conflicts on set, as well as errors in the collected data itself.

It is therefore a good practice to engage in a testing process in pre-production, in order to achieve successful and economic data collection for the visual effects process.

Before equipment is purchased or the cameras start rolling, the visual effects supervisor or on-set data wrangler(s) need to answer two basic questions:

•   Who gets the collected data?

•   How do they want it recorded and delivered?

Who gets the collected data breaks down into three separate people or entities:

1.  The first person would be the VFX Coordinator/Editor/Supervisor or visual effects facility. Even if the person collecting the data has worked with a facility directly before, it’s good to double check that previous data formats are still up-to-date and viable for the new production. Check to see if any improvements or changes can be made in how to record, capture, or deliver the data. If nothing else, new employees, or people in new positions, such as new supervisors or data handlers, may have different priorities compared to previous projects.

2.  The second “who gets the data” question is answered by which department(s), and which techniques, of the visual effects process are the target for the data capture. Are the camera notes, such as focal length, intended only for matchmoving, or will they also be used for modeling, texture capture, stage/set extension, lighting reference, general photogrammetry, or other processes? Each user of the same data may require specific details or an alteration to the data collection process that another user would not.

If the gathered data is for multiple visual effects processes, then clear priorities, as well as clear steps and expected results need to be defined and approved as early as possible. This must happen at the very least before production begins. Different processes have widely divergent data collection requirements, and it is common that one simply cannot reliably gather all of the data for all departments for any given shot. Setting established goals for the area(s) each person is covering depends on clear communication and priorities established during preproduction testing.

3.  The third “who gets the collected data” question would be answered by identifying specific individuals at any facility who will receive the data, any intermediary individuals who might sort, collect, or process the data, and finally the end users, or at least a lead or supervisory member or representative of each area.

Understanding the three parts of “who gets the collected data” provides the foundation for the tests and connections necessary to ensure that the collected data is in the correct format and is usable. It also establishes the correct priority and expectations and ensures that communications have been opened between the different areas and specialists to discuss any special requirements, potential problems, or additional issues—all before production is started.

Do a Complete Test Shot!

Before arriving on set for the actual production photography, it is good practice and highly recommended for each member of the data collection process to perform multiple examples or test shots. The object of this exercise is to cover a variety of situations. Also, working on a group of shots helps uncover any problems arising from overlapping data on multiple shots, such as will occur during real production.

This may mean having test or example shots created in a nearby building or other location that represent reasonable approximations of the expected sets or locations.

To put this in perspective, consider the following scenario, wherein a very simple mistake in standardized practices created massive havoc and wasted time and energy in post-production. This is a mistake that could have been easy to avoid.

It is common practice in production to use both head and tail slates in shooting motion picture footage. For a head slate, the slate is held upright and reads correctly as the film is projected or viewed. A head slate indicates that the information on the slate applies to the shot that is following that slate. A tail slate is always turned upside down, indicating that the slate information applies to the shot before the slate’s appearance. Additionally, a false start (without a slate) may be indicated by the camera operator holding his or her hand close in front of the lens.

The same rules apply to both still and video photography used in the visual effects data capture process.

In this example of the process gone wrong, during the rapid pace of production, the data gathering on set often used head slates, tail slates, and the hand in the frame techniques. Unfortunately, the individual accepting the raw, unsorted data was unaware of the standardized slating techniques listed above.

As a result, all the “hand” frames were deleted because it was assumed these were mistakes or irrelevant. Furthermore, the upside-down slates (indicating tail slates) were confusing and hard to read, so the slates were rotated to be easily read.

Problems started from the first day of using the collected photographs. Lighting reference images and their slates didn’t match the plates. Modeling and photogrammetry pictures were mixed up or seemed nonexistent, even when submitted notes indicated the material had been shot. It took weeks to figure out the problem and even longer to correct it. One person in the chain had reorganized the data based on ignorance, costing time, money, and expedience.

This example also illustrates a simple rule of data handling, which is “never, ever manipulate the original data.” If part of the process involves editing, reorganizing, or in any other way manipulating the original captured data, then first make a copy of the raw data and edit that, and leave the original version alone. A copy, even if it is much larger, disorganized, and potentially confusing, is left unaltered as insurance and a way to get back to a clean start.

Why Run Through Example or Test Shots?

Test shots help identify and avoid errors as in the preceding example. Test shots reveal the CG pipeline’s weakest links so they can be corrected before problems occur with a real shot. The rule of test shots is that every aspect must be handled as if it were a real shot, by the same people, using the same process. It is an invitation for disaster to have one group handle or accept the test shot data and another group use the real production data. Real shots handled by different people may unwittingly use different processes, skip key procedures, or proceed with completely different understandings, expectations, and results compared to those who receive the real, final production data.

Running through a multiple-shot test process may reveal that the slates have both good and bad data. They may have the needed information, but be too small, poorly lit, cut off, or illegible (for instance, from bad handwriting). Or the slates may simply need rearranging for better understanding, sorting, and easier access by the final users.

Camera report forms may need to be simplified or expanded. It may turn out that the data collection equipment doesn’t output the expected results or doesn’t connect correctly with other data. For instance, “up” may be z in one system and y in another.

An example of an equipment setup problem would be a still camera that isn’t set up for or capable of shooting the expected format. The camera may be recording single 24-bit JPEGs of the shot when the visual effects team expected 48-bit RAW images in a bracketed sequence. Or it may be that the camera and lens have unacceptable limitations or cropping compared to another preferred camera.

These sorts of problems may apply to other equipment as well, and the list of possible mismatches or missed expectations is quite lengthy. Running through multiple test scenarios will discover and correct these problems before production begins, and this testing procedure will fine-tune the equipment, the process, the people, and the expectations.

Complete testing also ensures that the data collection process will not suddenly change as production progresses. A change such as this could be complicated or confusing to stage personnel and the non-visual effects components of production, and it may be awkward or embarrassing, or changes to the data collection processes may even be impossible to accommodate in the middle of production once procedures have been established and planned.

Keep in mind that after running through one series of tests, it is prudent to run through them a second time to ensure that all the changes have been incorporated or accounted for correctly. That means doing more than one set of tests. At a minimum, two test shots or test sequences are needed to uncover and correct errors. Additional tests may find further problems or issues that need correction, resulting in yet another test sequence. The point is, do not go into production without knowing all of the expectations, techniques, and expected formats, and do not go into production without a system that has successfully produced a final set of test data that everyone agrees is good. If this can’t be produced, there is a large chance errors will appear in the collected production data, and the cost will be many times greater than the cost for one additional test.

Comprehensive tests like these, including the review and correction process, can take well over a month. While some people may initially object to this process, skipping it is a guarantee that the data collected will be troublesome, likely very expensive, possibly unusable, impossible to fix during production, and may take weeks to correct, if it can be corrected at all.

This bears repeating: Do not skip the testing process, which ensures that the data collection process is set up properly and will yield successful and economic data collection for the visual effects process.

DIGITAL CINEMATOGRAPHY

Dave Stump ASC, Marty Ollstein, David Reisner

Shooting with a digital camera has advantages and disadvantages. This section introduces the issues that a VFX Supervisor should consider when planning to work with a digital camera.

Collaboration between the VFX Supervisor and the Director of Photography (DP) is always important in production. However, when the image is recorded on a digital camera, that collaboration becomes essential. The choice of camera, recording medium, image format, and workflow all affect the quality and nature of the work of the VFX Supervisor and the success of the visual effects that are produced. The possibilities and limitations that each decision entails should be fully understood before starting work in the digital realm.

The goal in choosing a camera is to provide the highest quality image capture that the production can afford. The VFX Supervisor is required to be both artist and engineer in researching the broad spectrum of eligible cameras, while staying fiscally responsible to the production.

Digital Definitions

•   Digital cinematography refers to production shot with any digital camera on any digital medium.

•   High definition refers to high-resolution digital cameras that have at least 1920 pixels horizontally or, for higher quality releases, 2048 or more pixels horizontally (which minimizes image resizing).

•   High definition, in television and occasionally in cinema, refers to 1920 × 1080 cameras that record Rec. 709, subsampled, 8-bit color.²⁵

•   Digital cinema or d-cinema is the official Society of Motion Picture and Television Engineers (SMPTE) nomenclature for referring to feature motion pictures (and potentially other digital uses) distributed digitally and shown on digital projectors that meet SMPTE and DCI (Digital Cinema Initiatives²⁶) specifications/requirements.

Dynamic Range

Dynamic range quantifies the recording latitude of a device or medium. In cinematography it is measured in f-stops. Modern color negative films have a dynamic range of 11 to 12 f-stops of latitude.²⁷

In digital sensors, dynamic range is defined as the range of signals from the strongest to the weakest signal registered by the sensor. The dynamic range of the sensor is considered to be the difference between the photonic (electronic) noise in the system with no signal present, and the maximum signal the system can record without clipping or distorting the signal. This range, or latitude, of the sensor is measured in either units of decibels²⁸ (dB) or f-stops.

The signal-to-noise ratio (S/N) performance levels in digital cameras are stated in decibels to compare baseline noise to top signal power. To meaningfully understand the dynamic range of a photographic sensor, it is essential to define an acceptable level of noise at the lowest end of the scale. The level of noise selected will directly affect the image being recorded. The top end of the S/N equation is much easier to define. Even the novice can quickly learn to identify signal clip on a waveform monitor.

Noise is a factor in all digital cameras. Digital cameras record images in a very strict and methodical way. When presented with a quantity of light, they produce code values that describe an exact color and hue.

Noise is inherent in digital cameras and shows up as a bothersome dither or shifting in an otherwise stable and unchanging image.²⁹ This dither frequently attracts attention to itself and detracts from the storytelling. In certain lighting situations at the low end of the range, when the amount of light striking the image sensor varies minutely from frame to frame, the resultant change in tiny portions of the image can seem to flicker and change color. This results in a tendency to produce odd and unexpected image artifacts. These problems can be very difficult, if not impossible, to remedy in post-production.

Highlight Headroom

Highlight headroom is a complex concept. Put simply, it concerns the point at which the bright areas of the image are arbitrarily cut off and therefore not captured. Film negative handles overexposure extraordinarily well, but the highlight capture characteristics of digital recording have a tendency to lose overexposed highlight detail much sooner in its exposure cycle.

Digital cameras reach their saturation point decisively. When the light well of a photo site reaches saturation, it reaches that point with absolute certainty and is said to be at clip level. More light arriving at that sensor during that accumulation cycle is likely to spill out of that light well into adjacent light wells. This results in bloomed areas of overexposure that cannot be reduced in size or color corrected in post-production. Knowing the inherent differences in highlight and upper dynamic range issues between digital cameras is essential to success in digital cinematography and to all visual effects that utilize digital cinematography.

Sensitivity and ASA Ratings

The sensitivity of a digital camera’s sensor to light is commonly stated as an ASA rating.³⁰ A digital camera with a higher ASA rating will usually perform better in low-light situations and contain less noise than one with a lower ASA rating.

Sensitivity and ASA rating are very important factors in any camera—film or digital. Film and sensors with lower sensitivity (lower ISO³¹/ASA speed) require more light and/or a longer exposure to achieve good color reproduction. Film or sensors with higher sensitivity (higher ISO/ASA speed) can shoot the same scene with less light or a shorter exposure, but that increased sensitivity usually comes at the cost of increased noise levels or film grain.

Color Space

Color space describes the range or set of colors that can be represented by a given system, encoding, or piece of equipment. When talking about a color space, the standard starting reference point is usually the CIE XYZ color space, which was specifically designed to encompass all colors the average human can see. Color space is an area where film still holds a meaningful advantage over digital technology at the time of this writing.

Numerous factors must be understood when studying color space in digital cinema. For example, digital camera color space depends on the frequency response of the sensor(s) and on the characteristics of the path that light travels to reach those sensors—including elements used to split or filter the light. Additionally, digital color space is influenced by which sensor data is recorded by the camera, how it is modified in the process, and which camera mode and recording format are selected.

In both digital and film production, what the audience sees is ultimately dependent on the characteristics of the distribution medium. For a film release, there is a choice of print stock. For a digital release, it is the Digital Cinema Package (DCP). For the home, the DVD could be standard definition or Blu-ray (high definition). The display device used also affects the final perceived result, be it a 35mm or 70mm film projector, a 2K or 4K digital cinema projector, or a consumer video display.

Figure 3.57 CIE 1931 RGB color matching functions. (From http://en.wikipedia.org/wiki/File:CIE_1931_XYZ_Color_ Matching_Functions.svg. Reprinted with permission from Wikipedia.)

Color Space As It Relates to Cinematography

When colors are acquired by a digital camera or displayed on a monitor, they are usually defined in an RGB (red, green, and blue) color space. This is an additive process of making color, in which red, green, and blue light are added together to create color from darkness. The colors are defined by the amount of the red, green, and blue primaries that need to be emitted to produce a given color. Different display technologies produce different primary colors and thus display a different palette (sometimes referred to as range or gamut) of colors. While film intrinsically provides color references, digital cinematography uses display gamuts with standardized primary colors to allow the mapping from code values to specific colors.

When working in multiple media or with multiple technologies, it is sometimes necessary to convert the representation of a color from one color space to another. This color space conversion has the goal of making the colors of the transformed image in the new color space match as closely as possible the colors of the original image.

Cathode-ray tube (CRT) displays are driven by voltage signals for RGB. Full-bandwidth RGB signals require a relatively large capacity for storage and transmission. Reduced-bandwidth encodings such as YCbCr and Y’CbCr can do much of the same job using less capacity, but with some compromises and restrictions.

YCbCr and Y’CbCr (and their analog counterpart YPbPr) are practical ways of encoding RGB images as approximations of color with perceptual uniformity, but at much lower bandwidth for transmission. Y’CbCr is used to separate out a luma signal (Y’) that can be stored with high resolution or transmitted at high bandwidth, and two chroma components (Cb and Cr) that can be bandwidth reduced, subsampled, and compressed for improved system efficiency. However, the use of this signal encoding system throughout the workflow can introduce substantial visible errors. Creating visual effects from subsampled images can be difficult and troublesome. For example, bluescreen or greenscreen mattes are more difficult to extract cleanly from chroma subsampled images.

The Calculus of Color Sampling

Current production digital cameras offer several different dynamic ranges for recording: 8 bit, 10-bit log, and 16 bit, to cite a few. Eight-bit-per-channel color can be simply described in this way: The sensor creates 2 to the eighth power values (2⁸ = 256) for each of three color channels, that is, 256 shades of red, 256 shades of green, and 256 shades of blue. When the three color components are combined to define the entire color palette, 256 × 256 × 256, the result is 16,777,216 colors, which is referred to as 8-bit color. This color palette is an outgrowth of the display capability of phosphor CRT monitors, and it is intimately tied to the era of the genesis of television. CRTs are based on an old, inherently limited, and environmentally unfriendly technology that is now being replaced by LCDs, gas plasmas, LCOS, D-ILA, and DLP technology.

Some implementations of RGB use 16 bits per component for 48 bits total. Using the same color primaries, this results in the same range of color, but with a larger number of distinct colors (and smaller/finer steps between two adjacent colors). This is especially significant when working with wide-gamut color spaces where most of the more common colors are located relatively close together, or when a large number of digital transform algorithms are used consecutively, as is common in digital intermediate work.³²

Color negative film is capable of rendering 12 to 13 bits per color component, or roughly 11½ stops of latitude. When scanned into the digital realm using today’s standard production 10-bit log representation, each RGB color record is given between 886 and 1000 usable values out of the 1024 code values available in 10-bit encoding. When the three color records are combined, the overall resulting color space yields (loosely speaking) 1000 × 1000 × 1000 colors, meaning there are close to 1 billion colors from which to choose. Those colors are based on the color gamut that film can record, which is smaller than the color gamut that humans perceive (as documented in the CIE spec).

The Issues of Color Subsampling

What is 4:4:4? The term 4:4:4 has become HD slang for images and systems in which the three color records are sampled at equal maximum system frequencies. In the era of digital acquisition, 4:4:4 is a bit of a misnomer. Denoting an image as 4:4:4 traditionally refers to a standard-definition RGB (and occasionally YCbCr) video image in which all three color components have been sampled at 13.5 MHz. Because HD images are sampled at 74.25 MHz they should actually be said to be sampled at a 22:22:22 ratio. This indicates that none of the three color records has been subsampled at a lower rate to achieve system efficiency (and to save money).

Then, what is 4:2:2? Some smart engineers figured out that they could save money on storage and transmission if they reduced the signal through compression. The human visual system is much more sensitive to variations in brightness than in color, so a video system can be optimized by devoting more bandwidth to the brightness component (luma or Y’), than to the color components (color difference values Cb and Cr). A 4:2:2 Y’CbCr scheme requires only two-thirds of the bandwidth that 4:4:4 RGB requires, thus saving money in storage and transmission. Luma is sampled at full frequency, but the color components are sampled at half that frequency and then scaled back to full frequency using the brightness (Y’) channel as a multiplier for viewing.

The reduction in color bandwidth from 4:4:4 to 4:2:2 may only be perceived as a modest visual difference when viewed on a consumer CRT, but it can create significant problems for the visual effects compositor who is trying to create a convincing result from a greenscreen shot for theatrical release. The chroma subsampled images of 4:2:2 almost always result in matte lines and bad composites when viewed on larger screens. When creating visual effects for the big screen, then, you should shoot 4:4:4 or, technically, 22:22:22.

Subsampling Schemes

The 4:2:1, 4:1:1, 4:1:0, and 3:1:1 schemes are all subsampling schemes like 4:2:2, only worse. They are used in prosumer and HDV cameras. While any camera can produce the next Oscar for Best Cinematography, it is safer to ensure success with actual quality than to just hope for it with economy. A good rule of thumb is to capture the image with the best color you can afford, especially if visual effects work is involved.

The term 4:2:0 is used in many ways. All of them involve one form or another of subsampling of the chrominance channels in space (horizontal and vertical), and, in some cases, in time. The 4:2:0 subsampling scheme is used in MPEG-2 (the DVD format), DVCAM, HDV and VC-1 (SMPTE 421M).

Another way to reduce or subsample the huge amount of image data generated by high-definition cameras is to use the mosaic pattern sensor layouts implemented in most single-sensor cameras. In 2/3-inch, three-chip cameras such as the Sony F900, each photosite on each of the three chips is sampled and assigned a code number that directly determines the color of the pixel. In single-sensor cameras, however, a mosaic pattern of filtered photosites is used to determine color. Each photosite is covered by a red, green, or blue filter in a pattern, and the samples of several adjacent photosites are combined to determine the color of each pixel.

One common pattern, used by the Arri D-21 and Red One, is the Bayer Pattern Mosaic Sensor Layout.³³ The Bayer pattern is a physical, sensor-level implementation of 4:2:0 subsampling. The manufacturers of some cameras that use 4K Bayer pattern sensors, such as the Red One, claim that they yield images with 4K resolution. However, that is extremely misleading. Bayer pattern single sensors (and any single-sensor camera) cannot sample color from co-sited photodetectors. A Bayer pattern sensor takes the output of four adjacent photosites to yield one RGB color pixel. This process yields an effective resolution that is one-half of the total number of photosites in the horizontal axis and one-half the total number of photosites in the vertical axis. Thus, a 4K Bayer pattern sensor can only yield 2K resolution. The fact that the camera system packages that effective 2K resolution into a 4K frame is irrelevant.

Resolution

Resolution is an area that is hotly debated in digital acquisition. HD cameras output a 1920 × 1080 pixel picture. In real-world terms, 1920 × 1080 pixels projected onto a 40-x-22½-foot movie screen would present pixels 1/4 inch square. It takes two horizontal pixels (one white and one black) to create one visible cycle, so the HD projection would yield 960 cycles of viewable information on such a screen. Viewing such an image at a distance of one and a half screen heights (33.75 feet) yields about 30.76 pixels per degree, and therefore approximately 15.36 cycles per degree of perceptible detail. The human eye is generally capable of detecting 40 to 60 (and in some cases up to 80) cycles per degree of detail. Images of 10 to 12 cycles per degree are almost universally perceived as blurry, whereas images of more than 20 cycles per degree are generally perceived to be sharp. This analysis demonstrates that the HD imaging system that operates at 15 to 16 cycles per degree is only minimally satisfactory for use in large-screen theatrical applications.

The widely used science of the modulation transfer function (MTF) considers that many factors enter into the effectiveness of any acquisition system. However, it can be roughly surmised that an image with 1920 pixels (960 cycles at 15.36 cycles per degree) is at least slightly inferior to the de facto digital cinema standard of 2048 × 1556 for Super 35 film (16.45 cycles per degree), which is inferior to 4K scanned negative at 4096 × 3112, which yields about 32.8 cycles per degree—a number more in accord with human vision.³⁴

Keep in mind that once an image has been rendered at 1920 × 1080, that is the greatest resolution it can have. A camera original recorded at 1920 × 1080 is still 1920 × 1080 even when projected at 4K.³⁵ Likewise, once an image has been sampled into 8-bit color, it cannot acquire any additional color information or bits. Sampling and resampling errors are irreversible. Data lost is data lost forever. Color information lost in resampling cannot be recovered. Line resolution lost in resampling cannot be recovered.

Chip Size in Digital Cameras

The desire to work with a sensor the size of a 35mm film frame has become a strong force driving development in the area of sensor design, and many of the issues of digital cinema revolve around the basic differences between sensor technologies.

CCD vs. CMOS

CCD (charge-coupled device) and CMOS (complementary metal-oxide semiconductor) sensors both capture images by converting light into voltage, but they have different ways of converting the voltage into digital code values.

CCD imagers generally offer superior image performance and flexibility at the expense of system size and efficiency, whereas CMOS imagers offer better integration and smaller system size at the expense of image quality and flexibility.

Frame Rates

Loosely speaking, three frame rate standards are used with film cameras in motion picture and television production: 24 frames, 25 frames, and 30 frames per second.

The range of frame rates for acquisition with digital cameras is another issue. Many digital cameras are locked into a limited number of choices that correspond to the standard frame rates of broadcast television. Most digital cameras accommodate the frame rates of 23.98 fps (24/1.001), 24 fps, 29.97 fps (30/1.001), 30 fps, 50 fps, 59.98 fps (60/1.001), and 60 fps. Unlike film cameras, which may support a very wide range of frame rates, the current technology available for digital cameras is woefully lacking in its ability to acquire images at full resolution and full color bandwidth at more than 30 fps.

The term 60i refers to 59.94 interlaced fields combined to create 29.97 frames, commonly known as 30 fps. This has been the standard video field/frame rate used for NTSC since its inception. The term 50i refers to 50 interlaced fields combined to create 25 frames. This is the standard video field/frame rate used for European PAL and SECAM television.

The 24p (progressive) frame rate is a non-interlaced format and is now widely adopted for productions planning on migrating an HD video signal to film. The true 24-frame acquisition allows for direct, frame-for-frame, transfer to motion picture film. Additionally, cinematographers frequently turn to 24p for the cine look even if their productions are not going to be transferred to film, simply to acquire the look generated by the frame rate. The slower frame rate creates increased motion blur and flicker, both of which are associated with a film look. When transferred to NTSC television, the rate is effectively slowed to 23.976 fps, and when transferred to PAL or SECAM it is sped up to 25 fps. The 35mm movie cameras use a standard frame rate of 24 frames per second, though many digital cameras offer rates of 23.976 fps for NTSC television and 25 fps for PAL/SECAM.

The 25p format is a video format that records 25 progressive frames per second. This frame rate is derived from the original PAL television standard of 50i, which acquires 50 fields per second. Whereas 25p captures only half the motion resolution that normal 50i PAL registers, it yields a higher vertical resolution on moving subjects. It is also better suited to progressive-scan output, such as a transfer to film. Like 24p, 25p is often used to achieve a film look.

The term 30p means 30-frame progressive, a non-interlaced format that records at 30 frames per second. This format evolved from 60i in much the same way that 25p evolved from the original 50i PAL standard. Progressive (non-interlaced) scanning mimics a film camera’s frame-by-frame image capture and creates a cinematic-like appearance. Shooting at 30p offers video with no temporal interlace artifacts, resulting in greater clarity for moving subjects.

The 50p and 60p formats are new progressive formats used in high-end HDTV systems. While not technically yet part of the ATSC broadcast standard, they are quickly gaining popularity due to the superior image quality they offer.

Film Camera Aperture Sizes and HD Sensor Sizes

Digital camera sensors range from the size of a 35mm negative to a small fraction of that size. These sensors capture and reproduce images with varying degrees of success. Generally, the larger the sensor, the more faithfully it can reproduce the scene it records.

Lenses made for the 35mm format are not necessarily good for use on smaller format cameras, and lenses made for smaller format cameras are not necessarily good for use on larger format cameras. A lens made for a 35mm size sensor was designed and optimized to cover a much larger (and mono-planar) target surface area than that of a 2/3-inch sensor, and as a result, will almost always underperform when used with a smaller sensor. Still lenses made for full-frame 35mm film still photography will almost always underperform on cameras with smaller sensors.

Another issue is exclusive to three-chip digital cameras. The manufacturers of 2/3-inch three-chip cameras deliberately set the three color sensors at different flange depths from each other. Red light is notoriously hard to focus on mono-planar sensors, so the manufacturers of 2/3-inch three-chip cameras set the red sensor at a very different distance from the lens node than the green and blue sensors. Aware of this adjustment, the manufacturers of lenses dedicated to 2/3-inch three-chip cameras (such as the Zeiss Digi-Primes) expend great effort to focus red, green, and blue light at the particular distances set in the cameras, while bending each color of light to mitigate the resulting effects of chromatic aberration. Because of this, a lens made for a 2/3-inch three-chip camera would most likely exhibit very bad chromatic aberration if used on a larger mono-planar sensor.

Much has been discussed about the differences between the depth-of-field characteristics of 35mm mono-planar sensor cameras and 2/3-inch sensor cameras. In fact, their depth of field is subject to relatively simple mathematical formulas. The depth-of-field characteristics of 2/3-inch lenses can theoretically be matched to their 35mm equivalents by opening the f-stop as a scaled factor of the sensor size (that is, by a factor of 2.5), just as the focal length equivalent of a lens used with a 35mm sensor lens differs by a factor of 2.5 from the equivalent focal length of that lens used with a 2/3-inch sensor. The equivalent focal length and depth of field vary by the same factor. Accordingly, the focal length equivalent of a 50mm lens on a 35mm camera is a 20mm lens on a 2/3-inch camera. And the depth-of-field equivalent of a 35mm camera at f5.6 would be f2.4 on a 2/3-inch camera.

Red Channel Focus

The fact that the three sensors are set at different flange depths in cameras with 2/3-inch sensors makes a huge difference in perceived depth of field. No manufacturer of lenses for the 35mm format has ever succeeded in perfectly focusing all three color records on a mono-planar sensor, or for that matter, a film negative! Here’s the biggest dirty little secret about film lenses: The red color record of every movie ever made on single-strip film was recorded out of focus!

What color is the human face? It is mostly red. So what is the most likely result of perfectly focusing red light on a 2/3-inch sensor? It will at least result in a very different-looking rendering of human facial features. Every pore, blemish, and nose hair will be in crisp focus. This is generally not desirable.

Being aware of this issue, there are many ways to deal with the variation in focus between the color channels produced by three-chip cameras. During post, some productions apply selective 1/2-, 1/3-, and 1/4-pixel Gaussian blurs to the red records with great success. Camera filters in front of the lens can also help. Keep in mind that the filter effects you might expect on 35mm (film or single-sensor) will be multiplied by a factor of 2.5 when used on 2/3-inch sensors.

Figure 3.58 Film camera apertures versus HD sensors. (Image courtesy of Bennett Cain, Negative Spaces, LLC. © 2010 www.negativespaces.com.)

Figure 3.59 Digital camera comparison chart. (Image courtesy of Fletcher Camera.)

Lenses

When prepping to shoot a production, begin by selecting the best lenses that the production can afford. Putting a poor lens on a good camera will make the camera look poor. Putting a high-quality lens on a lesser quality camera will make that camera look as good as it can. Rental houses offer various types of high-quality lenses that can be used on digital cameras, both with 2/3-inch and 35mm sensors.

Keep in mind that lenses designed for 2/3-inch sensors yield much greater depth of field than lenses designed for 35mm sensors. This is important to consider when selecting lenses.

The Viewing System

The monitoring of the camera image on set is immensely important when shooting digital. A camera system can bake in³⁶ the color space, the resolution, the bit depth, end even the noise level. However, in the event of a bad decision or a change of direction, a change may need to be made. And if the mistakes are baked in, they may not be able to be undone. There are several ways to prepare for such a situation:

1.  Set up a Camera Control center with sophisticated color correction capability—the “lab-on-the-set” approach. This arrangement allows the Digital Imaging Technician (DIT) to carefully monitor many aspects of the digital recording and adjust levels as necessary. The role of the DIT in this work style is critical because the changes made have a direct effect on the recorded image (i.e., they are baked in) and in many instances cannot be reversed or modified at a later date.

2.  Set up a Camera Control center with a Look Management System only. No color correction is baked in to the image being recorded on the set. The portable Look Management System allows the cinematographer to compare, adjust, and create looks on the set without risk.

3.  Set up a Camera Control center where the image data is recorded in a raw format. The monitor is used exclusively for viewing.

The shooting work style is usually determined by the cinematographer, in conjunction with the director and the producer. Each style has its advantages.

Even without a viewing monitor on the set, the camera operator can always check the last few seconds of each take with the camera’s record review function. The cinematographer and director can also use a portable HD monitor to observe and check details that concern them.

In the case of a live production, or one with a tight delivery schedule, on-set color correction can eliminate the need for a long and expensive post-production period. A disadvantage of this strategy, however, is that the visual choices made on the set get baked in. Highlight or shadow detail may be permanently lost due to choices made in exposure and contrast, and the color record may be biased in a direction that, upon later viewing, is seen as not serving the intent of the production, yet cannot be undone.

Viewing Environment

Human perception is very sensitive to ambient light. The color and intensity of the light surrounding a monitor or screen being used to evaluate an image will significantly affect any judgment made regarding the density, contrast, or color of that image.

SMPTE has published recommended practices to create a viewing environment for the evaluation of color television images (SMPTE RP-166). It specifies standards for monitor setup, screen size, viewing distance, the color and brightness of surrounding surfaces, and the color and intensity of ambient light. Most critical are the color and brightness of the area immediately surrounding and behind the monitor—what is seen in the viewer’s field of view.

It may be difficult to meet all of these standards on a practical set. However, a shaded corner (created with flags), a gray piece of cloth draped behind the monitor, and a backlight setting the neutral reference source (approximately 6500°) may be adequate.

The Recording System

Media used to record digital images include magnetic tape, disk drives, and solid-state media. Digital cameras may record to magnetic tape, a disk drive, or flash memory in a wide range of configurations.

Several popular digital cameras (such as the Panavision Genesis) can have an on-board digital tape recorder. This gives the camera a similar configuration to a film camera. It can be handheld without the need for a tether cable connection to a remote recorder. Tapes must be changed, though less often than with film magazines.

By far, the most common digital tape recording format used today with cinema-level cameras is the Sony HDCAM-SR. SR recording offers two compression options: HQ (2.7:1) and SQ (4.2:1). HQ records 4:4:4 10-bit; SQ records 4:2:2 10-bit. SR compression is considered by some to be visually lossless, but like many compression schemes, can sometimes present problems in compositing and greenscreen or bluescreen work. Testing is always recommended.

Other on-camera digital tape recording systems include HDCAM and DVCPRO-HD. HDCAM and DVCPRO-HD are suitable for high-definition television, but they are generally not suitable for digital cinema work.

Several popular digital cameras have solid-state digital recording systems. Flash memory cards or solid-state recording decks mount on the camera. Other devices record remotely with a cable connecting to the camera. Solid-state memory systems have several advantages: They are lighter, more rugged, and easy to back up or transfer to external storage.

Some digital cameras do not support any on-board recording system. These cameras must be tethered—connected to an external remote recording system. This remote system may range from a tape deck (HDCAM-SR deck with processor) to a breadbox (an S.two hard-drive system) to a computer disk array the size of a small refrigerator. External disk and solid-state systems require additional time and resources to back up data and transfer data to post, so as to free up space on the drive for future recordings. Some data recorder vendors (Codex, S.two) have devoted significant effort to addressing these issues. In the process, they have made it easier to record in data mode (which must be rendered) while still generating DPX files or other formats that can be used for immediate viewing.

Tethering a camera may sound inconvenient, and it does put restrictions on shooting, but it also brings some conveniences. The cable leads can often be quite long—300 feet or more when optical fiber is used. The tether provides a high-quality connection to the video village. The camera itself may be quite small and light and able to process high data rates, which allow for the high frame rates used for specialty purposes. The tether may also support two-way communication, motion control, and power.

Tape-based recording systems work in a streaming workflow. Cameras and sound recorders record to tape in real time. Signals play back from tape in real time. This is the closest digital equivalent to the analog arrangement where devices are connected by simple wires. Communication between devices by an HD-SDI connection is usually an indication that this approach being used.

File-based workflows treat data in a more computer-like way. Once data has been produced by a camera, typically recorded to disk or a solid-state device, it can be manipulated in a more versatile fashion. The random access retrieval capability of data saves time. Devices in a file-based workflow are connected with a computer networking protocol. There is a large installed base of streaming-workflow equipment. Although the industry is now in a transitional period, ultimately, all post-production will likely be done using file-based workflows.

Scene Files

Scene files allow camera users to move quickly from one saved look to another, without having to manually make multiple setting adjustments in the menu of the camera. These scene files feature preset menu settings that are suitable for sports, news, beauty, music videos, night scenes, and various creative looks.

They can be used to store and transport looks such as black-and-white, high contrast, low contrast, bleach bypass, as well as lens maps and tables for individually color-mapping zoom lenses and prime lens sets. Note that scene files vary widely in format from one brand of camera to another and are rarely interchangeable between makes and models of cameras.

Scene files have the serious disadvantage that when used, they destructively and permanently bake in the look being applied. They are not reversible in post-production, so use this method with caution, especially in visual effects work. Use of scene files is often unavoidable and necessary, such as when using an 8-bit camera system like the Sony F-900 and recording to its onboard HDCAM deck.

On-Set Look Management

Ideally, the full dynamic range of the original uncorrected scenes should be recorded during production. A look can be created or applied on set, either as an ASC CDL (color decision list) recipe or as a LUT (look-up table). The look would not be baked in, but instead attached to the original scenes as metadata and used throughout the workflow nondestructively to apply the cinematographer’s look to the original camera images as displayed via a calibrated monitor or digital projector.

The basic (RGB primary color grading) look for a scene can be established on set and saved as a 3D LUT or as an ASC CDL (with compliant applications). It can then be used to maintain a consistent look for dailies, editorial, and preview screenings. With LUTs, this cinematographer’s look reference can only provide a visual reference for final color correction in the DI. With the ASC CDL, this cinematographer’s look reference can actually be used as a starting point for the final color correction.

LUT-based corrections are very powerful but hard to modify. A LUT-based workflow can often require applying several modifications to image data in series, which may introduce problematic amounts of noise. For maximum flexibility in final color grading, an ASC CDL-based on-set look should be used. In either case, the look should not be baked in to the original recorded image—until the final DI color correction sessions.

ASC Color Decision List (ASC CDL)

The ASC CDL is a cross-platform data exchange format for RGB primary color grading/color correction systems. It is a framework that allows the interchange of basic color corrections between color correction systems made by different manufacturers and different facilities. The ASC CDL defines functions and interchange formats for the basic primary correction operations of slope, offset, and power (similar to the standard lift, gain, and gamma, but consistent in all implementations) as well as saturation. It allows for the meaningful exchange of these values from one proprietary software or hardware platform to another.

FILMING LIVE-ACTION PLATES TO BE USED IN VFX

Bill Taylor, ASC

Large-format still rear projection came into its own in the 1930s alongside motion picture rear projection in the major studios. The extremely high-quality 8 × 10 images were shot on glass plate negatives with bellows-type view cameras. Plate became the generic term for any still or motion picture background photography.

Even though most composites are now created digitally in post, the background, or any live-action footage that will be used for visual effects, is still called the plate. For simplicity’s sake, assume a simple two-element shot, containing a background (the original plate photography) and a foreground, which could be any added element, from an actor driving a car to gigantic fighting robots. Of course, the added element could actually appear to be behind the plate, as in a typical matte painting.

The requirements for plate photography vary from the simple to the complex. To produce a convincing composite, even a simple locked-off background has to match the eventual composited foreground in lighting, camera position, the angle of view of the lens, and the camera tilt. For some shots, clean plates are also required, that is, plates made without some specific element. (See the earlier section on clean plates.)

Camera Position (Station Point)

The camera height must match the height, or apparent height, of the foreground camera, unless a deliberate scaling effect is desired. Obviously, if the ground is not visible in the shot, some plausible place where the actor could be standing can be inferred, as on a mountain path where only the deep background is visible.

The distances to key elements in the foreground and background must match. If the camera cannot get far enough back to include the foreground, going to a mismatched wider lens may cause serious problems. Instead consider a pan or new composition that includes the desired elements.

Angle of View

If the entire background image will be used as the background to the composite scene, the horizontal angles must match. If the foreground and background are shot in different formats, equivalent lenses (matching degrees of view) must be used.

It is good practice to shoot the background plate one lens size wider than the foreground, allowing some movement within the frame. If the photographic quality is good, the slight blowup required will not be noticeable. The background image can be much wider than the foreground if multiple cameras are used. (See below.)

Lighting Considerations

Record the sun position and angle, as well as key/fill ratio and color temperature information (see the section on data capture). Regardless of the compositing technique, it’s always preferable to shoot the background first. (For rear projection, the background obviously must be shot first.) The foreground will be lit to match the background, so the background lighting must be appropriate and congenial to the foreground.

For greenscreen compositing, it’s possible to shoot foregrounds first (although not recommended), in the hope that the matching lighting condition can be found when the plate is shot. It’s not always possible in practice if the weather doesn’t cooperate, which can lead to great difficulties. [On Mike Nichols’ The Birdcage (1996), the writer had a crew waiting fruitlessly for 10 days for sun in Florida to match high-contrast foregrounds already shot.] When there’s no alternative, a study of maps of the eventual location will at least yield correct sun direction and elevation at the appropriate times of the day.

It’s a challenging job to simulate single-source hard daylight on the stage. Fill light (supposedly sky light and bounce light from the environment) must be shadowless and the correct color balance to the key light. These issues are touched on earlier in this chapter in Greenscreen and Bluescreen Photography.

Camera Tilt

Tilt is probably the most important dimension to match, especially in scenes where the new foreground character must walk in depth (on the z axis) in the shot. The horizon (or vanishing point) in the foreground must match the horizon in the background. Even though overt perspective cues to the horizon may not be visible in a greenscreen foreground, it can become painfully obvious when there is a mismatch.

Background Quality

Because the background image must sometimes be scaled, heavily color corrected, and otherwise manipulated, the background should be of the highest possible quality to begin with. If the background is shot on film, use the finest grain, longest tonal-range film that’s practical. Kodak Vision2 50D is perfect for day exteriors. If the foreground is 4-perf 35mm, consider 8-perf (VistaVision) for the background. Available 8-perf cameras such as the Beaucam are small and light, and though negative costs double for the plates, it’s usually a minor increase in cost over the photography of an entire feature film.

If the show is digital, consider shooting the backgrounds on film, a more forgiving medium with (usually) a longer tonal range, especially for potentially harsh day exterior photography. If backgrounds are shot digitally, shoot and record full-bandwidth 4:4:4 uncompressed or with the minimum compression the camera system can deliver. Expose carefully to protect highlight details. Consider 3K or 4K photography for the backgrounds (essential if the rest of the film is 4K) as new high-resolution cameras come on line.

If shooting digitally, disable all sharpening or detail functions. Any needed sharpening can be added in the composite.

Moving Plates

Shooting from a moving camera adds new issues, since several reference points are no longer fixed. Because it’s a typical problem, a car shot will serve as an example.

The first thing to establish is what the final composite is intended to simulate. Is the camera supposed to be mounted to the car, or is it supposed to be on a tow vehicle or a chase vehicle?

The second issue is mounting the camera or cameras so the correct point of view is accurately simulated. Mismatches can yield disconcerting or unintended hilarious results (such as giant cars looming in the background); in either case the illusion is destroyed.

Scouting the Camera Positions

With stand-ins in the key seats of the hero car, walk around the car with a finder and locate where the cameras should be to cover the scene. Find the appropriate focal length for each setup. Then measure each camera position and shoot still images from that position showing the actors and a slate.

Measure the camera height from the ground.

Locate the camera relative to the hero car: Once the height is known a triangulated measurement from two prominent points on the car to the lens will locate it with more than adequate precision. (Please refer to Triangulation as a Method of Recording Camera Data earlier in this chapter.) For more precision, measure to the iris diaphragm of the lens, which is usually at the nodal point, the place inside the lens where the principal rays cross. For example, the lens might be 8 feet 6 inches from the top left corner of the windshield and 9 feet 2 inches from the top right corner.

The vertical angle is easily measured with an inclinometer, but the horizontal angle is a bit trickier. It’s very straightforward if the camera head has the pan calibrated in degrees: Zero the head on the centerline of the car and note the angle to the left or right of zero. If a calibrated pan is not available, find and note a feature on the car that falls somewhere on the vertical centerline of the viewer. Back that up with a visual estimate of the angle.

A Case Study

In a typical example, two actors have a long dialog scene in the front seat of the car. The storyboards show that the director wants to cover the scene in a two-shot looking back, two singles looking back, and angled side shots from both sides of the car. In an important distinction, the boards show that the two singles must be from cameras on separate parallel lines, rather than from a single vantage point matching the two-shot.

The distinction is important, because if the two singles were from the station point of the two-shot, it’s likely that scaling and repositioning the background made for the two-shot would have sufficed for the singles if the background image were sufficiently high in quality. In any case, all three backgrounds must be in sync, since the editor will likely cut from the two-shot to the singles and back.

Any scheme that can reduce the number of trips down the road is worth considering. The time required to return to the start point can be significant, especially if there is a big entourage of picture cars to be repositioned. The rental costs of extra cameras is small compared the cost of extra days of shooting, risking mismatching weather, etc.

In the case in question, it was a straightforward job to rig three cameras on a camera car to cover all three rear-looking plates at once, in one trip down the road. On the head end of the setup, the hero car was lined up with stand-ins in the correct orientation to the road. A slate was shot on the film camera as well as still images from off-axis that showed the relative positions of the cameras and actors. Not only was there no doubt as to which scene those backgrounds belonged to, the actual lighting was recorded as a reference for the stage shoot.

Camera Cars

Camera cars have been designed for plate shooting that range from high-powered, high-speed electric carts to custom-built trucks like the Shotmaker. The most important feature on any camera car is a suspension that produces a smooth ride free of bumps and vibration. All camera cars feature fittings for camera mounts on all sides at any height from scraping the ground upward. Many of the trucks will accept booms for operated shots on the road. Booming shots are rarely useful for plates, but the booms allow quick repositioning of the camera, which is important when time is tight. Some booms allow mounting of multiple cameras, permitting (for example) tight and wider versions of the same backgrounds to be shot simultaneously.

Cameras are typically connected to monitors on the chassis or in the cab. These monitors are never big enough for critical playback inspection, so the footage shot should be checked on big screens before the shots are signed off and a retake becomes impossible.

Camera Car Safety Issues

Crew members have been killed or injured far too often on camera cars.

Camera crews and, particularly, crane operators riding on the chassis are potentially in danger. Crews riding in the cab are far safer than those riding outside.

Just for starters: Every rider on a moving camera car must wear a crash helmet and be belted in. The moving car must have a protective buffer around it provided by police vehicles. If the job includes photographing stunt action by other vehicles, it should go without saying that it must be both safe and well rehearsed.

Purpose-Built Crane Cars

The Ultimate Arm (and similar equipment) is a remotely operated camera arm mounted to the top of a high-powered SUV It can rotate 360 degrees around the vehicle and rise 12 feet or higher. Some arms can mount two lightweight cameras like the Arri 250 side by side.

Advantages of Crane Cars

Crane cars are much faster and more maneuverable than a vehicle like the Shotmaker.

Repositioning the camera between setups can be very fast, since there is no need to detach the camera, reattach it, reroute and reconnect cables, etc. Where a combination of plate work and location work must be shot, the crane can often substitute for a tripod for static shots. Particularly at freeway speeds, crane cars are much safer than cranes grip operated from the back of a truck.

Limitations of Crane Cars

Crane cars lack the flexibility of bigger vehicles like Shotmaker, don’t have the variety of camera attachment points, and have limited space for crew members inside the vehicle. When operated at high road speeds, cameras on booms are vulnerable to wind buffeting. Matte boxes have a high wind load, so they should be kept small and, if possible, be replaced with a lens shade.

Monitors in crane cars tend to be even smaller than monitors in bigger vehicles, so it’s even more essential to review work during the day on big screens to be certain that vibration, missed focus and equipment glitches have not ruined otherwise good takes.

Vibration and Camera Stabilization

A good camera vehicle will remove enough vibration by itself to produce useful footage on the highway with wider lenses (in 35mm film photography, 50mm lenses and wider). With longer lenses and on rougher roads, camera stabilization becomes important.

Note that the issue is not just stability of the filmed image. Unsteady film can be stabilized in post with common software. However, the sudden jarring motion produced by road roughness or wind buffeting can streak or blur the image, effectively destroying the frame. A software fix can interpolate a replacement frame from the frames on either side but this is not a sure thing if there are many frames to be fixed. It’s much better to shoot an unblurred shot to begin with.

Gyroscopic stabilizers like the Libra head can produce stable shots with long lenses and big zooms under most circumstances. There’s a wide choice of equipment in this category that is well worth testing.

Road Speed

High road speeds are sometimes unnecessary even if called for in the script. If all the foreground cars are under the control of production and background action permits, consider shooting at a slower road speed and undercranking. Wind loads increase geometrically with speed so even a modest road speed reduction can yield substantial dividends in vibrations and buffeting.

Road Speed Related to Camera Angle

When the camera is 90 degrees to the direction of travel, objects near the camera appear to whip past quickly and disproportionately to the actual road speed. Due to this effect it has been common practice to reduce road speed to 60% when shooting side plates, 80% when shooting at 45 degrees, and so on (60 mph becomes 36 mph at 90 degrees, and so forth). Obviously this is a problem when shooting front and side plates simultaneously unless the background action permits under-or overcranking of one of the cameras.

When the subject matter permits, consider overcranking the side-looking cameras. For example, if the forward-looking cameras shoot at 24 fps, shoot the side-looking footage at 40 fps. The footage can be sped back up in post if necessary.

The whole issue of modifying road speed to camera angle is somewhat subjective. As more and more traveling footage is shot on location with car-mounted cameras photographing the real actors from all angles, audiences have become more accustomed to the speed effect in the side angles. Some directors find the effect unobjectionable. If composited plates must cut in with real car-mount photography, they should be shot at the matching speed relationship to the real photography.

Of course, if the plate is shot with a panoramic multicamera rig like CircleVision, camera speed must be the same in all cameras.

Precautions

Lens flare must be carefully controlled (beware the vibrating sun shield), and constant vigilance maintained for dirt on the front filter surface. An air jet-type rain deflector can also deflect dust and smaller bugs.

Panoramic Rigs

When a sequence requires wide-ranging camera movement, or when it’s uncertain what part of the background scene must be in frame at a given moment, a panoramic background can be created with a combination of several images. Usually the multiple images are filmed simultaneously with several cameras, but in some cases the background can be shot with a single camera at several successive angles.

On the first Fast and Furious (2001) film, all the night race car interiors were shot on a bluescreen stage. VFX Supervisor Mike Wassel suggested shooting the background plates with a modified CircleVision rig, which was built by Jim Dickson. (CircleVision normally shoots a full 360 degrees for special venue films with little overlap between panels. In the background plate mode, the lenses are one step wider than normal to allow a comfortable overlap for stitching the images together.)

To achieve the desired low vantage point and high road speed, the rig was mounted on a motorcycle with three of the usual nine cameras removed to allow for the motorcycle driver. Thad Beier and Hammerhead Productions took the resulting six strips of film, stabilized them, warped them, stitched them together, and projected the result onto a 240-degree digital cyclorama. (The same plate material was used as texture for a simple 3D model of the street.) The rig was flipped on the motorcycle to shoot the opposite side of the street. Only two trips down the street were necessary thanks to six cameras.

Variations on this idea involving wide lenses on 8-perf VistaVision cameras and 65mm cameras have been used on many other films for car and train backgrounds. Mark Weingartner has shot many HDRI panoramas, both still and motion picture, for The Dark Knight (2008) and several other films, in some cases creating full hemispherical coverage. Please refer to his section Shooting Elements for Compositing in this chapter.

Digital panoramic systems are under development. Dynamic range and bandwidth requirements for daylight, exterior, relatively uncontrolled photography stretch current digital systems to their limits. One obvious benefit is that the 30-minute reload for multicamera film systems is unnecessary.

On the Water

Gyro stabilization is vital on water to keep the horizon level. In Bruce Almighty (2003) an extended greenscreen sequence took place at Niagara Falls, supposedly on a tour boat. Panoramic plates were shot from a tugboat in the river below the falls with the Beaucam, a small, lightweight 8-perf camera, on the Libra head. The Libra kept the horizon level and ironed out the high level of vibration from the tug’s engines, which had to work hard to hold the boat in place against the current.

Speedboats are even more of a challenge. The Perfect Horizon head from Motion Picture Marine is powerful enough to deal with heavy cameras and high rates of pitch and roll.

On the water it’s critical to keep spray off the front filter surface. There are two basic approaches: electric rain deflectors in which a clear disk spins in front of the lens and throws off droplets by centrifugal force and an air squeegee system in which air nozzles blow the front filter surface dry. Camera equipment rental services often provide a choice of types.

The larger spinning deflectors drive the disk from the center while the lens sees through a radial off-center section of the disk. The disk must therefore be quite large to cover the front element of a big lens. The other approach is to drive the disk from the circumference, resulting in a much smaller and lighter package. A small air jet is needed to clear the tiny droplet that remains at the center of the spinning disk.

Don’t use polarizers with spinning disk systems. The combination can produce flickering color shift.

Air squeegee systems position multiple nozzles around the front glass, usually on adjustable segmented tubes. These setups add less weight to the camera than spinners and occupy less space, but they use a lot of air, which must come from nitrogen bottles or a compressor.

Air to Air

The most versatile systems for aerial plates are the helicopter-mounted gyro-stabilized balls. A skilled pilot can “thread the needle” with the camera, making possible precision hook-ups and handoffs to other cameras, as in the main titles of The Birdcage (1996) and The Fighting Temptations (2003). In those two films, the shot began on a helicopter and was handed off invisibly to a camera crane and then to a Steadicam operator, who walked into the final set. The best known system of this type is Ron Goodman’s pioneering Spacecam, which can be mounted to the nose or the side of several models of aircraft. Spacecam can shoot 4-perf and 8-perf 35mm film, as well as 5-perf and 15-perf (IMAX size) 65mm film, and can accommodate most digital cameras.

Spacecam can record very accurate GPS data synchronized to the camera, including aircraft speed, height, and direction as well as pan-tilt-zoom lens axis data. This data provides much of the information needed for motion tracking and matchmoving the plates, a real plus for visual effects use.

Small 35mm cameras and digital cameras can fly on purpose-built miniature radio-controlled helicopters. In the hands of skilled pilots, they can produce good results. They have the advantage that they can land and take off anywhere with little ground support. In practice there are some significant limitations. These small craft are easily blown off course by crosswinds and have a maximum height of 500 feet. They cannot be flown safely over people (think “upside-down, flying lawn mower”) and in the past have had reliability problems in the field. There may be specific applications for which a test is justified.

To shoot high-speed fixed wing aircraft (and high air-speed photography in general), there’s no substitute for a jet camera platform. One widely used system, the Nettman Vector Vision rig, is built into a Lear jet and can shoot digital, 35mm, 65mm 8-perf, and 65mm 15-perf from either top- or belly-mount lenses. The image is optically relayed to the camera inside the aircraft, so reloads can be done in the air. Although the optical path is complex, the resulting image is very high in quality.

Cable Systems

When the required plate camera POV is relatively near to the ground (below 200 feet), cable systems fill the gap between helicopters and camera cranes. Cable systems can operate safely over people and sensitive locations, with no prop downwash or dust.

Early cable systems flew a Steadicam operator point to point in a minimal cable car, similar to a ski lift. The operator could dismount from the car and continue the shot seamlessly on the ground. James Muro, ASC, operated the first shot of this type in The Mighty Quinn (1989).

Garrett Brown’s original Skycam was the first multicable system that could fly the camera anywhere within the 3D space defined by the three or four computer-controlled winch stations. The prototype Skycam shot the plates for The Boy Who Could Fly (1986), but later found much more use in video sports photography than in film.

A highly refined system that has been through many evolutionary stages, Jim Rodnunsky’s Cablecam, can fly film or digital cine cameras in very large 3D volumes (the size of a football stadium) or over great distances, all at very high speeds. Unlike earlier systems, Cablecam uses a central winch station running cables to high, lightweight stanchions, which can be erected nearly anywhere. Cablecam is the nearest thing to the proverbial “skyhook” with the ability to put a camera nearly anywhere with great precision. The only limitation is cable clearance, which can be overcome with clever set design. For example, set elements that conflict with the cables can be built to move out of the way at the appropriate moment, or they can be digital models tracked into the shot.

Spydercam is the overall trade name for a quartet of overhead cable systems. The system first became famous for traveling vertically at high speeds down the sides of buildings in Spider-Man (2001), but it can also travel long distances from point to point. Some Spydercam systems can now operate in 3D space. One Spydercam system (Talon) is motion-control repeatable, while another (Bullet) can fly 1 mile at 100 mph.

Both Spydercam and Cablecam can previsualize and program in Maya.

Believability

As with visual effects in general, the underlying secret to convincing plate photography is to ask, “How would this be photographed if it could really be photographed? Where would the camera really be, and how would it cover the action?” If the answer is simply impossible (the camera is hovering in space 2 feet from a surfer’s face, for instance), proceed with caution! The result may never be believable, however well executed.

SHOOTING ELEMENTS FOR COMPOSITING

Mark H. Weingartner

The VFX Supervisor of the 21st century has access to the most complex 3D procedural solutions in order to generate smoke, clouds, flames, water, debris, and even crowds. Sometimes, however, the fastest, easiest, cheapest, and most rewarding way to get the elements is to have a special effects technician come to the nearest parking lot or stage, set up a camera, and just shoot!

What Is an Element?

In broad terms, an element is something or someone that is generally shot in such a way as to make it easy to separate from its background—grist for the compositor’s mill—as distinguished from a plate, which is generally used as the background for a shot. Elements might be shot against black, white, gray, blue, green, red, or sky, depending on the material being photographed. Technically speaking, elements run the gamut from generic water dribbles and wafts of smoke all the way to multi-million-dollar movie stars acting in front of green screens or blue screens. Shooting elements can be complex—involving the lighting of giant green screens for high-speed photography or complicated motion control shots—or as simple as lining up a few coworkers in costume and shooting them with a prosumer video camera in order to populate a distant crowd scene.

The decision to shoot elements rather than to create them in a CG environment is situational. Since most visual effects facilities have abandoned their stages and camera equipment in order to make room for more workstations, the costs involved in putting together an element shoot can look daunting, but compared to the timeline involved in seeing successive versions of something being built in the CG world, an experienced visual effects camera crew with the support of a competent special effects team can experiment quickly and film many different versions of an effect in a short period of time.³⁷

Generally, photography paid for by a studio is considered work for hire and rights to its use are retained by the studio. For this reason, it may be beneficial for the visual effects facility to arrange its own element shoots and to retain ownership of the material for future use on other projects.

Stock Footage

Some element needs can be filled with stock footage. Several agencies sell stock footage in various formats. Some of them even distribute specific collections of clouds, flames, or generic outdoor backgrounds. Finding appropriate footage involves wading through a lot of material and hoping that it is available in a usable format, but sometimes it works out. The explosion sequence of the big spaceship in Event Horizon (1997), a Paramount picture, actually incorporated a small piece of a space explosion from another Paramount film.

Types of Elements

In discussing element photography, it is useful to categorize the types of elements a couple of different ways. One can differentiate between full-sized elements and scaled elements, and one can also differentiate between shot-specific elements and more generic ones. These two sets of distinctions are not mutually exclusive—either miniature or full-sized elements can be destined for a specific shot. Likewise, one can shoot a library of generic elements as scale elements, full-size elements, or a combination of both.

Generic versus Shot-Specific Elements

Resizing elements in the digital world is only a mouse-click away, but even with this ease of adjustment, there are benefits to shooting elements in a shot-specific way. In shooting an element for a specific shot, one has the advantage of being able to choose the appropriate lens and distance to match the perspective of the shot, the appropriate framing to make the shot work, and the appropriate lighting so as to integrate the element into the scene. Any camera position data from the original shot can be used to aid in line-up, but with elements such as water, pyro, flames, or atmospherics that are likely to break the edges of frame, it is best to overshoot (shoot a slightly wider shot), which will allow the compositor a bit of freedom in repositioning the element to best advantage without bits of the action disappearing off the edge of the frame. VFX Directors of Photography often use all of the area of full-frame 35mm or, when possible, VistaVision,³⁸ in order to allow for the most repositioning without sacrificing resolution when resizing or repositioning the image in post.

When it comes to shooting actors in front of blue screen, green screen, or sky, once the correct distance and angle have been worked out to ensure proper perspective, one frequent practice is to select a longer focal length lens in order to magnify the subject. As long as the subject does not break the edge of frame, this magnification yields better edge detail, which allows for easier, cleaner matte extraction.

Even though there are obvious advantages to shooting elements for specific shots, once the equipment and crew are assembled, one may realize a great return on investment by shooting a library of elements with variations in size, focal length, orientation, action, and lighting. These generic elements can be used as necessary in building many different shots, scheduled or added.

When shooting either generic or shot-specific elements such as flame, dust hits, atmosphere, etc., it makes sense to shoot some lighting, focal length, and angle variations—sometimes the correct setup ends up not looking nearly as good as some variation shot just in case.

Determining Element Needs

During the bidding and pre-production of a project, the numbers and types of elements needed for various shots are usually broken down and categorized.³⁹ During production it is common to keep a running list of new elements that are required. Additional elements may be added to the list after a review of the shots in the edited sequence or during the post-production work on those shots.

The first step is to review the list of desired elements with an SFX Supervisor and a VFX DP if at all possible. Their expertise can make things much easier and provide insights to alternate techniques. A skilled special effects person can do wonders with devices they have or with devices they can construct rapidly. Certainly elements that require anything potentially dangerous such as flames or explosions will require a special effects person with the appropriate license.

Cheating

The only hard and fast rule with regard to shooting elements is that they have to look right when they are incorporated into a shot. Since individual elements are shot one at a time, all sorts of tricks can be used when creating them. One can composite together elements shot in different scales, at different frame rates, with different camera orientations, and even with different formats of cameras. An element can be flipped and flopped, played backward, recolored, pushed out of focus, made partially transparent—in short, the VFX Supervisor has an arsenal that includes all of the different types of shooting tricks plus all of the various compositing tricks for creating the desired look.

Even as the bar is constantly being raised in creating realistic digital effects, numerous tricks of the trade of the old school creators of special photographic effects⁴⁰ are still valid. To paraphrase Duke Ellington, “If it looks good, it is good.” Some types of atmospheric elements, such as smoke, steam, or fog, can be shot small and scaled up, often being shot overcranked in order to give the correct sense of scale. Water effects can be scaled up somewhat, though issues with surface tension limit the degree to which this works. Different materials can be used in clever ways; for Star Wars: The Phantom Menace (1999) large amounts of salt⁴¹ were poured over a black background to create the illusion of waterfalls seen in a Naboo matte painting. For Independence Day (1996) a comprehensive library of steam, dust, glass, and debris elements was shot on a single day of thousand-frame-per-second photography. Those elements were massaged in countless ways and incorporated in many shots of destruction. To create the effect of a man whose skin was on fire in a zero-gravity environment for Event Horizon (1997) hundreds of individual bits of paraffin-soaked paper were attached to a specially constructed puppeted mannequin and burned in small groups to create small patches of flame that were not drawn together by convection. For the free-fall sequence in Vanilla Sky (2001), Tom Cruise was suspended upside-down and wind-blown with E-fans while the camera, turned sideways or upside-down at various times, hovered around him on a crane or whizzed past him on a 90-foot-long motion control track.

Miniature pyrotechnics that are supposed to be in space are often shot looking up (or at least upside-down) so that the arcshaped paths of debris and convection-driven smoke propagation patterns don’t give away the presence of gravity.

It is easier to cheat at some games than others, of course. As humans, we are highly attuned to the way other humans look, and when human elements are shot from the wrong angle, the wrong height, or the wrong distance, even the most nontechnical audience member will often sense that something is wrong with a shot. The cues that trigger this response usually have to do with a perspective mismatch—and the most common error in shooting actors on green screen is to shoot from too short a distance with too wide a lens, resulting in unnatural relative sizes of different parts of the subject. The best protection against this danger is to line up the shot with the same lens, distance, and angle as was used in the plate shot or as dictated by the action of the shot, but when this is not possible, a decent rule of thumb regarding perspective on a human that is allegedly being seen from a great distance is to get the camera at least 25 or 30 feet away.

Backgrounds

The choice of background against which to shoot is dependent on the type of element.

The object of the game is to end up with an image of the element with no background stuck to it so that the compositor can add the element to an existing plate. Ideally, the only criterion for choosing the background color would be to allow for the easiest and highest quality matte extraction, but in the real world, the costs involved in lighting blue screens or green screens (and the likelihood of damaging them) can weigh heavily in these decisions. Whenever possible, discuss the pros and cons of various backgrounds with both the VFX DP and the supervisor at the visual effects facility that will be responsible for the shots—different facilities have different preferences based on their particular experiences and the toolsets that they have created. A detailed explanation of bluescreen and greenscreen photography appears elsewhere in this book, but it is worth noting here that elements that contain transparent or translucent areas, motion-blurred edges, or highly reflective surfaces can create significant challenges when shot on blue screen or green screen unless all of the lighting and shooting parameters are very carefully controlled.

Black Backgrounds

One of the beauties of shooting against a black background is that in some circumstances, if the exposure on the background is low enough compared to the element being photographed, no matte extraction is needed at all. If a matte is needed, it can often be created as a luminance matte. Good candidates for shooting against black include flames, smoke, dust, debris, liquid splashes, and explosions.

Three types of black material are in common use:

•   Velvet: The most luxurious (and expensive) option, velvet absorbs stray light wonderfully, but must be kept dry and unwrinkled. It is expensive to buy and hard to rent.

•   Velour: Physically heavier than velvet, velour is the material of choice for theatrical drapes and masking and can be readily rented from theatrical supply companies. Like velvet, velour has a nap of fibers that extend out from the cloth and absorb light.

•   Duvetyne: Sometimes referred to as Commando Cloth, duvetyne is much less expensive than velour or velvet but is not as light absorptive. This is the material most commonly used in making flags and solid frames or overheads used for controlling light on set. It is available for rent in various standard sizes and for purchase as raw goods by the roll.

Line-Up

If actors, objects, or atmospherics have to fit into a specific area of a shot, the camera should be positioned to match its position relative to the subject of the original shot.⁴² This is a relatively simple prospect if the data from on set is accurate and available, but even without it, an experienced VFX Director of Photography should be able to match the angle and distance by analyzing the original shot. If the shot involves actor translational movement or any element interaction with the ground plane, it is vital that the line-up be accurate. Few things look worse than repositioning and tracking an element into a shot in order to solve a problem created by shooting that element incorrectly. Actors can be given tape marks and green or blue objects carefully placed in the set that correspond to the original scene. See the Monster Sticks section earlier in this chapter for more details about providing actors with interactive guides.

Traditionally line-up was accomplished by inserting a frame (or clip) of the original shot into a clip holder in the camera’s viewing system, but in the modern world, this is facilitated by using a video switcher to combine video of the original shot with the output of the camera’s video tap. Since film camera video taps show a larger image than what appears on the final framed shot, one must either use the video tap image recorded when the original shot was made or resize one of the images to match the other.⁴³ This calls for a video switcher that has the capability of magnifying or shrinking the image from one of its inputs⁴⁴ (or presizing the video to match using compositing software). The switcher should be capable of doing DXs⁴⁵ (semitransparent superimpositions) as well as rudimentary chroma keys. A number of tape- and hard drive-based video playback systems are used in the industry that are capable of doing all this, and hiring someone with all the right equipment is frequently easier than putting a system together for a single shoot.⁴⁶

The most important rule for successful line-up work is this: Perspective is determined by position, not focal length.

Once the camera is positioned relative to the subject to create the right size image in the correct place in frame using the matching focal length, it is often desirable to change lenses to a longer lens that magnifies the subject more, preserving more detail for better matte extraction. In choosing to push in, be careful not to magnify the image so much as to allow any of the action to break the edge of frame—any action that is outside the field of view of the camera is gone for good, and any benefit gained from a magnified image is more than offset by the loss of usable action.

If the camera move is complicated and/or creates significant perspective changes, the best way to shoot the elements might require the use of a motion control camera system, either driven by matchmove data from the original plate or from encoded or motion control data from the original plate shoot. See Supervising Motion Control in this chapter for specific information.

Figure 3.60 Author with motion control legs for flame, steam, and water reflection passes. (Photos courtesy of Mark H. Weingartner.)

Camera Format Considerations

A number of factors affect the camera and format choices. It is best to discuss these issues with the VFX DP as the shooting plan is developed:

•   Frame rate: Does the shot need to be overcranked? To give miniature elements the proper sense of weight when interacting with atmospheric elements, wind, debris, flames, or explosions, it is necessary to overcrank the camera, but how fast? A starting point for choosing a frame rate is to multiply the base frame rate (generally 24 fps) by the square root of the scale of the miniature.⁴⁷ For example, if the miniature is 1:4 scale, the shooting frame rate would be 24 × √4 = 48 fps. If the miniature were 1 inch = 1 foot (1:12 scale), the frame rate would be 24 × √12 ≈ 83 fps. It is important to test frame rates in order to find the right look for each situation. This formula does not take into account slowing down of action for aesthetic purposes but assumes that the intent is to show the action in real time on screen.

•   Shutter angle: Does the shot require the use of a narrower shutter angle to reduce motion blur? The effect of narrowing the shutter angle is to shorten the exposure time. While this can benefit matte extraction, it can also lead to choppy-looking motion or strobing, depending on the subject’s motion with respect to the frame. Be very careful when intermixing elements with different shutter rates from the original live action. When in doubt, test or shoot multiple versions.

•   Contrast ratios: Film can handle a greater contrast range than any of the electronic cameras without clipping, but the tradeoff is cost and processing and scanning time.

•   Resolution: At one end of the scale—65mm and VistaVision film.⁴⁸ At the other end—MiniDV video. For elements that will be reduced considerably in the final composite, sometimes a standard-definition video camera is good enough. For projects as dissimilar as Star Wars Episode I: The Phantom Menace (1999) and The Passion of the Christ (2004) some of the deep background people were shot full frame with MiniDV video cameras and shrunk way down in the comp. These shrunk-down elements can be used directly or in more sophisticated ways. To fill the seats of the senate in The Phantom Menace (1990), the costumed actors were filmed by three MiniDV video cameras mounted at different heights and angles. The people were rotated and photographed from those different angles. A computer script was written to select the appropriate video clip to use for a given senate seat based on its angle to camera within a given shot.

Assorted Methods for Shooting Elements

Actors and Props

Actors are usually filmed against blue or green screen with attention to shot-specific angles, distances, and focal lengths for line-up. For The Passion of the Christ (2004), puppeteers in greenscreen suits wielded canes and whips impacting green-wrapped sandbags in order to provide realistic interactive elements to composite onto the flagellation scenes of the film (which were shot with whip-less handles for obvious reasons.) Van Helsing (2004) used actresses dressed in blue against a blue screen to photograph their faces. They were suspended by wires and puppeted by people dressed in blue to match the action of the previs that had been created for an existing background shot. An infrared motion capture system was used to capture the actresses’ movements, and that MoCap data was applied to the final animation so that the perspective of the photographic and CG elements matched.

Actors can be hung from wires, supported on pylons, rotated in rigs, or bounced around on motion bases to simulate horseback, monsterback, motorcycle, or other modes of travel.

Occasionally other backgrounds are appropriate. When a backing screen is not feasible, some exterior stunts have been shot against sky. For Apocalypto (2006), the descender-rig actor shots used in the waterfall jump-off scene were filmed by multiple cameras against an 18% gray-painted high-rise building wall.⁴⁹

Explosions

Explosions are either shot against blue or green screens or against black, depending on the nature of the expected debris and the degree to which they are expected to be self-illuminated. One of the challenges in shooting scale model explosions is the need to shoot at very high frame rates, necessitating very high illumination levels for blue or green screens. Large explosions are frequently shot outside at night. An experienced special effects pyrotechnic technician will be able to offer up a number of different types of explosive effects by varying the proportions, placement, and deployment of different ingredients in the custom-made pyro charges. Safety is of prime importance—no shot is worth an injured crew member, and due diligence should be done with regard to protecting the camera, not only from fire and debris, but from water should it become necessary to extinguish a burning set. Some pyro recipes generate more smoke than others. This should be discussed with the SFX Supervisor. For some types of explosions, the meat of the action is over before the smoke has a chance to occlude what is being shot, but in other cases the smoke can create problems.

One of the benefits of shooting straight up for zero-gravity explosions is that convection drives the smoke up and away from camera. Shooting elements upside-down with the camera upside-down at least sends the smoke in an unexpected direction, and debris paths curve slightly upward instead of slightly downward in frame, which takes the curse off of the gravitational disadvantages of shooting on earth. Be sure to adjust framing to take into account the path of the debris. If necessary, shoot wider than needed—debris will break the edge of frame. Different materials create explosions with very different propagation speeds—the pyro technician and VFX DP will be able to advise with regard to suitable frame rates for different situations.

Flames

Flames are usually shot against black.⁵⁰ They are shot at normal speed if the flames are full size or at higher rates if the flames are to be scaled up or for aesthetic reasons. Some interesting effects can be achieved with different combinations of camera speed and shutter angle, but if one departs from normal practice, then testing or shooting multiple variations is vital. The special effects team is likely to have all manner of oddly named nozzles, flame bars, and flexible burners as well as different mixes of gas and various fuels and chemicals to create flames with different characteristics and colors. The flame can be further animated with fans. Safety is of prime importance. Aside from the dangers of fire, flame effects can throw off a tremendous amount of radiant heat—enough to damage things that are not remotely in contact with the flames themselves. Most decent blue screens and green screens are not fire retardant, and while much of the duvetyne used in the motion picture industry is treated with a fire retardant when new, once it has been wetted or washed, the retardant leaches out.

Figure 3.61 High-speed VistaVision camera in flame-box being engulfed in flame. (Image courtesy of Mark H. Weingartner.)

Water Splashes, Swirls, and Streams

Water can be shot in an existing tank, but an impromptu shallow tank can be made by building a frame of 2 by 12 lumber on edge, forming a square or rectangular trough and lining it with black Visqueen or similar polyethylene plastic. In spite of the specular reflections from the surface of the plastic, resist the urge to submerge duvetyne in the tank, because the fire retardant will immediately start leaching out of the fabric, clouding the water in a nonuniform way. If the reflections are troublesome, consider using various woven fabrics designed for agricultural use that can be weighted down in the tank.

Water splashes usually look best when side and back lit, while ripples generally do best when large soft light sources are suspended over or behind the water. As a last recourse, adding a small amount of white tempera paint, milk, or soluble oil⁵¹ to the water will reduce its transparency, but once that has been done, removing the adulterant requires draining and cleaning the tank. To create specific interactive splashes, use a black object of the appropriate shape and size and drop it in or pull it through the water. If shooting a small-scale proxy of the true object, overcranking the camera will be necessary, but be aware that due to the surface tension inherent in the makeup of water, the effects can only scale so far. Once again, a quick test can be cheap insurance for getting lifelike elements. Air hoses and water hoses can be used to agitate the water, and all sorts of streams can be created with different types of nozzles and water pressures. Rainfall on water can be simulated with “rainbirds” from the special effects department. The special effects folk know how to generate everything from a mist to a torrent. Don’t forget to bring a towel.

Underwater

Elements can be shot underwater to simulate weightlessness, slow motion, or just to create amazing swirly effects with hair and wardrobe. Water must be filtered in order to be clean enough to shoot through, and if people are working in the water without wetsuits (actors, for instance) the water should be heated. Some studios have shooting tanks with underwater windows that allow shooting without putting the camera in a housing and then in the water. A swimming pool with observation ports sometimes works just as well.

Lighting objects underwater carries with it its own challenges and dangers—make sure to use crew who are certified to work underwater and experienced with underwater lighting and the safe use of electricity around water. Any time crew is working underwater, expect the schedule to be slow—communication is difficult enough when people are face to face in air—underwater communication at its best is not so great.

Dust, Debris, and Breaking Glass

Interactive dust elements are frequently shot to add to the realism of large creatures or objects walking, falling, or otherwise impacting the ground. This can be done in a number of ways. Even if the “hero dust” in the final shot is going to be dark, the easiest way to film these elements is by using light-colored dust and black velvet mounted on a wooden backing. Dust is gently applied to the fabric and the light is flagged off of the velvet itself. The interactive dust hits can be created by dropping a black velvet-covered weight onto the velvet field or by shooting compressed air at it. As with any small dust or particle work, use air masks and eye protection.

Debris can be shot out of air mortars, dropped out of hoppers, or thrown by crew members in front of black, blue, or green screen. As with water effects, sidelight and backlight are often more effective in shooting debris than front light.

Breaking glass effects can be shot against black, of course, but sometimes work better against green screen or blue screen, especially when interactive reflections of various pieces of glass are to be incorporated into the composite. Different effects can be filmed looking up at glass (with the camera in a protective box, of course) looking down at glass, and looking across at glass as it is broken or shattered. The special effects team will have multiple ways of breaking different types of glass in order to create different types of effects. Glass effects will look very different depending on how they are lit. Large soft planar light sources create one type of look; very hard light creates a totally different sparkle. Be sure to order enough glass to allow some experimentation (and to allow for accidents).

Clouds

Clouds can of course be created by means of matte paintings or computer graphics. They can also be created in a liquid such as was done for Close Encounters of the Third Kind (1977) using a large clear tank (7 × 7 × 4 feet in this case) filled with saltwater on the bottom half and freshwater on the top half. White tempera paint was injected in the freshwater to create the illusion of cumulus clouds, which rested on the inversion layer of the saltwater. Other puffy clouds can be simulated by injecting soluble oil into clear water using various types of nozzles, and spreader bars can yield wonderful variations. Independence Day (1996) made good use of cloud tank elements for the atmospherics accompanying the arrival of the Destroyers.

Clouds can also be done using synthetic fiberfill mounted onto clear acrylic sheets as was done on Buckaroo Banzai (1984). Stratocumulus clouds can be built with cotton wool or fiberglass insulation.

Smoke and Fog

A number of different types of low-lying fog can be generated using dry ice fog machines alone or in concert with smoke generators and liquid nitrogen. Interesting effects can be obtained by sending dry ice fog over warm standing water, for instance. Different types of smoke machines are available, each with its strengths and weaknesses. Smoke and fog are generally shot against black. It is easy enough to recolor the element in post-production rather than trying to shoot dark smoke, but interesting effects can be created by burning acetylene and photographing the wispy, sooty smoke that it generates when burning. Consult the special effects team and the VFX DP for help making an informed decision.

Goo and Lava

Goo can be made using various food thickeners. There are various children’s toy goo mixtures and simple things such as gum can be used for goo. Consider these examples of the use of goo in small amounts to help the finished shots: In Van Helsing (2004), a werewolf literally rips off his fur. Goo elements were shot to help with the final effect of the stretching, breaking flesh. Goo was also filmed by an animator working on Dragonheart (1996) using a DV video camera. This was manipulated and used to help show the saliva in the dragon’s mouth. Lava can also be created as a separate photographic element, but will probably require some R&D before an agreed-on look is achieved.

Conclusion

Above and beyond elements containing actors, photographing elements of all sorts can be fast, easy, cost effective, and downright enjoyable. Sometimes the precise control made possible by developing CG elements procedurally is very important, but often the happy accidents that occur when special effects and visual effects get together to shoot elements can be truly wonderful, and the energy that this type of collaboration injects into the process can be quite beneficial.

HIGH-SPEED PHOTOGRAPHY AND FILMING ELEMENTS

Jim Matlosz

High-Speed Photography

High-speed photography, defined as a camera capturing moving images at a rate above 60 fps, in motion pictures was once relegated to miniatures and special effects. Today, however, with the advent of faster film stocks, bigger lights, and high-speed digital acquisition, high-speed photography is accessible to more filmmakers and artists and used by projects both great and small. In this section a number of topics are discussed to educate and inform those who have a need or use for high-speed photography, but also possess a basic understanding of cinematography and visual effects.

Cameras

The choice of cameras varies and many of these choices are covered in other sections of this book; however, two charts have been created to make your choice of high-speed cameras and deciphering a bit easier (Figures 3.62 and 3.63). The charts begin with film cameras, from the fastest 35mm on through the slowest of the 16mm. (Note: There are a number of high-speed, large-format IMAX 65mm 15-perf, 10-perf, and IWERKS 65mm 8-perf cameras in existence, but these cameras are seldom used anymore and have very limited application.) Then they list digital acquisition cameras from the fastest, highest resolution to the slowest, lowest resolution. Highlighted features include actual measured image size, pixel dimensions, capture medium, and weights of cameras.

Figure 3.62 High-speed camera spec chart. (Image courtesy of Jim Matlosz.)

Figure 3.63 High-speed camera physical chart. (Image courtesy of Jim Matlosz.)

In Figures 3.62 and 3.63, the cameras listed are the most popular and those in current use as of this writing (early 2010). Photosonics, no doubt the leader in high-speed film acquisition, has the largest collection of cameras and formats available. These cameras are built and many of them are maintained around the world to this day by Photosonics Inc. itself. The reason for a detail such as this is that you must consider reliability when using high-speed photography. (For example, the Photosonics 4C will photograph up to 2500 fps, which means the film is traveling through the gate at roughly 102 mph—a thousand feet of film lasts a mere 8 seconds. With these types of speeds, a well-tuned machine is of utmost importance to not only capture the shot but also capture it well.) Photosonics also offers the fastest of all film cameras on the market to this day for motion picture use.

Other cameras that offer high speed options are: Wilcam, with its high speed Vista-Vision, 8-perf cameras; Arriflex, with the 435, peaking at an impressive and very stable 150 fps; and the legendary Arri III, a workhorse of the motion picture industry. In addition, Panavision offers the Panastar and there is the classic Mitchell camera, still in use nearly 100 years after it was first constructed. There were many makes of Mitchell cameras and all seem to look the same. The best way to determine if the Mitchell you are using is high speed is to determine whether it has a metal drive gear. Also a myth about the Mitchell camera is that it need be literally dripping with oil to run properly. Proper oiling is just that—proper: no more, no less. Too much camera oil can potentially leak onto the film and cause issues in processing and image quality. With 16mm high-speed film cameras, the choices are more limited. Once again Photosonics has the greatest number of offerings in both super and standard 16mm. Otherwise there is the Arri 416+HS, a camera that was introduced in 2008, which possesses all the latest bits of technology including speed ramps and integrated iris and remote capabilities.

Digital high-speed acquisition, still in its infancy, but no doubt moving as fast as all other types of digital cameras, has some very useful applications for both visual effects as well as practical effects. The largest manufacturer of high-speed digital cameras is Vision Research with its varied line of Phantom cameras; other cameras are the Weisscam HS-2, NAC, Photron, and the Red One. There are some pitfalls to be aware of with high-speed digital though. Almost all of these cameras can achieve higher frame rates by reducing image size and resolution. Often, however, this may require windowing in on the sensor (i.e., using a smaller portion of the sensor, acquiring less resolution; for example, reducing a 35mm frame to 16mm). This is a detail that is very important and something that may be overlooked or considered a nonissue. If confronted with this issue, there is most likely a full-resolution camera available to achieve your high-speed goals. As always, proper research and application should be considered to achieve the highest quality results.

All of the above-mentioned cameras, both film and digital, do a fairly good job of capturing images, some better than others. For best results, testing is highly recommended. Do not rely on sales brochures, salespeople, conjecture, the Internet, fancy demo reels, or films you have seen in the theater to make a decision. Unless you were there working on the film with the camera and seeing it in post you may get yourself in trouble. With all of the intense post techniques used today, only an educated VFX Supervisor can make the proper decision.

Technicians

Along with just about any of these cameras, film or digital, comes a technician: a person dedicated to keeping the camera working, answering any and all questions, as well as proactively troubleshooting any issues that may occur, fixing them on site, and extending the education of the procedure beyond this writing. It cannot be stressed enough how valuable a good, well-trained, seasoned technician is. Many of these professionals have had years of experience with these cameras. They are kept up-to-date on the latest developments and are skilled in fixing most issues that may occur. These technicians also have a lifeline to manufacturers and other seasoned technicians that allows them to master and overcome many, if not all, potential issues. This is not to say that high-speed photography has issues, but one must consider, with film cameras, the speed at which the film is traveling, or with digital cameras, the speed at which it is capturing data and a superior high-resolution image. At times the cameras are teetering on the brink of discovery, every moment counts, and chances are when shooting high speed, the actual moment is fleeting.

Director of Photography

It is not a question of whether the DP is a master of high-speed photography, but more a question of does he or she have a good understanding of high-speed photography? Common mistakes in high-speed photography are underexposed elements and using the wrong lights. In fact, if you ask most technicians they will tell you the same. And while the VFX Supervisor may have no control over who the DP is, a few conversations and a bit of research prior to shoot day may help to alleviate many headaches and unwanted disagreements.

Lighting

When it comes to lighting for high-speed photography one must simply consider that for every time the speed doubles, one stop of light is lost (i.e., 120 to 240 fps is a one-stop loss), and with this loss comes the need for a greater amount of light. It’s not uncommon to light a water droplet with a smattering of 20k lights. With this same thought comes the concern about lighting a larger area for proper exposure at the proper speed. When dealing with film cameras and light, one must first understand that many film high-speed cameras have limited shutter angles (see chart), which equates to less exposure and again the need for more light.

In addition to the concern about amount of light one must consider the type of light. While HMI lighting may offer a greater amount of light relative to tungsten, the electronic ballast needed to power the HMI light renders a flicker past 150 fps. Although this theory may be disputed, it has been proven time and again, to be true. That is not to say that HMI lighting cannot be used in flicker-free mode, but consider that even after testing, the light may flicker differently at different angles, relative to the camera. It may also flicker depending on the age of the ballast and duration of light use in a specific shot. The bottom line is that there is no surefire way to know flicker exists even through testing using HMI lights.

At the same time with digital cameras, one can see light flicker on a waveform monitor and troubleshooting can begin. Most qualified technicians can easily identify the flicker and inform the proper channels prior to shooting. As far as exposure goes, it is still a good idea to overexpose—get a solid healthy negative when shooting film. Digital on the other hand is much more critical; overexposure of any kind will lead to unwanted fringing and white levels that are deemed unusable. Underexposure in digital cinematography can lead to noisy black levels and render a muddy, unappealing, unusable image with both film and digital. Appropriate contrast levels should be maintained for both film and digital.

Application

As stated earlier high-speed photography was once limited to miniatures and simple visual effects, it eventually began to migrate into sports and be used for dramatic impact within movies and commercials. With the advent of digital high speed cameras, high-speed photography can now be used for the simplest of creative ventures. For the VFX Supervisor the question of which camera is right for the job is at hand. While there are arguments for both film and digital acquisition, it is still up to the educated professional to make that decision, a decision that is often based on application of effect, budget at hand, and complexity of elements to be shot. The choice of shooting speeds and needs will vary greatly and seem to be a bit more open due to digital postproduction techniques; although slowing down an image in post to achieve a slower frame rate (i.e., shooting at 48 fps slowing to 96 fps) is possible, it is not recommended. However, shooting an element at a higher rate (i.e., shooting at 500 fps speeding up to 250 fps) and then speeding up in post has been very effective, so the new and simple rule for the most part is that if you are unsure, shoot it as fast as possible. Be forewarned that this philosophy does not mean that proper homework and planning are not needed; the more prepared and educated one is on proper speeds, the less reasoning and potentially future post work will be needed.

Elements

When considering elements, pieces of a visual effects puzzle shot in separate pieces, such as dirt, body parts, and debris, one must still consult proper frame rates per scale. These scale charts are tried and true and it is best to stick with these techniques. When shooting on film it is possible to shoot these elements with the most logical shooting technique, blue screen, green screen, black or white. As long as the screen and the elements are lit properly, the elements will be fine. Shooting on blue and green screen with digital high speed cameras is not as simple as shooting with film or even basic digital cameras. Testing exposures and processing the data as far as possible, through pipeline to finish, is highly recommended. These cameras, although great for elements, occasionally have issues with process screens. When it comes to water and fire, more thought must be used. Water only has one size, but variable speeds and proper depth of field will imply mass. Although one may be inclined to shoot a pool of water for the ocean at the highest frame rate possible, one must also consider depth of field. In many cases it is better to have more depth and less speed to convey size, and since depth of field (or f-stop) and frame rate work hand in hand relative to impact on exposure, choosing the proper speed for the proper scale should be observed. Elements should be shot as large as possible. Often overscale models of elements are created to avoid side affects like strobing and to increase the quality of the element themselves. Otherwise shooting the element in the proper framing should be observed. Things like lens distortion and travel of the element through the frame should also be observed.

As for fire, it is pretty much agreed that film will still render a better fire element than digital. It’s a fact that film just has greater latitude and fire has a massive range of exposure from the brightest white to the darkest black. Popular opinion states that it is of the utmost importance to capture the greatest nuances of the fire to ensure the best look. When shooting fire, tests should be photographed, with a full range of exposures and frame rates, once again observing f-stops and speed to convey the right scale relative to your production.

Technique

There are certainly a few techniques and tips that should be observed to make your high-speed shoot go well. First and foremost, if you are shooting film, be very aware of start/stop times and how much film you will be using. When shooting with film and a camera technician or 1st AC, it is best to establish a streamlined verbal or visual cueing system of camera roll, camera speed, action, and cut. This will help to not only conserve film but also establish a safe, fluid, and productive shooting environment.

The same techniques should also be observed when shooting on a digital camera. However, the run and stop times on most digital high-speed cameras can be varied greatly. Techniques like post-trigger work more like a film camera, where you hit record and the camera records until it runs out of memory and stops. Another technique is pre-trigger, which means the camera will continuously record as much data as the buffer can handle and loop until the camera is cut. This second technique is generally more favorable, and it should be discussed with your technician to allow for maximum productivity.

Locking Down the Camera

Many film cameras have a tendency to vibrate, some more than others when running at selected high speeds. To ensure a solid and steady plate, the grip department, with the assistance of the camera technician, should lock down the camera and reduce this vibration. Of note, the Photosonics 4C, although an amazing camera, does have the largest amount of vibration as well as image weave due to the fact that it is not a pin-registered camera. Photosonics Inc. has made every effort to reduce these issues, often with great results. If, however, what you are shooting allows for camera movement, you will notice that much of the weave and vibration are considerably reduced upon viewing. Digital cameras have no vibration because basically they have no moving parts; however, it is still a good idea to lock this camera down when shooting critical plates to avoid any unnecessary human error.

Video Assist

All high-speed cameras have video assist. Film cameras all have standard-definition 30-fps video assist, and most are flicker free. High-speed digital cameras on the other hand offer a wide variety of real-time high-definition output; the HD video varies according to manufacturers and even your needs. It is highly recommended to take advantage of this HD signal; there is more advantage to capturing it than one could argue for not capturing it. Note, however, that this HD output is not always recommended for use as your hero footage. Although the image may appear to be very usable, on second inspection one may see anomalies generated by the camera performing a real-time conversion of a raw image.

Post

Post-production for film cameras is standard procedure; prep, process, and transfers require no special handling. Currently, digital cameras vary greatly depending on the manufacturer, current software, post facility, and what the final delivery format is. You should capture a RAW image whenever possible, first and foremost. Testing of the post technique you would like to establish should be performed. Capturing a standard Macbeth color chart and a focus chart is a good place to begin to establish color, focus, and resolution. Because all of these factors can vary greatly depending on the post technique, testing should be performed in a number of ways.

SUPERVISING MOTION CONTROL

Mark H. Weingartner

Motion control—these two words strike fear in the hearts of ADs and put scowls on the faces of UPMs,⁵² bile on the tongues of DPs, and lines on the foreheads of directors.⁵³ It is considered by many to be the methodology of last recourse, and while developments in camera tracking software have reduced the need for motion control on live-action sets, judicious use of motion control can make difficult shots easy and impossible ones possible.

What Is Motion Control?

A motion control camera rig is one that uses computer-controlled motors for accurate reproduction of camera movement with respect to time. Motion control rigs range from motorized pan/tilt heads to cranes with eight or more motorized axes of movement. A motion control programmer operates the rig with its highly specialized computer—choreographing camera moves that can then be edited in order to smooth out bumps, change timing, or adjust framing. In addition to moving the camera, the computer can be used to control lighting effects and movement of models or prop elements. Different types of rigs have different capabilities, strengths, and weaknesses—an understanding of how motion control can be used will inform equipment choices.

Four Basic Uses for Motion Control:

1.  Precise choreography of camera movement

2.  Repeat passes of a camera move

3.  Scaling (temporal and/or spatial)

4.  Import/export of camera move data

The first three functions use pretty much the same hardware, while the last function can be accomplished either with traditional motion control equipment or with encoder sets, which can be attached to some live-action dollies, cranes, and associated grip equipment. In all of these cases, the choice of equipment is important. The supervisor must balance portability and speed of setup and execution against reach and the capability of the rig. A smaller lighter rig that is faster to set up is useless if it cannot get the camera where the director and DP want the camera to go. Similarly, choosing a large rig running on wide, heavy track sections for a simple shot that does not require that much reach only slows production down and reminds them how much they hate motion control. Some rigs are much noisier than others, whether due to their mechanical construction, motor and drive systems, or both. Knowing whether a shot is an MOS⁵⁴ shot or a sync sound shot will influence rig choice.

While traditional motion control programming involves carefully setting key frames and using the software to fill in and modify the resultant camera move, many motion control vendors have developed multiple input devices that allow live-action crew members to control the rig directly while lining up and shooting the first pass of a shot. These input devices range from encoder wheels used to operate a pan/tilt head and follow focus controllers operated by the focus-puller, to encoded pan/tilt heads that can be used as Waldos⁵⁵ to control the swing and lift axes of a motion control crane.

The decision to program using key frames versus learning a move controlled by live-action crew members using these input devices is situational. Often the programmer will program one or more axes (track or lift, for instance) for a move before the shot and then have the camera operator control pan and tilt while the 1st AC pulls focus to film the first pass. Sometimes a dolly grip will control the track axis of a rig in order to feather the move based on the action. Again, there are political as well as technical aspects to these choices: keeping a Director of Photography in his or her comfort zone by letting the shoot crew control the camera movement directly can be a useful goal in and of itself.

Figure 3.64 Grip operates motion control crane with Waldo. (Image courtesy of Mark H. Weingartner.)

Performance Choreography

Motion control can often make a difficult shot much easier to operate. Examples of this use of motion control include the classic “pull out from an ECU⁵⁶ of an eyeball to a wide shot” and its close cousin, the “push in from a wide shot to an ECU of an eyeball.” Motion control has also been used for very precisely choreographed shots such as the slow reveals used in the popular Corona beer commercials. The capability of programming the combination of zooming, focusing, dollying, panning, and tilting necessary to create these shots allows a degree of precision that human technicians cannot easily achieve. The ability to program the various axes separately and to edit the moves on the computer to smooth out problems allows the director and DP to fine-tune the shot until they are completely satisfied.

Sometimes a shot calls for a camera move that requires either a faster motion or more abrupt acceleration or deceleration than can be achieved with human-powered camera equipment. These shots can often be achieved with motion control rigs because of their powerful motorized axes.

Multiple-Pass Photography

Obvious examples of multiple-pass live-action motion control are twin shots where one actor plays two or more characters within a shot. Less obvious are soft split shots where actors and animals act in different passes, or actors run away in the first pass from a physical effect shot in the second pass. Another use of live-action motion control has been for shooting clean passes for rig removal—removing special effects tracks or other rigging for vehicle stunts, for instance.⁵⁷ Multiple-pass work also includes situations where motion control is used to duplicate a foreground greenscreen camera move when shooting the background plate, or conversely when playing back a move in order to shoot a greenscreen element to be composited into an existing shot.

Scaling

Motion control can be a powerful tool when either spatial or temporal scaling is required.

A simple example of temporal scaling would be a live-action shot with a time-lapse sky—clouds scudding across the sky and sun angles changing rapidly while the foreground action moves at normal speed. Accelerated actor motion would be another case of temporal scaling. These effects are accomplished by shooting different passes at different speeds. One of the big gotchas in temporal scaling occurs when one shoots a pass at normal speed and then attempts to play it back at a higher speed to create a slow motion effect. When playing a move back at a higher speed than recorded or programmed, the increased speed may exceed the physical limits of the rig itself, causing motors to stall or causing the rig to fail to repeat the move accurately. For this reason, it is advisable either to:

1.  shoot the fastest pass first or

2.  run a quick speed test before signing off on a hero pass if the hero pass is the slow pass.

While many people are familiar with the scaling done when combining live-action motion control shots with miniature motion control model work, a less familiar use of motion control involves shooting live-action shots of a normal-sized character interacting with a miniature or giant character. In these situations, a motion control move is scaled up or down and repeated with the othersized character. The two elements are then composited in post.

When shots are scaled up or down, the rule of thumb is that angular camera movements and focal lengths remain the same, but linear movements scale proportionally. For example, if the big shot involved the camera tracking in from 10 feet to 5 feet and panning left 20 degrees, in a half-scale version of the shot, the camera would start 5 feet away and track in to 2 feet 6 inches away while panning left 20 degrees.

Now the tricky part: While the actual camera move scales according to this rule of thumb, the individual axes of the motion control rig do not necessarily scale that simply. Due to the geometry of the rig, the horizontal component of the arm’s arc when it is going up or down is very different at different angles, so lowering the arm to get the second pass starting height correct, for instance, changes how that arm’s move will scale.

The good news is that the software designers who have created the two most prevalent motion control packages have created tools within the software that allow the programmer to scale complex moves by taking all of this into account. This complexity, however, can lead to the same issues of exceeding the speed or acceleration limits of one or more axes of a rig. As with temporal scaling, it is important either to shoot the big shot first or to do a quick speed test after shooting the small shot before resetting for the next pass. Additionally, one runs the risk of creating a shot that the motion control rig will not be able to scale up due to its physical size. With these pitfalls in mind, it is prudent to spend the time in consultation with your programmer or motion control supplier before the job to ensure that you have the right rig on set when you start working.

Import and Export of Camera Move Data

In a world before optical flow tracking, motion control was one of the technologies of choice for matching bluescreen or green-screen foregrounds to miniature or location background plates, and vice versa. The development of camera tracking software packages has greatly simplified and automated the painstaking task of extrapolating camera movement in a shot and applying it to the other elements for the composite. Situations still exist, however, where recording camera positional data against a frame-accurate time base or streaming camera positional data to drive a real or virtual camera’s movement is advisable.

The equipment used for this in the live-action world can be divided into two types:

1.  encoding devices that record angular or linear movement of the various axes of movement of a camera, whether it is moving on a pan/tilt head, dolly, jib, or crane, and

2.  motorized motion control rigs that move the camera using a motion control program and programmer.

Data obtained by encoding live-action dollies, cranes, heads, etc., can be used to drive a motion control rig to shoot another element for the same shot. This data can also be applied to a virtual camera in a 3D program to generate the correct movement on CG elements or to drive a real-time visualization system such as EncodaCam.

Several suppliers of motion control equipment have created encoder kits that clamp on to existing cranes, dollies, and heads. There are also modified fluid heads and geared heads that have integral encoders. Some camera systems can output lens focal length and focus data as well, though these are not universally available. When using encoding data to drive a motion control rig, it is important to choose a rig that is capable of executing the move defined by the data with respect to both the physical operating envelope and the speed.

The Data

It is important to understand how the data will be used and to coordinate the formatting and transfer of the data from the motion control programmer to the visual effects house. When using a motion control rig, the programmer may choose to work in virtual axes, a mode in which the data recorded by the program is expressed as pan, tilt, roll, and xyz translations or, alternatively, by programming the various articulation axes directly, with the same results as with encoded rigs as described above.

If the data is not in virtual axes, it is vital for the facility to have all of the relevant dimensions of the rig on which the camera is moving so that they can model it in their 3D software. The actual move data is generally delivered as ASCII⁵⁸ files, which either provide a numerical value for each movement axis for each frame of the move or, if in virtual axes, a linear numerical value for xyz and an angular numerical value for pan, tilt, and roll, as well as look-up table outputs for focus and zoom. The beauty of using ASCII text files is that they can be displayed, printed, and read by humans—and it is very easy to write a routine that looks at an ASCII text file and imports that data into a 3D CG program. If the data is going to be used to drive a motion control rig, it may be preferable to deliver the data as part of a move file—a proprietary file that can be read by another motion control computer. The motion control programmer will be able to advise which type of file will be of more use, but when in doubt, copy both, since these files are trivial in size.

A text file full of numerical angle and distance values is pretty much useless unless the user knows what they relate to. When sending encoder or motion control data to a facility or to a stage where the shot will be played back on another motion control rig, it is imperative that the end user of that data know how it relates to the set on which it was recorded. The data should be accompanied by data sheets showing the placement and orientation of the camera rig (whether motion control or encoded live-action equipment) relative to the set and to the actors, and describing the units, direction, and calibration of each of the move axes. Lenses should be noted by focal length and serial number. If the camera is not mounted nodally,⁵⁹ the nodal offsets should also be described and, if necessary, diagrammed.

Types of Motion Control Systems

Motion control rigs can be roughly split into live-action rigs and model/miniature/stop-motion rigs. The differentiation is primarily between rigs that can work at real-time speeds (i.e., working with production cameras that film at 24 fps) and rigs that are designed to move slowly while photographing at much slower frame rates, typically on the order of one or two frames per second. That said, there is no reason that one cannot shoot a scale model with a rig that is capable of live-action speeds.

Live-Action Rigs

Live-action motion control rigs are usually used to shoot multiple passes of a shot that is meant to look as though it has been shot as straightforward 1st unit photography, and to some degree, form follows function. Live-action rigs are designed to interface with live-action cameras, lenses, and accessories, and their operating envelopes roughly emulate those of the heads, dollies, and jibs used in 1st unit photography.

Pan/Tilt Heads

The most basic of motion control rigs, a pan/tilt system consists of a motorized pan/tilt head, focus zoom and iris motors, a motion control computer with motor drivers, and a bloop light (which is held in frame at the beginning or end of the shot in order to put a one-frame flash or image—a bloop—on the film to aid in synchronizing passes). This type of system will typically include a set of encoder wheels that allows a camera operator to control the head in real time (as with any remote head) and a set of focus/zoom/iris controls that allows the camera assistant to adjust these settings on the fly when laying down the first pass.⁶⁰ Alternatively, the motion control programmer may program a camera move, controlling all of these parameters directly from the computer.

The pan/tilt head may be mounted anywhere, but it can only execute a true motion control shot if the device it is mounted on is completely static with respect to the scene being shot. Some motion control heads can accommodate a roll axis as well. This rig is capable of emulating a pan/tilt head on a tripod or other fixed mounting position and is ideal for simple twinning shots, morphs, and background changes where the shot is not all about camera movement. It is also a handy way to operate a camera remotely when a human operator might be in danger (e.g., car stunts) in the specific case where a second pass is beneficial for rig removal.⁶¹

Pan/Tilt Head on Track

The next step upward in complexity is the addition of a sled dolly on a track. Generally this is done by putting a tripod or fixed rig on a motion control sled dolly. The advantages of this type of rig are its portability and ease of setup. The disadvantage is that once the camera is tracking, the Director of Photography will often bemoan the inability to boom up or down during the shot, and the cost savings incurred by renting this simple rig can be somewhat offset by a Director or Director of Photography not getting exactly what he or she wanted.

Small Crane on Track

These rigs have jib arms of 4 to 6 feet in length and their movement capability tends to emulate a camera on a live-action dolly, but with some additional abilities. Though various designs appear to be quite different, conceptually, they can be described in terms of axes.

•   Track: generally straight track, though some rigs can run on special curved track.

•   Swing: the rotation of the entire mechanism above the track dolly.

•   Lift: the raising and lowering axis of the jib arm.

•   Pan, tilt, focus, iris, and zoom: as described earlier.

This is the most prevalent type of rig in live-action use—representing a good compromise between operating envelope and ease of setup and use. Not only can this rig achieve the full range of motion available on a live-action dolly on straight track, but with judicious use of the swing axis and careful placement of straight track, this rig can often emulate a dolly move that includes a combination of straight and curved track. Additionally, since the camera operator is operating the head remotely, it is possible to make shots that could not be done on a standard dolly or even a small jib arm with a manual head.

Figure 3.65 A very large motion control crane on its trailer-mounted track. (Image courtesy of Tom Barron.)

Big Cranes

There are a number of larger cranes that are still relatively easy to deploy on location. Built with the same axis configuration as small cranes on a track, these rigs sport jib arms of 8 × 12 feet and, in a few cases, as long as 24 feet. While the small cranes generally run on 24.5-inch gauge or 32-inch gauge track, most of the big cranes run on wider track and their larger size and wider track require a bit more time and effort to set up.

A couple of cranes on the market feature the best of both worlds: narrow, lightweight track and components coupled with a long arm. These recent designs take advantage of experience and modern materials and also take into account the fact that in the world of contemporary digital compositing, very small errors are not that hard to accommodate, compared to optical compositing, where steadying images was very difficult indeed.

Figure 3.66 VistaVision camera on a large motion control crane. (Image courtesy of Mark H. Weingartner.)