9
Widescreen composition and TV
Introduction
After more than 20 years of argument about what should be the technical standards for a universal higher definition television system (see Chapter 7), the only consensus arrived at was to change the shape of the screen. The new aspect ratio was to be 16:9. In most countries, apart from a gradual transition from analogue to digital production/reception, this new screen shape was the only change that survived the international aspiration towards an HDTV service.
The adoption of a 16:9 television screen was a compromise that enabled programme makers to avoid producing programmes in two aspect ratios during the transitional period, provided precautions were taken in the composition of shots during production. That is, each shot was a compromise between the two aspect ratios (see ‘Protect and save’, below). Just as film makers had faced many compositional problems in trying to accommodate two or more incompatible viewing formats on the same negative, so television programme makers, 30 or 40 years later, were faced with similar irreconcilable framings.
Secondly, it allowed improved transmission of feature films although still requiring either the much disliked (by the film industry) panning and scanning or, alternatively, transmitting the whole frame with black bands at the top and bottom of the new 16:9 screen.
Letterboxing
Cinema widescreen film production continues in aspect ratios of 2.35:1, 1.85:1, 1.66:1 and others. There is also a huge library of film and television material shot in the ratio of 1.33:1 (4:3). If the whole of a frame of a format wider than 16:9 is transmitted on television then it will fill only a portion of the 16:9 screen. The remaining part of the screen will be filled with black bands top and bottom. The wider the aspect ratio, the broader the bands. This is called letterboxing. If uncropped 4:3 aspect ratio productions are shown on a 16:9 screen they will be accompanied by black vertical bands left and right of the screen (side curtains).
The use of only a portion of their display screen was resisted by viewers in some countries, for example, some UK viewers complained to the BBC that they paid a full licence fee and therefore they wanted a full screen image! Other countries, those who were accustomed to viewing foreign programmes with subtitles inserted in this bottom black band letterbox area, were unworried by watching the whole of a widescreen feature film with no cropping. Perhaps broadcasters should attempt to educate their viewers with the truth that by filling their TV screens with a film image they are not getting more for their money but less. When viewing on 4:3 screens, they are denying themselves up to 40 per cent of the film they are watching (see Figure 8.1).
Similar complaints were voiced when colour TV was introduced and a monochrome film was transmitted. The viewer then argued: ‘I bought a colour TV set and I demand that all programmes I receive should be in colour.’ This led to the synthetic colourizing of classic black and white films such as ‘Casablanca’ (1942), Laurel and Hardy, and the Astaire/Rogers films in order to make them more ‘acceptable’ to a section of the viewing public.
Aspect ratio conversion
As we have discussed, some widescreen film makers, when they saw their compositions massacred on TV, adopted a composition policy of keeping their main groupings and action in the centre of frame. This defeated the advantages of the widescreen shape but it safeguarded their product for a bigger market. Television programme makers have had to adopt the same policy to service display screens that have different aspect ratios.
Electronically, a picture could be cropped or expanded to fit any shape, but this would lead to loss of information, loss of resolution and possibly picture distortion when images are stretched to fit a different shape to their production aspect ratio. It would also destroy the compositional skills of the originators of the programmes.
Some type of aspect ratio conversion has to be employed either before the programme is transmitted or at the receiver. Several countries utilize a compromise aspect ratio of 14:9 to bridge the gap between 16:9 production demands and 4:3 receivers. The ratio converter chops out portions of the left and right of the frame for 4:3 viewers who watch with a small black border top and bottom of the frame.
16:9 set-top aspect ratio conversion is also under the control of the viewer who can select full frame with black side curtains left and right of the image when watching a 4:3 transmission or partial expansion of the 4:3 frame to a 14:9 shape when information is equally lost at top and bottom of frame. Full expansion of the 4:3 image to fill the 16:9 frame (zooming in) with information lost balanced between top and bottom or distributed according to picture content. Picture content, of course, changes with each shot. Some set-top aspect ratio converters also provide for non-linear distortion of the horizontal part of the 4:3 frame to fit a 16:9 TV set.
Figure 9.1 The viewfinder is set to display the full 16:9 picture with a graticule superimposed showing the border of a 14:9 frame and a 4:3 frame
Figure 9.2 If the ‘protect and save’ viewfinder indicators are ignored when framing, for example, a golf shot in 16:9 aspect ratio (a), viewers watching a 4:3 picture will have a poorly composed picture (b) and no indication, when the ball exits right on their viewed picture, if the ball has entered the hole. With this framing, it is likely that even if the programme is transmitted in 14:9 aspect ratio, many viewers (with over-scanned TV receivers) will not be able to see the hole on the right of the frame present in the original 16:9 framing
Transmitting a mixture of aspect ratio formats will always need conversion unless the unlikely step is taken to scrap all 4:3 programme material when everyone has converted to 16:9 reception. Black and white movies continue to be popular 40 years after the introduction of colour television. The decision to change the TV aspect ratio was not simply a technological change, it has and will have continuous programme making and programme watching implications.
Protect and save
Cameramen shooting in 16:9 follow a ‘shoot and protect’ framing policy. The viewfinder is set to display the full 16:9 picture with a graticule superimposed showing the border of a 14:9 frame and a 4:3 frame (Figure 9.1). Significant subject matter is kept within the 14:9 border or, if there is a likelihood of the production being transmitted in 4:3, within the smaller 4:3 frame. The area between 16:9 and 14:9 must be still usable for future full digital transmissions and therefore must be kept clear of unwanted subject matter. This is similar to the problems experienced in feature film productions that were shot in 4:3 but were intended to be projected in the cinema with a hard matte in widescreen. ‘Shoot and protect’ attempts to avoid the hazards of multi-aspect viewing by centring most of the essential information and avoiding any unwanted elements at the extreme edge of the horizontal frame that may be seen in the future.
Shooting for two formats
Shoot and protect negates the claimed advantages of the widescreen shape because for the transitional period the full widescreen potential cannot be used. The film and television widescreen format has the potential for dynamic and arresting compositions; the problem is that the width of the format both in film and TV is rarely exploited. More often than not, the director or cameraman has to safeguard the composition for showing in other format sizes.
Some broadcasters who have adopted a 14:9 transmission aspect ratio have not fully converted all their cameras to 16:9 shooting. Camera operators using 4:3 cameras need to increase the headroom on a shot because the top and bottom of the frame will be cropped when it passes through an aspect ratio converter. This often leads to poor framing because the only guide the operator has is usually an inaccurate piece of gaffer tape indicating a notional top of a 14:9 frame. Not for the first time, in many cameramen's experience, has ‘state-of-the-art’ technology been made workable with gaffer tape. For some years, one of the first questions a cameraman will ask is ‘what aspect ratio is the programme going to be transmitted in?’.
Composing for 16:9
If anything, television is more of a ‘talking heads’ medium than cinema but the advent of the larger, wider aspect TV screen has tended to emphasize location and setting. There are compositional advantages and disadvantages in using different aspect ratios. Widescreen is good at showing relationships between people and location. Sports coverage benefits from the extra width in following live events. Composing closer shots of faces is usually easier in the 4:3 aspect ratio but, as in film, during the transition to widescreen framing during the 1950s, new framing conventions are being developed and old 4:3 compositional conventions that do not work are abandoned. The shared priority in working in any aspect ratio is knowing under what conditions the audience will view the image.
One of the main compositional conventions with 4:3 television framing is the search for ways of tightening up the overall composition. This is partly due to fashion but also because viewing a TV picture occupies a much smaller zone of the human field of view compared with cinema viewing. Wide shots on television with small detail are not easily perceived on an average size receiver. Tight compositions eliminating all but the essential information have traditionally been preferred.
A conventional TV single can cause problems in framing in wide-screen and bits of people tend to intrude into the edge of frame. This is sometimes called a ‘dirty single’. Headroom has tended to be smaller than 4:3 framing and there are some problems in editing, particularly if the cut is motivated by action on the edge of a 16:9 frame, which may not be visible to 14:9 or 4:3 screen viewers.
The problem with the video compositional transition to widescreen is the inhibition to use the full potential of the 16:9 shape because the composition has to be all things to all viewers. It must fit the 14:9 shape but also satisfy the 4:3 viewer. It is difficult to know when the full potential of the widescreen shape can be utilized because, even if the majority of countries switch off analogue transmissions at some time in the first decades of the century, there will probably be billions of TV sets worldwide that will still be 4:3 analogue.
Widescreen television composition faces some of the same problems that film solved 40 years ago. Many film and television scripts require the speaker and the listener to be in the same frame. Two people talking created the mixture of close-ups, medium close-ups and overthe-shoulder two shots that form the basic shot pattern of many scenes. Shots tighter than MCU can be difficult to frame for 16:9 and the tendency is to continually tighten to lose the ‘space’ around the ears. Low angles appear more dynamic than similar 4:3 aspect ratio shots but hand-held camerawork in 16:9 can be very obtrusive and distracting.
When people are being interviewed, there is an optimum distance between them where they both feel at ease. The single shot on 16:9 has the problem of avoiding being too tight and producing a ‘looking through a letterbox’ look whilst avoiding being too loose and getting the interviewer in shot in the ‘looking space’ of the interviewee. The compromise is an over-the-shoulder two shot but care must be taken in the reverse to get good continuity in body posture, etc. There needs to be greater physical separation between presenters, interviewers, etc., to avoid edges of arms, shoulders creeping into the edge of the frame. This has a knock-on effect on the front-on two shot where the participants now appear to have too much space between them.
The advantage claimed for 16:9 (especially HDTV) is that the increased size of screen and the improved definition in wide shots is so good that fewer close-ups are required. This can create its own problems in editing. Wide shots need to be sufficiently different in their distribution of similar objects to avoid jump cuts in editing. Typical bad cuts can happen with seascape horizons (yacht racing, etc.) where the yachts jump in and out of frame if the horizon is in the same position in successive shots. The same ‘jump’ can happen with some types of landscape. A good cut needs a change in shot size or significant change in content to be invisible. Decisions about edit points on slow entrances and exits that hover on the edge of frame can be very difficult when shot 16:9 if the majority of viewers are watching a 14:9 frame on a 4:3 screen.
There is the advantage of 16:9 allowing a wider shot with less sky or ground, and square-on shots of buildings can replace the angled shot necessary in a 4:3 frame to include all the structure. Close-ups of strong vertical subjects (e.g., fingering on clarinets and saxophones) are a problem but keyboard shots are easier, and ‘edgy’ objects on the edge of frame do not seem to be so distracting in the wide format.
Film at least had one advantage over TV widescreen. At some time in the film's history it was usually shown in full frame width in a cinema. Widescreen TV at the moment, in many countries, is in effect 14:9 –nearly widescreen. It does not satisfy the 16:9 viewer, and it doesn't help the 4:3 viewer who is now getting a smaller image on his set.
Fidgety zooms
Widescreen working has brought with it a new camera-operating pitfall. Whilst broadcasting an opera, two singers were framed in mid-shot and transmitted to analogue viewers in 14:9 and to digital viewers in 16:9. They swayed away from each other towards their respective edge of frames with a gap opening between them. It became an awkward, clumsy composition that quickly needed adjustment.
The cameraman watching his/her 16:9 viewfinder with a 14:9 ‘safe area’ obviously let them get close to the edge of the 14:9 frame. In a 16:9 viewfinder picture it probably began to make quite a good composition with the two-shot spread across the frame. In 14:9 it began to get very uncomfortable. A simple operating solution, when the action begins to burst out of the frame, is to loosen the shot. It is almost nearly always unobtrusive because the action naturally forces the shot wider. It is good ‘invisible’ technique, which is the bedrock of camerawork. Watching in 16:9 however, correcting the shot for the 14:9 viewer would result in a fidgety zoom-out as the two subjects were swaying into a more balanced composition. In 16:9, there was no obvious visual reason to slightly widen the shot. It would become one of those nasty fidgety zoom corrections that are inevitable in live TV, forced on the cameramen when covering unrehearsed action.
So the worst of both worlds. The 14:9 viewer gets an edgy, unbalanced shot followed by the 16:9 viewer suddenly having his balanced widescreen shot loosened to unbalance the shot again. In most countries this transitional period is due to last for a minimum of ten years but mixed aspect ratio problems will roll on across the broadcasting world for very much longer and probably will never be resolved.
Transitional period
The worldwide change-over period from mass viewing on a 4:3 analogue set to mass viewing on a 16:9 digital monitor, and therefore mass programme production for 16:9 television, will take many years. The transition period will require a compromise composition but the compositional problems do not end there. The back-library of 4:3 programmes and films is enormous and valuable and will continue to be transmitted across a wide range of channels in the future. The complete image can be viewed on a 16:9 screen if black bars are displayed either side of the frame. They can be viewed by filling the full width of the 16:9 display at the cost of cutting part of the top and bottom of the frame or, at the viewer's discretion, they can be viewed by a non-linear expansion of picture width, progressively distorting the edges of the frame to fill the screen 4:3 aspect ratio.
The same size camera viewfinders used for 4:3 aspect ratio are often switched to a 16:9 display. This in effect gives a smaller picture area if the 14:9 ‘shoot and protect’ centre of frame framing is used and makes focus and following distant action more difficult. Also, the majority of video cameramen are probably the only viewers still watching colour TV pictures in monochrome. Colour is not only essential to pick up individuals in sports events such as football, where opposing team shirts may look identical in monochrome, but in all forms of programme production colour plays a dominant role in composition.
From a cameraman's point of view, the biggest difficulty during the transition period is attempting to find a compositional compromise between the two aspect ratios. If a 16:9 image is transmitted in a letterbox format then all shots can be framed with respect to the 16:9 border. However, most broadcasters still provide an analogue 4:3/14:9 service. There is very little satisfactory compromise that can be made in an attempt to compose for both formats at the same time.
Composition problems will continue while 16:9 and 4:3 simultaneous productions are being shot during the analogue/digital changeover. They neither take full advantage of the width of 16:9 nor do they fit comfortably with the old 4:3 shape. Possibly ten years of dual format compromise production will then join the back library and be transmitted from then on. The only safe solution is the ‘protect and save’ advice of putting essential information in the centre of frame, but that is a sad limitation on the compositional potential of the widescreen shape.
The viewer takes control
Often the widescreen signal is embedded with active format descriptor (AFD) – widescreen signalling to the set, which automatically selects the appropriate display format for the incoming programme. Many 16:9 digital receivers also provide the viewer with control of how the picture is to be displayed. They can select full frame with black side curtains left and right of the image when watching a 4:3 transmission or select a partial expansion of the 4:3 frame to a 14:9 shape when information is equally lost at top and bottom of frame. They can choose full expansion of the 4:3 image to fill the 16:9 frame (zooming in) with information lost balanced between top and bottom or distributed according to picture content. Picture content, of course, changes with each shot, therefore the viewer would need to monitor and adjust for every shot. Some receiver aspect ratio converters also provide for non-linear distortion of the horizontal part of the 4:3 frame to fit a 16:9 TV set.
Figure 9.3 It is a paradox that in some countries, viewers actively dislike the black band side curtains (b) when watching a 4:3 picture on a 16:9 receiver and appear to prefer either the distortion introduced when stretching the 4:3 picture to fit the 16:9 screen (c) or to crop the top and bottom of the transmitted image (d). Unless the viewer constantly monitors and adjust this last option (zooming), subsequent shots may suffer loss of essential information (e)
Some 16:9 TV receivers have a progressive distortion (anamorphic) facility to stretch a 4:3 transmitted image. This progressive rate of expansion across the screen results in the snooker example (Figure 9.3) of the ball's shape changing as it travels across the frame. It is ironic that decades of research and development expended on producing perfect electronic images that are free from geometric distortion can be negated by the touch of a button on a channel changer. Weather presenters change shape from fat to thin as they walk across the frame, or shoot out their arm to double its apparent length when this aspect ratio conversion option is selected.
Inserting 4:3 material into a 16:9 production
In many types of programme there is often the need to use archive material. For example, a sports programme profiling a well known sportsman may be shot in 16:9 when interviewing the main subject but will insert 4:3 material of past sporting success. The 4:3 material can be inserted via an aspect ratio converter into a 16:9 frame but will have large black ‘side curtains’. These will pass unnoticed by a 4:3 viewer watching a 14:9 version of the programme but a 16:9 viewer will have constant obtrusive vertical black bars jumping in and out of the frame whenever the archive material is used.
To make the 4:3 material less intrusive to the widescreen viewer the producer/director may, in post-production, expand the 4:3 material to fit a 14:9 frame losing variable proportions of top or bottom of the 4:3 image. Depending on content, this may smooth out the transition between the formats whilst still inserting narrow vertical bars in the 16:9 viewer's display whenever the archive footage is used. The difficulty comes when, for example, the head, feet and football are hard framed in the 4:3 material allowing no flexibility in cropping top or bottom. Either the head is cut off or the football goes out of frame. This is often solved by expanding the 4:3 image horizontally to fit the 14:9 frame but, of course, this causes distortion to the image and a thin footballer instantly gains weight and becomes fatter and wider. Although this distortion may seem unacceptable it is in fact seen in programmes most weeks.
Film has had similar problems of incorporating different aspect ratios in the same production. In the widescreen film ‘The Guns of Navarone’ (1961) 4:3 aspect ratio film is intercut with widescreen. François Truffaut's ‘Jules and Jim’ (1962) uses Academy ratio first war footage expanded to fill the widescreen aspect ratio. This obviously distorts the early film footage.
If a production is originally shot in 4:3 and is then converted to a 16:9 version by using a selected area of the frame, further loss of image can occur when the 16:9 version is then transmitted in a 4:3 broadcast. Credits and graphics often suffer the most from aspect ratio conversion. An early victim was the opening titles of ‘Picnic’ (1955) starring William Holden. His credit was transmitted on US television as William Ho.
Compilation programmes
One hundred years of film footage provide the raw material for television history programmes that reflect social history and the lives of ordinary people as reflected in film and TV archives. There is no distortion of this primary source, other than what is present in the script treatment, if the images are presented in their original aspect ratio. However, with the growth of 16:9 TV, this authentic material is subject to distortion and cropping to fit the newer aspect ratio. It is, in effect, a distortion of historical material that would be unacceptable to historians if the primary source was written archives. Compilation programmes that review popular culture in the past by showing clips of popular 4:3 television shows also crop and chop the originals to fit the newer shape. They are not doing what they purport to be doing –showing the original programme. They are showing a portion of the original programme.
One of the fallacies voiced when 16:9 screen format was adopted was that, although there would be incompatibilities, these would soon pass after a transitional period. This is based on the assumption that when all new programmes are produced in 16:9 there would be no problems when viewed on 16:9 screens. This limited view completely ignores the huge back library of 4:3 programmes. These programmes are not only very popular and therefore a commercial asset, but there is no sign of a decline in the demand for such titles (e.g., Tom and Jerry cartoons, classic feature films). It is as if a library with a standard shelf size decides to reduce its shelf height and thereafter requires all new books to fit this module. Anything written and printed before this changeover is deemed obsolete. Old book sizes that won't fit are casually cropped or squashed to fit the new shelf height.
What was also ignored in the decision to change to 16:9 was the huge international trade in television and film programming that would involve shooting in one ratio, converting to another and possibly shown in a third aspect ratio with each conversion losing part of the image.
Distortion and definition
Twentieth Century Fox engineer, Lorin Grignon, provided a report on the Cinerama format when it was first introduced and suggested that it had inherent picture distortion. Continually in television studios, in-vision monitors have cropped or distorted images. Distorted pictures in mixed-format programmes are transmitted as standard, which 20 years ago would have been rejected. A magazine advertisement urging the consumer to buy a widescreen television to increase their enjoyment of television football matches, displays a TV set with a footballer whose head is almost out of the top of the frame whilst kicking a football that is almost out of the bottom of the frame. For the advert, they have simply inserted a 4:3 picture into a 16:9 set and obviously missed the irony of urging people to buy a TV set that displays less of their favourite sport than their existing receiver.
Widescreen equals spectacle
The assumption that widescreen equates with spectacle is a throwback to the Hollywood attempts in the 1950s to meet the growing competition of television with ‘spectacular’ productions that TV could not provide. Since that period, there have been many productions that have demonstrated how effective widescreen is when shooting interiors. The ‘visual fluff’ at the edge of a 2.35:1 widescreen image, as one technical commentator described it, ‘was unnecessary, and could always be cropped when transmitted on TV’. The implication of this thinking by vested interests, eager to persuade the public to change the shape of their television sets, is that widescreen composition is simply 4:3 with a little bit of ‘visual fluff’ tacked on to each end of the frame.
The disturbing element in this aspect ratio debate is that frequently the technical quality and economic viability is argued in detail whereas the knock-on effects of cropping and compositional distortions are considered a side issue. The justification of widescreen in the first place was its ability to engage the audience. The practicalities of achieving a compatible widescreen/Academy size television system appear to have swept past that basic point.
A Hollywood studio boss, Adolph Zukor, claimed that Twentieth Century Fox's emphasis on technology in the 1950s, when they introduced CinemaScope, had blinded them to their chief responsibility, which was to make good films. Of course, Zukor at that time was chairman of rival company Paramount who had not signed up to CinemaScope.
Screen size
There is a further consideration in the aspect ratio debate that concerns size of screen. Someone sitting in a front-row cinema seat may have as much as 58° of their field of view taken up by the screen image where the viewing distance is 0.9 times the picture height. This can be reduced to as little as 9.5° when the same screen is viewed from the back row of the cinema at a viewing distance of 6.0 times the picture height. A television viewer watching a 51-cm diagonal tube (21") at a viewing distance of 6.3 times the picture height will have only 9.2° of their field of view taken up by the TV picture. Human vision is aware of about 200° in the horizontal plane (although only fully concentrating on a small proportion of this) and therefore the proportion of the visual area of a 51-cm screen in the home occupies only approximately 4.6 per cent of the maximum field of view of the viewer.
TV or cinema pictures with rapid movement produce no visual fatigue if viewed at a distance equal to about four times picture height, although the resolution of the viewed image should be good enough to allow a viewing distance of three times the picture height. According to NHK research, most people prefer a 5:3 aspect ratio television screen with increased definition although, when viewing landscapes and sport, many people favoured a 2:1 ratio.
1.5″ viewfinders
One of the most significant differences between normal perceptual experience and the experience of viewing an image in a viewfinder, is size. The 1.5" viewfinder image on a standard portable video camera is very small and therefore a condensed version of what can be perceived. The eye can quickly scan a great diversity of detail in the viewfinder image that would not be possible in the original scale. The subject is scaled down and perceptually dealt with in a different manner than the original.
HDTV has a greatly enhanced definition and a greatly increased screen size. Finding focus and the limits of focus on a 1.5" monochrome viewfinder will become more and more critical and demanding. Monochrome TV camera viewfinders are two generations behind the technology of other areas of television engineering.
Endnote, or in a different aspect ratio, NDNOT
The correct aspect ratio in film and television production is virtually ignored except by the director and cameraman who laboured over the original image. The final display of this image is in the hands of commerce whose visual dead eye only takes into consideration stars and action except, of course, when the screen shape is promoted to sell more cinema tickets or to urge consumers to buy new TV receivers.
It will never happen, but the intellectual property control of film makers concerning the correct aspect ratio for their production should be guaranteed from camera to audience display. The audience should see the image uncropped, or squeezed or aspect ratio converted exactly as the programme maker intended. Aspect ratios are not compatible if the chosen frame shape is fully creatively exploited. No programme maker should be asked/instructed to provide images that will fit different formats. To frame for several formats at once will inevitably degrade the production values and result in an inferior product. Panning and scanning destroys the craftsmanship expended on a feature film. Broadcasters should resist audience clamour for a cropped image simply to fill their TV screen. ‘Casablanca’ (1942) in colour, is a different and inferior film to ‘Casablanca’ in its original black and white. ‘2001: A Space Odyssey’ (1968), when viewed on a panned and scanned TV screen is shortchanging the audience even if they want to be duped. They are not enriched by a screen full of image. They are cheated of the complete experience of the film
Summary
In many countries, the 16:9 screen shape will be the only element left of the new worldwide HDTV standard proposed in the 1980s and 1990s.
In order to cater for programme production for two, or possibly three aspect ratio display systems, (16:9, 14:9, 4:3), a policy of ‘protect and save’ has been adopted. This requires essential subject or information to be contained within a 14:9 or a 4:3 aspect ratio.
There are aspect ratio conversion problems when incorporating 4:3 material into a widescreen programme.