INTRODUCTION TO CONVERSION

There are a number of ways in which a video waveform can digitally be represented, but the most useful and therefore common is pulse code modulation, or PCM, which was introduced in Chapter 1. The input is a continuous-time, continuous-voltage video waveform, and this is converted into a discrete-time, discrete-voltage format by a combination of sampling and quantizing. As these two processes are orthogonal (a $64,000 word for at right angles to one another) they are totally independent and can be performed in either order. Figure 4.1a shows an analog sampler preceding a quantizer, whereas Figure 4.1b shows an asynchronous quantizer preceding a digital sampler. Ideally, both will give the same results; in practice each has different advantages and suffers from different deficiencies. Both approaches will be found in real equipment.

The independence of sampling and quantizing allows each to be discussed quite separately in some detail, prior to combining the processes for a full understanding of conversion.

Whilst sampling an analog video waveform takes place in the time domain in an electrical ADC (analog-to-digital convertor), this is because the analog waveform is the result of scanning an image. In reality the image has been spatially sampled in two dimensions (lines and pixels) and temporally sampled into pictures along a third dimension. Sampling in a single dimension will be considered before moving on to more dimensions.

FIGURE 4.1

Because sampling and quantizing are orthogonal, the order in which they are performed is not important. In (a) sampling is performed first and the samples are quantized. This is common in audio convertors. In (b) the analog input is quantized into an asynchronous binary code. Sampling takes place when this code is latched on sampling clock edges. This approach is universal in video convertors.

SAMPLING AND ALIASING

Sampling is no more than periodic measurement, and it will be shown here that there is no theoretical need for sampling to be detectable. Practical television equipment is, of course, less than ideal, particularly in the case of temporal sampling.

Video sampling must be regular, because the process of time base correction prior to conversion back to a conventional analog waveform assumes a regular original process as was shown in Chapter 1. The sampling process originates with a pulse train, which is shown in Figure 4.2a to be of constant amplitude and period. The video waveform amplitude-modulates the pulse train in much the same way as the carrier is modulated in an AM radio transmitter. One must be careful to avoid overmodulating the pulse train, as shown in Figure 4.2b, and this is achieved by applying a DC offset to the analog waveform so that blanking corresponds to a level partway up the pulses as in Figure 4.2c.

FIGURE 4.2

(a) The sampling process requires a constant-amplitude pulse train. This is amplitude modulated by the waveform to be sampled. (b) If the input waveform has excessive amplitude or incorrect level, the pulse train clips. (c) For a bipolar waveform, the greatest signal level is possible when an offset of half the pulse amplitude is used to centre the waveform.

In the same way that AM radio produces sidebands or images above and below the carrier, sampling also produces sidebands, although the carrier is now a pulse train and has an infinite series of harmonics as shown in Figure 4.3a. The sidebands repeat above and below each harmonic of the sampling rate as shown in Figure 4.3b.

The sampled signal can be returned to the continuous-time domain simply by passing it into a low-pass filter. This filter has a frequency response that prevents the images from passing, and only the baseband signal emerges, completely unchanged. If considered in the frequency domain, this filter can be called an anti-image filter; if considered in the time domain it can be called a reconstruction filter. It can also be considered as a spatial filter if a sampled still image is being returned to a continuous image. Such a filter will be two-dimensional.

FIGURE 4.3

(a) Spectrum of sampling pulses. (b) Spectrum of samples. (c) Aliasing due to sideband overlap. (d) Beat-frequency production. (e) 4X oversampling.

If an input is supplied having an excessive bandwidth for the sampling rate in use, the sidebands will overlap (Figure 4.3c) and the result is aliasing, in which certain output frequencies are not the same as their input frequencies but instead become difference frequencies (Figure 4.3d). It will be seen from Figure 4.3 that aliasing does not occur when the input frequency is equal to or less than half the sampling rate, and this derives the most fundamental rule of sampling, which is that the sampling rate must be at least twice the input bandwidth. Sampling theory is usually attributed to Shannon,^1,2 who applied it to information theory at around the same time as Kotelnikov in Russia. These applications were pre-dated by Whittaker. Despite that it is often referred to as Nyquist's theorem.

FIGURE 4.4

In (a), the sampling is adequate to reconstruct the original signal. In (b) the sampling rate is inadequate, and reconstruction produces the wrong waveform (detailed). Aliasing has taken place.

Whilst aliasing has been described above in the frequency domain, it can be described equally well in the time domain. In Figure 4.4a the sampling rate is obviously adequate to describe the waveform, but in Figure 4.4b it is inadequate and aliasing has occurred.

One often has no control over the spectrum of input signals and in practice it is necessary also to have a low-pass filter at the input to prevent aliasing. This anti-aliasing filter prevents frequencies of more than half the sampling rate from reaching the sampling stage. The requirement for an anti-aliasing filter extends to the spatial domain in devices such as CCD sensors.

Whilst electrical or optical anti-aliasing filters are quite feasible, there is no corresponding device that can precede the image sampling at frame or field rate in film or TV cameras and as a result aliasing is commonly seen on television and in the cinema, owing to the relatively low frame rates used.

With a frame rate of 24 Hz, a film camera will alias on any object changing at more than 12 Hz. Such objects include the spokes of stagecoach wheels. When the spoke-passing frequency reaches 24 Hz the wheels appear to stop. Temporal aliasing in television is less visible than might be thought because of the way in which the eye perceives motion. This was discussed in Chapter 2.

RECONSTRUCTION

If ideal low-pass anti-aliasing and anti-image filters are assumed, having a vertical cutoff slope at half the sampling rate, an ideal spectrum shown in Figure 4.5a is obtained. It was shown in Chapter 3 that the impulse response of a phase-linear ideal low-pass filter is a sin x/x waveform in the time domain, and this is repeated in Figure 4.5b. Such a waveform passes through zero Volts periodically. If the cutoff frequency of the filter is one-half of the sampling rate, the impulse passes through zero at the sites of all other samples. It can be seen from Figure 4.5c that at the output of such a filter, the voltage at the centre of a sample is due to that sample alone, because the value of all other samples is zero at that instant. In other words the continuous time output waveform must join up the tops of the input samples.

FIGURE 4.5

If ideal “brick wall” filters are assumed, the efficient spectrum of (a) results. An ideal low-pass filter has an impulse response shown in (b). The impulse passes through zero at intervals equal to the sampling period. When convolved with a pulse train at the sampling rate, as shown in (c), the voltage at each sample instant is due to that sample alone as the impulses from all other samples pass through zero there.

FIGURE 4.6

As filters with finite slope are needed in practical systems, the sampling rate is raised slightly beyond twice the highest frequency in the baseband.

In between the sample instants, the output of the filter is the sum of the contributions from many impulses, and the waveform smoothly joins the tops of the samples. If the time domain is being considered, the anti-image filter of the frequency domain can equally well be called the reconstruction filter. It is a consequence of the band-limiting of the original anti-aliasing filter that the filtered analog waveform could travel between the sample points in only one way. As the reconstruction filter has the same frequency response, the reconstructed output waveform must be identical to the original band-limited waveform prior to sampling. A rigorous mathematical proof of reconstruction can be found in Betts.³

The ideal filter with a vertical “brick-wall” cutoff slope is difficult to implement. As the slope tends to vertical, the delay caused by the filter goes to infinity. In practice, a filter with a finite slope has to be accepted as shown in Figure 4.6. The cutoff slope begins at the edge of the required band, and consequently the sampling rate has to be raised a little to drive aliasing products to an acceptably low level. There is no absolute factor by which the sampling rate must be raised; it depends upon the filters that are available and the level of aliasing products that are acceptable. The latter will depend upon the word length to which the signal will be quantized.

FILTER DESIGN

It is not easy to specify anti-aliasing and reconstruction filters, particularly the amount of stop band rejection needed. The resulting aliasing would depend on, among other things, the amount of out-of-band energy in the input signal. As a further complication, an out-of-band signal will be attenuated by the response of the anti-aliasing filter to that frequency, but the residual signal will then alias, and the reconstruction filter will reject it according to its attenuation at the new frequency to which it has aliased. To take the opposite extreme, if a camera that had no response at all above the video band were used, no anti-aliasing filter would be needed.

FIGURE 4.7

The important features and terminology of low-pass filters used for anti-aliasing and reconstruction.

It would also be acceptable to bypass one of the filters involved in a copy from one digital machine to another via the analog domain, although a digital transfer is, of course, to be preferred.

The nature of the filters used has a great bearing on the subjective quality of the system. Entire books have been written about analog filters, and they will be treated only briefly here.

Figure 4.7 shows the terminology used to describe the common elliptic low-pass filter. These are popular because they can be realized with fewer components than other filters of similar response. It is a characteristic of these elliptic filters that there are ripples in the passband and stop band. In much equipment the anti-aliasing filter and the reconstruction filter will have the same specification, so that the passband ripple is doubled. Sometimes slightly different filters are used to reduce the effect.

Active filters can simulate inductors using op–amp techniques, but they tend to suffer non-linearity at high frequencies at which the falling open-loop gain reduces the effect of feedback. Active filters also can contribute noise, but this is not necessarily a bad thing in controlled amounts, because it can act as a dither source.

For video applications, the phase response of such filters must be linear (see Chapter 2). Because a sharp cutoff is generally achieved by cascading many filter sections that cut at a similar frequency, the phase responses of these sections will accumulate. The phase may start to leave linearity at only a half of the passband frequency, and near the cutoff frequency the phase error may be severe. Effective group delay equalization is necessary.

It is possible to construct a ripple-free phase-linear filter with the required stop-band rejection, but it may be expensive due to the amount of design effort needed and the component complexity, and it might drift out of specification as components age. The money may be better spent in avoiding the need for such a filter. Much effort can be saved in analog filter design by using oversampling. Chapter 3 showed that digital filters are inherently phase-linear and, using LSIs, can be inexpensive to construct. The technical superiority of oversampling convertors along with economics means that they are increasingly used, which is why the subject is more prominent in this book than the treatment of filter design.

TWO-DIMENSIONAL SAMPLING SPECTRA

Analog video is sampled in the time domain and vertically, whereas a two-dimensional still image such as a photograph must be sampled horizontally and vertically. In both cases a two-dimensional spectrum will result, one vertical/temporal and one vertical/horizontal.

Figure 4.8a shows a square matrix of sampling sites that has an identical spatial sampling frequency both vertically and horizontally. The corresponding spectrum is shown in Figure 4.8b. The baseband spectrum is in the centre of the diagram, and the repeating sampling sideband spectrum extends vertically and horizontally. The star-shaped spectrum results from viewing an image of a man-made object, such as a building, containing primarily horizontal and vertical elements. A more natural scene such as foliage would result in a more circular or elliptical spectrum. To return to the baseband image, the sidebands must be filtered out with a two-dimensional spatial filter. The shape of the two-dimensional frequency response shown in Figure 4.8c is known as a Brillouin zone.

Figure 4.8d shows an alternative sampling-site matrix known as quincunx sampling because of the similarity to the pattern of five dots on a die. The resultant spectrum has the same characteristic pattern as shown in Figure 4.8e. The corresponding Brillouin zones are shown in (f). Quincunx sampling offers a better compromise between diagonal and horizontal/vertical resolution but is complex to implement.

FIGURE 4.8

Image sampling spectra. The rectangular array of (a) has a spectrum, shown in (b), having a rectangular repeating structure. Filtering to return to the baseband requires a two-dimensional filter whose response lies within the Brillouin zone shown in (c). Quincunx sampling is shown in (d) to have a similar spectral structure (e). An appropriate Brillouin zone is required as in (f).

It is highly desirable to prevent spatial aliasing, because the result is visually irritating. In tube cameras the spatial aliasing will be in the vertical dimension only, because the horizontal dimension is continuously scanned. Such cameras seldom attempt to prevent vertical aliasing. CCD sensors can, however, alias in both horizontal and vertical dimensions, and so an anti-aliasing optical filter is generally fitted between the lens and the sensor. This takes the form of a plate, which diffuses the image formed by the lens. Such a device can never have a sharp cutoff nor will the aperture be rectangular. The aperture of the anti-aliasing plate is in series with the aperture effect of the CCD elements, and the combination of the two effectively prevents spatial aliasing and generally gives a good balance between horizontal and vertical resolution, allowing the picture a natural appearance.

With a conventional approach, there are effectively two choices. If aliasing is permitted, the theoretical information rate of the system can be approached. If aliasing is prevented, realizable anti-aliasing filters cannot sharp cut, and the information conveyed is below system capacity.

These considerations also apply at the television display. The display must filter out spatial frequencies above one-half the sampling rate. In a conventional CRT this means that a vertical optical filter should be fitted in front of the screen to render the raster invisible. Again the aperture of a simply realizable filter would attenuate too much of the wanted spectrum, and so the technique is not used.

Figure 4.9 shows the spectrum of analog monochrome video (or of an analog component). The use of interlace has an effect on the vertical/temporal spectrum that is similar to the use of quincunx sampling on the vertical/horizontal spectrum. The concept of the Brillouin zone cannot really be applied to reconstruction in the spatial/temporal domains. This is partly due to there being two different units in which the sampling rates are measured and partly because the temporal sampling process cannot prevent aliasing in real systems.

Sampling conventional video along the line to create pixels makes the horizontal axis of the three-dimensional spectrum repeat at multiples of the sampling rate. Thus combining the three-dimensional spectrum of analog luminance shown in Figure 4.9 with the sampling spectrum of Figure 4.3b gives the final spectrum shown in Figure 4.10. Colour difference signals will have a similar structure but often use a lower sampling rate and thereby have less horizontal resolution or bandwidth.

FIGURE 4.9

The vertical/temporal spectrum of monochrome video due to interlace.

FIGURE 4.10

The spectrum of digital luminance is the baseband spectrum repeating around multiples of the sampling rate.

SAMPLING CLOCK JITTER

The instants at which samples are taken in an ADC and the instants at which DACs (digital-to-analog convertors) make conversions must be evenly spaced, otherwise unwanted signals can be added to the video. Figure 4.14 shows the effect of sampling clock jitter on a sloping waveform. Samples are taken at the wrong times. When these samples have passed through a system, the time base correction stage prior to the DAC will remove the jitter, and the result is shown in Figure 4.15. The magnitude of the unwanted signal is proportional to the slope of the audio waveform and so the amount of jitter that can be tolerated falls at 6dB per octave. As the resolution of the system is increased by the use of longer sample word length, tolerance to jitter is further reduced. The nature of the unwanted signal depends on the spectrum of the jitter. If the jitter is random, the effect is noise-like and relatively benign unless the amplitude is excessive. Figure 4.16 shows the effects of differing amounts of random jitter with respect to the noise floor of various word lengths. Note that even small amounts of jitter can degrade a 10-bit convertor to the performance of a good 8-bit unit. There is thus no point in upgrading to higher-resolution convertors if the clock stability of the system is insufficient to allow their performance to be realized.

FIGURE 4.14

The effect of sampling timing jitter on noise. A sloping signal sampled with jitter has error proportional to the slope.

FIGURE 4.15

The effect of sampling timing jitter on noise. When jitter is removed by reclocking, the result is noise.

FIGURE 4.16

The effects of sampling clock jitter on signal-to-noise ratio at various frequencies, compared with the theoretical noise floors with different word lengths.

Clock jitter is not necessarily random. Figure 4.17 shows that one source of clock jitter is cross talk or interference on the clock signal, although a balanced clock line will be more immune to such cross talk. The unwanted additional signal changes the time at which the sloping clock signal appears to cross the threshold voltage of the clock receiver. This is simply the same phenomenon as that of Figure 4.14 but in reverse. The threshold itself may be changed by ripple on the clock receiver power supply. There is no reason these effects should be random; they may be periodic and potentially visible.⁴

The allowable jitter is measured in picoseconds and clearly steps must be taken to eliminate it by design. Convertor clocks must be generated from clean power supplies that are well decoupled from the power used by the logic because a convertor clock must have a signal-to-noise ratio on the same order as that of the signal. Otherwise noise on the clock causes jitter, which in turn causes noise in the video. The same effect will be found in digital audio signals, which are perhaps more critical.

FIGURE 4.17

Cross talk in transmission can result in unwanted signals being added to the clock waveform. It can be seen here that a low-frequency interference signal affects the slicing of the clock and causes a periodic jitter.

QUANTIZING

Quantizing is the process of expressing some infinitely variable quantity by discrete or stepped values. Quantizing turns up in a remarkable number of everyday guises. Figure 4.18 shows that an inclined ramp enables infinitely variable height to be achieved, whereas a stepladder allows only discrete heights to be had. A stepladder quantizes height. When accountants round off sums of money to the nearest pound or dollar they are quantizing. Time passes continuously, but the display on a digital clock changes suddenly every minute because the clock is quantizing time.

In video and audio the values to be quantized are infinitely variable voltages from an analog source. Strict quantizing is a process that operates in the voltage domain only. For the purpose of studying the quantizing of a single sample, time is assumed to stand still. This is achieved in practice either by the use of a track-hold circuit or the adoption of a quantizer technology such as a flash convertor, which operates before the sampling stage.

Figure 4.20 shows that the process of quantizing divides the input voltage range up into quantizing intervals Q, also referred to as steps S. In applications such as telephony these may advantageously be of differing size, but for digital video the quantizing intervals are made as identical as possible. If this is done, the binary numbers that result are truly proportional to the original analog voltage, and the digital equivalents of mixing and gain changing can be performed by adding and multiplying sample values. If the quantizing intervals are unequal this cannot be done. When all quantizing intervals are the same, the term “uniform quantizing” is used. The term “linear quantizing” will be found, but this is, like military intelligence, a contradiction in terms.

FIGURE 4.18

An analog parameter is continuous, whereas a quantized parameter is restricted to certain values. Here the sloping side of a ramp can be used to obtain any height, whereas a ladder allows only discrete heights.

The term LSB (least significant bit) will also be found in place of quantizing interval in some treatments, but this is a poor term because quantizing works in the voltage domain. A bit is not a unit of voltage and can have only two values. In studying quantizing, voltages within a quantizing interval will be discussed, but there is no such thing as a fraction of a bit.

Whatever the exact voltage of the input signal, the quantizer will locate the quantizing interval in which it lies. In what may be considered a separate step, the quantizing interval is then allocated a code value, which is typically some form of binary number. The information sent is the number of the quantizing interval in which the input voltage lies. Whereabouts that voltage lies within the interval is not conveyed, and this mechanism puts a limit on the accuracy of the quantizer. When the number of the quantizing interval is converted back to the analog domain, it will result in a voltage at the centre of the quantizing interval, as this minimizes the magnitude of the error between input and output. The number range is limited by the word length of the binary numbers used. In an 8-bit system, 256 different quantizing intervals exist, although in digital video those at the extreme ends of the range are reserved for synchronising.

QUANTIZING ERROR

It is possible to draw a transfer function for such an ideal quantizer followed by an ideal DAC, and this is also shown in Figure 4.19. A transfer function is simply a graph of the output with respect to the input. In audio, when the term linearity is used, this generally means the overall straightness of the transfer function. Linearity is a goal in video and audio, yet it will be seen that an ideal quantizer is anything but linear.

FIGURE 4.19

Quantizing assigns discrete numbers to variable voltages. All voltages within the same quantizing interval are assigned the same number, which causes a DAC to produce the voltage at the centre of the intervals shown by the dashed lines.

Figure 4.20a shows that the transfer function is somewhat like a staircase, and the blanking level is halfway up a quantizing interval, or on the centre of a tread. This is the so-called mid-tread quantizer, which is universally used in video and audio. Figure 4.20b shows the alternative midriser transfer function, which causes difficulty because it does not have a code value at blanking level and as a result the numerical code value is not proportional to the analog signal voltage.

Quantizing causes a voltage error in the sample, which is given by the difference between the actual staircase transfer function and the ideal straight line. This is shown in Figure 4.20c to be a sawtooth-like function, which is periodic in Q. The amplitude cannot exceed ±½Q peak-to-peak unless the input is so large that clipping occurs.

Quantizing error can also be studied in the time domain where it is better to avoid complicating matters with the aperture effect of the DAC. For this reason it is assumed here that output samples are of negligible duration. Then impulses from the DAC can be compared with the original analog waveform and the difference will be impulses representing the quantizing error waveform.

FIGURE 4.20

Quantizing assigns discrete numbers to variable voltages. This is the characteristic of the mid-tread quantizer shown in (a). An alternative system is the midriser system shown in (b). Here zero Volts analog falls between two codes and there is no code for zero. Such quantizing cannot be used prior to signal processing because the number is no longer proportional to the voltage. Quantizing error cannot exceed ±½Q as shown in (c).

This has been done in Figure 4.21. The horizontal lines in the drawing are the boundaries between the quantizing intervals, and the curve is the input waveform. The vertical bars are the quantized samples, which reach to the centre of the quantizing interval. The quantizing error waveform shown in Figure 4.21b can be thought of as an unwanted signal, which the quantizing process adds to the perfect original. If a very small input signal remains within one quantizing interval, the quantizing error is the signal.

As the transfer function is nonlinear, ideal quantizing can cause distortion. As a result practical digital video equipment deliberately uses non-ideal quantizers to achieve linearity. The quantizing error of an ideal quantizer is a complex function, and it has been researched in great depth.^5–8 It is not intended to go into such depth here. The characteristics of an ideal quantizer will be pursued only far enough to convince the reader that such a device cannot be used in quality video or audio applications.

FIGURE 4.21

In (a) an arbitrary signal is represented to finite accuracy by PAM needles, whose peaks are at the centre of the quantizing intervals. The errors caused can be thought of as an unwanted signal (b) added to the original. In (c) the amplitude of a quantizing error needle will be from −½Q to +½Q with equal probability. Note, however, that white noise in analog circuits generally has Gaussian amplitude distribution, shown in (d).

As the magnitude of the quantizing error is limited, its effect can be minimized by making the signal larger. This will require more quantizing intervals and more bits to express them. The number of quantizing intervals multiplied by their size gives the quantizing range of the convertor. A signal outside the range will be clipped. Provided that clipping is avoided, the larger the signal, the less will be the effect of the quantizing error.

Where the input signal exercises the whole quantizing range and has a complex waveform (such as from a contrasty, detailed scene), successive samples will have widely varying numerical values and the quantizing error on a given sample will be independent of that on others. In this case the size of the quantizing error will be distributed with equal probability between the limits. Figure 4.21c shows the resultant uniform probability density. In this case the unwanted signal added by quantizing is an additive broadband noise uncorrelated with the signal, and it is appropriate in this case to call it quantizing noise. This is not quite the same as thermal noise, which has a Gaussian probability shown in Figure 4.21d (see Chapter 1, Transmission, for a treatment of statistics). The difference is of no consequence as in the large-signal case the noise is masked by the signal. Under these conditions, a meaningful signal-to-noise ratio (SNR) can be calculated as follows.

In a system using n-bit words, there will be 2ⁿ quantizing intervals. The largest sinusoid that can fit without clipping will have this peak-to-peak amplitude. The peak amplitude will be half as great, i.e., 2ⁿ⁻¹ 1Q, and the rms amplitude will be this value divided by √2. The quantizing error has an amplitude of ½Q peak, which is the equivalent of Q √12rms. The signal-to-noise ratio for the large signal case is then given by:

(4.1)

By way of example, an 8-bit system will offer very nearly 50dB SNR.

Whilst the above result is true for a large complex input waveform, treatments that then assume that quantizing error is always noise give results that are at variance with reality. The expression above is valid only if the probability density of the quantizing error is uniform. Unfortunately at low depths of modulations, and particularly with flat fields or simple pictures, this is not the case.

At low modulation depth, quantizing error ceases to be random and becomes a function of the input waveform and the quantizing structure as Figure 4.21 shows. Once an unwanted signal becomes a deterministic function of the wanted signal, it has to be classed as a distortion rather than a noise. Distortion can also be predicted from the nonlinearity, or staircase nature, of the transfer function. With a large signal, there are so many steps involved that we must stand well back, and a staircase with 256 steps appears to be a slope. With a small signal there are few steps and they can no longer be ignored.

The effect can be visualized readily by considering a television camera viewing a uniformly painted wall. The geometry of the lighting and the coverage of the lens mean that the brightness is not absolutely uniform, but falls slightly at the ends of the TV lines. After quantizing, the gently sloping waveform is replaced by one that stays at a constant quantizing level for many sampling periods and then suddenly jumps to the next quantizing level. The picture then consists of areas of constant brightness with steps between, resembling nothing more than a contour map, hence the use of the term contouring to describe the effect.

Needless to say, the occurrence of contouring precludes the use of an ideal quantizer for high-quality work. There is little point in studying the adverse effects further as they should be and can be eliminated completely in practical equipment by the use of dither. The importance of correctly dithering a quantizer cannot be emphasized enough, because failure to dither irrevocably distorts the converted signal: there can be no process that will subsequently remove that distortion. The signal-to-noise ratio derived above has no relevance to practical applications as it will be modified by the dither.

INTRODUCTION TO DITHER

At high signal levels, quantizing error is effectively noise. As the depth of modulation falls, the quantizing error of an ideal quantizer becomes more strongly correlated with the signal and the result is distortion, visible as contouring. If the quantizing error can be decorrelated from the input in some way, the system can remain linear but noisy. Dither performs the job of decorrelation by making the action of the quantizer unpredictable and gives the system a noise floor like an analog system.^9,10

In one approach, pseudo-random noise (see Chapter 3) with rectangular probability and a peak-to-peak amplitude of Q was added to the input signal prior to quantizing, but was subtracted after reconversion to analog. This is known as subtractive dither and was investigated by Schuchman¹¹ and much later by Sherwood.¹² Subtractive dither has the advantages that the dither amplitude is non-critical, the noise has full statistical independence from the signal⁷ and has the same level as the quantizing error in the large-signal undithered case.¹³ Unfortunately, it suffers from practical drawbacks, because the original noise waveform must accompany the samples or must be synchronously re-created at the DAC. This is virtually impossible in a system in which the signal may have been edited or where its level has been changed by processing, as the noise needs to remain synchronous and be processed in the same way. All practical digital video systems use non-subtractive dither, where the dither signal is added prior to quantization and no attempt is made to remove it at the DAC.¹⁴ The introduction of dither prior to a conventional quantizer inevitably causes a slight reduction in the signal-to-noise ratio attainable, but this reduction is a small price to pay for the elimination of nonlinearities.

The ideal (noiseless) quantizer of Figure 4.20 has fixed quantizing intervals and must always produce the same quantizing error from the same signal. In Figure 4.22 it can be seen that an ideal quantizer can be dithered by linearly adding a controlled level of noise, either to the input signal or to the reference voltage, which is used to derive the quantizing intervals. There are several ways of considering how dither works, all of which are equally valid.

FIGURE 4.22

Dither can be applied to a quantizer in one of two ways. In (a) the dither is linearly added to the analog input signal, whereas in (b) it is added to the reference voltages of the quantizer.

The addition of dither means that successive samples effectively find the quantizing intervals in different places on the voltage scale. The quantizing error becomes a function of the dither, rather than a predictable function of the input signal. The quantizing error is not eliminated, but the subjectively unacceptable distortion is converted into a broadband noise, which is more benign.

Some alternative ways of looking at dither are shown in Figure 4.23. Consider the situation in which a low-level input signal is changing slowly within a quantizing interval. Without dither, the same numerical code is output for a number of sample periods, and the variations within the interval are lost. Dither has the effect of forcing the quantizer to switch between two or more states. The higher the voltage of the input signal within a given interval, the more probable it becomes that the output code will take on the next higher value. The lower the input voltage within the interval, the more probable it is that the output code will take the next lower value. The dither has resulted in a form of duty cycle modulation, and the resolution of the system has been extended indefinitely instead of being limited by the size of the steps.

Dither can also be understood by considering what it does to the transfer function of the quantizer. This is normally a perfect staircase, but in the presence of dither it is smeared horizontally until with dither of a certain amplitude the average transfer function becomes straight.

REQUANTIZING AND DIGITAL DITHER

Recent ADC technology allows the resolution of video samples to be raised from 8 bits to 10 or even 12 bits. The situation then arises that an existing 8-bit device such as a digital VTR needs to be connected to the output of an ADC with greater word length. The words need to be shortened in some way.

It will be seen in Chapter 5 that when a sample value is attenuated, the extra low-order bits that come into existence below the radix point preserve the resolution of the signal and the dither in the least significant bit(s), which linearizes the system. The same word extension will occur in any process involving multiplication, such as digital filtering. It will subsequently be necessary to shorten the word length. Low-order bits must be removed to reduce the resolution whilst keeping the signal magnitude the same. Even if the original conversion was correctly dithered, the random element in the low-order bits will now be some way below the end of the intended word. If the word is simply truncated by discarding the unwanted low-order bits, or rounded to the nearest integer, the linearizing effect of the original dither will be lost.

FIGURE 4.23

Wideband dither of the appropriate level linearizes the transfer function to produce noise instead of distortion. This can be confirmed by spectral analysis. In the voltage domain, dither causes frequent switching between codes and preserves resolution in the duty cycle of the switching.

FIGURE 4.24

Shortening the word length of a sample reduces the number of codes that can describe the voltage of the waveform. This makes the quantizing steps bigger, hence the term “requantizing.” It can be seen that simple truncation or omission of the bits does not give analogous behaviour. Rounding is necessary to give the same result as if the larger steps had been used in the original conversion.

Shortening the word length of a sample reduces the number of quantizing intervals available without changing the signal amplitude. As Figure 4.24 shows, the quantizing intervals become larger and the original signal is requantized with the new interval structure. This will introduce requantizing distortion having the same characteristics as quantizing distortion in an ADC. It then is obvious that when shortening the word length of a 10-bit convertor to 8 bits, the 2 low-order bits must be removed in a way that displays the same overall quantizing structure as if the original convertor had been only of 8-bit word length. It will be seen from Figure 4.24 that truncation cannot be used because it does not meet the above requirement, but results in signal-dependent offsets because it always rounds in the same direction. Proper numerical rounding is essential in video applications because it accurately simulates analog quantizing to the new interval size. Unfortunately the 10-bit convertor will have a dither amplitude appropriate to quantizing intervals one-quarter the size of an 8-bit unit and the result will be highly nonlinear.

In practice, the word length of samples must be shortened in such a way that the requantizing error is converted to noise rather than distortion. One technique that meets this requirement is to use digital dithering¹⁵ prior to rounding. This is directly equivalent to the analog dithering in an ADC.

FIGURE 4.25

In a simple digital dithering system, two's complement values from a random-number generator are added to low-order bits of the input. The dithered values are then rounded up or down according to the value of the bits to be removed. The dither linearizes the requantizing.

Digital dither is a pseudo-random sequence of numbers. If it is required to simulate the analog dither signal of Figures 4.20 and 4.21, then it is obvious that the noise must be bipolar so that it can have an average voltage of zero. Two's complement coding must be used for the dither values.

Figure 4.25 shows a simple digital dithering system (i.e., one without noise shaping) for shortening sample word length. The output of a two's complement pseudo-random sequence generator (see Chapter 3) of appropriate word length is added to input samples prior to rounding. The most significant of the bits to be discarded is examined to determine whether the bits to be removed sum to more or less than half a quantizing interval. The dithered sample is either rounded down, i.e., the unwanted bits are simply discarded, or rounded up, i.e., the unwanted bits are discarded but 1 is added to the value of the new short word. The rounding process is no longer deterministic because of the added dither, which provides a linearizing random component.

If this process is compared with that of Figure 4.22 it will be seen that the principles of analog and digital dither are identical; the processes simply take place in different domains using two's complement numbers, which are rounded, or voltages, which are quantized, as appropriate. In fact quantization of an analog-dithered waveform is identical to the hypothetical case of rounding after bipolar digital dither in which the number of bits to be removed is infinite, and remains identical for practical purposes when as few as 8 bits are to be removed. Analog dither may actually be generated from bipolar digital dither (which is no more than random numbers with certain properties) using a DAC.

BASIC DIGITAL-TO-ANALOG CONVERSION

This direction of conversion will be discussed first, because ADCs often use embedded DACs in feedback loops. The purpose of a digital-to-analog convertor is to take numerical values and reproduce the continuous waveform that they represent. Figure 4.27 shows the major elements of a conventional conversion subsystem, i.e., one in which oversampling is not employed. The jitter in the clock needs to be removed with a VCO or VCXO. Sample values are buffered in a latch and fed to the convertor element, which operates on each cycle of the clean clock. The output is then a voltage proportional to the number for at least a part of the sample period. A resampling stage may be found next, to remove switching transients, reduce the aperture ratio, or allow the use of a convertor, which takes a substantial part of the sample period to operate. The resampled waveform is then presented to a reconstruction filter, which rejects frequencies above the video band.

This section is primarily concerned with the implementation of the convertor element. The most common way of achieving this conversion is to control binary-weighted currents and sum them in a virtual earth. Figure 4.28 shows the classical R–2R DAC structure. This is relatively simple to construct, but the resistors have to be extremely accurate. To see why this is so, consider the example of Figure 4.29. In (a) the binary code is about to have a major overflow, and all the low-order currents are flowing. In (b), the binary input has increased by one, and only the most significant current flows. This current must equal the sum of all the others plus one. The accuracy must be such that the step size is within the required limits. In this 8-bit example, if the step size needs to be a rather casual 10 percent accurate, the necessary accuracy is only one part in 2560, but for a 10-bit system it would become one part in 10,240. This degree of accuracy is difficult to achieve and maintain in the presence of ageing and temperature change.

FIGURE 4.27

The components of a conventional convertor. A jitter-free clock drives the voltage conversion, whose output may be resampled prior to reconstruction.

FIGURE 4.28

The classical R–2R DAC requires precise resistance values and “perfect” switches.

FIGURE 4.29

(a) Current flow with an input of 0111 is shown. (b) Current flow with an input code 1 greater.

BASIC ANALOG-TO-DIGITAL CONVERSION

The general principle of a quantizer is that different quantized voltages are compared with the unknown analog input until the closest quantized voltage is found. The code corresponding to this becomes the output. The comparisons can be made in turn with the minimal amount of hardware or simultaneously with more hardware.

The flash convertor is probably the simplest technique available for PCM video conversion. The principle is shown in Figure 4.30. The threshold voltage of every quantizing interval is provided by a resistor chain, which is fed by a reference voltage. This reference voltage can be varied to determine the sensitivity of the input. There is one voltage comparator connected to every reference voltage, and the other input of all the comparators is connected to the analog input. A comparator can be considered to be a one-bit ADC. The input voltage determines how many of the comparators will have a true output. As one comparator is necessary for each quantizing interval, then, for example, in an 8-bit system there will be 255 binary comparator outputs, and it is necessary to use a priority encoder to convert these to a binary code. Note that the quantizing stage is asynchronous; comparators change state as and when the variations in the input waveform result in a reference voltage being crossed. Sampling takes place when the comparator outputs are clocked into a subsequent latch. This is an example of quantizing before sampling, as was illustrated in Figure 4.1. Although the device is simple in principle, it contains a lot of circuitry and can be practicably implemented only on a chip. The analog signal has to drive many inputs, which results in a significant parallel capacitance, and a low-impedance driver is essential to avoid restricting the slewing rate of the input. The extreme speed of a flash convertor is a distinct advantage in oversampling. Because computation of all bits is performed simultaneously, no track/hold circuit is required, and droop is eliminated. Figure 4.30c shows a flash convertor chip. Note the resistor ladder and the comparators followed by the priority encoder. The MSB can be selectively inverted so that the device can be used either in offset binary or in two's complement mode.

The flash convertor is ubiquitous in digital video because of the high speed necessary. For audio purposes, many more conversion techniques are available and these are considered in Chapter 7.

FIGURE 4.30

The flash convertor. In (a) each quantizing interval has its own comparator, resulting in the waveforms of (b). A priority encoder is necessary to convert the comparator outputs to a binary code. Shown in (c) is a typical 8-bit flash convertor primarily intended for video applications (courtesy of TRW).

FACTORS AFFECTING CONVERTOR QUALITY

In theory the quality of a digital audio system comprising an ideal ADC followed by an ideal DAC is determined at the ADC. The ADC parameters such as the sampling rate, the word length, and any noise shaping used put limits on the quality that can be achieved. Conversely the DAC itself may be transparent, because it converts only data whose quality is already determined back to the analog domain. In other words, the ADC determines the system quality and the DAC does not make things any worse.

In practice both ADCs and DACs can fall short of the ideal, but with modern convertor components and attention to detail the theoretical limits can be approached very closely and at reasonable cost. Shortcomings may be the result of an inadequacy in an individual component, such as a convertor chip, or due to incorporating a high-quality component in a poorly thought-out system. Poor system design can destroy the performance of a convertor. Whilst oversampling is a powerful technique for realizing high-quality convertors, its use depends on digital interpolators and decimators, whose quality affects the overall conversion quality.

ADCs and DACs have the same transfer function; they are distinguished only by the direction of operation, and therefore the same terminology can be used to classify the possible shortcomings of both. Figure 4.38 shows the transfer functions resulting from the main types of convertor error:

(a) Offset error: A constant appears to have been added to the digital signal. This has a serious effect in video systems because it alters the black level. Offset error is sometimes cancelled by digitally sampling the convertor output during blanking and feeding it back to the analog input as a small control voltage.

(b) Gain error: The slope of the transfer function is incorrect. Because convertors are often referred to one end of the range, gain error causes an offset error. Severe gain error causes clipping.

(c) Integral linearity: This is the deviation of the dithered transfer function from a straight line. It has exactly the same significance and consequences as linearity in analog circuits, because if it is inadequate, harmonic distortion will be caused.

(d) Differential nonlinearity: This is the amount by which adjacent quantizing intervals differ in size. It is usually expressed as a fraction of a quantizing interval.

FIGURE 4.38

Main convertor errors (solid line) compared with perfect transfer function (dotted line). These graphs hold for ADCs and DACs, and the axes are interchangeable; if one is chosen to be analog, the other will be digital.

(e) Monotonicity: Monotonicity is a special case of differential nonlinearity. Non-monotonicity means that the output does not increase for an increase in input. Figure 4.29 showed how this can happen in a DAC. With a convertor input code of 01111111 (127 decimal), the seven low-order current sources of the convertor will be on. The next code is 10000000 (128 decimal), where only the eighth current source is operating. If the current it supplies is in error on the low side, the analog output for 128 may be less than that for 127. In an ADC non-monotonicity can result in missing codes. This means that certain binary combinations within the range cannot be generated by any analog voltage. If a device has better than 12Q linearity it must be monotonic. It is not possible for a 1-bit convertor to be non-monotonic.

(f) Absolute accuracy: This is the difference between actual and ideal output for a given input. For video and audio it is rather less important than linearity. For example, if all the current sources in a convertor have good thermal tracking, linearity will be maintained, even though the absolute accuracy drifts.

FIGURE 4.44

Additive mixing colour systems can reproduce colours only within a triangle in which the primaries lie on each vertex.

COLOUR IN THE DIGITAL DOMAIN

Colour cameras and most graphics computers produce three signals, or components, R, G, and B, which are essentially monochrome video signals representing an image in each primary colour. Figure 4.44 shows that the three primaries are spaced apart in the chromaticity diagram and the only colours that can be generated fall within the resultant triangle.

RGB signals are strictly compatible only if the colour primaries assumed in the source are present in the display. If there is a primary difference the reproduced colours are different. Clearly broadcast television must have a standard set of primaries. The EBU television systems have only ever had one set of primaries. NTSC started off with one set and then adopted another because the phosphors were brighter. Computer displays have any number of standards because initially all computer colour was false, i.e., synthetic, and the concept of accurate reproduction did not arise. Now that computer displays are going to be used for television it will be necessary for them to adopt standard phosphors or to use colorimetric transcoding on the signals.

Fortunately the human visual system is quite accommodating. The colour of daylight changes throughout the day, so everything changes colour with it. Humans, however, accommodate to that. We tend to see the colour we expect rather than the actual colour. The colour reproduction of printing and photographic media is pretty appalling, but so is human colour memory, and so it is acceptable.

On the other hand, our ability to discriminate between colours presented simultaneously is remarkably good, hence the difficulty car repairers have in getting paint to match.

RGB and Y signals are incompatible, yet when colour television was introduced it was a practical necessity that it should be possible to display colour signals on a monochrome display and vice versa.

Creating or transcoding a luminance signal from RGB is relatively easy. The spectral response of the eye has a peak in the green region. Green objects will produce a larger stimulus than red objects of the same brightness, with blue objects producing the least stimulus. A luminance signal can be obtained by adding R, G, and B together, not in equal amounts, but in a sum that is weighted by the relative response of the human visual system. Thus:

Y = 0.299R + 0.587G + 0.114B.

Note that the factors add up to 1. If Y is derived in this way, a monochrome display will show nearly the same result as if a monochrome camera had been used in the first place. The results are not identical because of the nonlinearities introduced by gamma correction and by imperfect colour filters.

As colour pictures require three signals, it should be possible to send Y and two other signals, which a colour display could arithmetically convert back to RGB. There are two important factors that restrict the form that the other two signals may take. One is to achieve reverse compatibility. If the source is a monochrome camera, it can produce only Y and the other two signals will be completely absent. A colour display should be able to operate on the Y signal only and show a monochrome picture. The other is the requirement to conserve bandwidth for economic reasons.

These requirements are met in the analog domain by creating two colour difference signals, R–Y and B–Y. In the digital domain the equivalents are C_R and C_B.

Whilst signals such as Y, R, G, and B are unipolar or positive only, colour difference signals are bipolar and may meaningfully take on negative values. Figure 4.45a shows the colour space available in 8-bit RGB. In computers, 8-bit RGB is common and we often see claims that 16 million different colours are possible. This is utter nonsense.

FIGURE 4.45

RGB transformed to colour difference space. This is done because R − Y (C_R) and B − Y (C_B) can be sent with reduced bandwidth. (a) RGB cube. The white/black axis is diagonal and all locations within the cube are legal. (b) RGB to colour difference transform. (c) RGB cube mapped into colour difference space is no longer a cube. Only combinations of Y, C_R, and C_B that fall within the three-dimensional space shown are legal. Projection of the space downward creates the familiar vectorscope display.

A colour is a given combination of hue and saturation and is independent of brightness. Consequently all sets of RGB values having the same ratios produce the same colour. For example, R = G = B always gives the same colour whether the pixel value is 0 or 255. Thus there are 256 brightnesses that have the same colour, allowing a more believable 65,000 different colours.

Figure 4.45(c) shows the RGB cube mapped into 8-bit colour difference space so that it is no longer a cube. Now the grey axis goes straight up the middle because greys correspond to both C_R and C_B being zero. To visualize colour difference space, imagine looking down along the grey axis. This makes the black and white corners coincide in the centre. The remaining six corners of the legal colour difference space now correspond to the six boxes on a component vectorscope. Although there are still 16 million combinations, many of these are now illegal. For example, as black or white is approached, the colour differences must fall to zero.

From an information theory standpoint, colour difference space is redundant. With some tedious geometry, it can be shown that fewer than a quarter of the codes are legal. The luminance resolution remains the same, but there is about half as much information in each colour axis. This is because the colour difference signals are bipolar. If the signal resolution has to be maintained, 8-bit RGB should be transformed to a longer word length in the colour difference domain, 9 bits being adequate. At this stage the colour difference transform doesn't seem efficient because 24-bit RGB converts to 26-bit Y, C_R, C_B.

In most cases the loss of colour resolution is invisible to the eye, and 8-bit resolution is retained. The results of the transform computation must be digitally dithered to avoid posterizing.

The inverse transform to obtain RGB again at the display is straightforward. R and B are readily obtained by adding Y to the two colour difference signals. G is obtained by rearranging the expression for Y above such that

If a monochrome source having only a Y output is supplied to a colour display, CR and CB will be zero. It is reasonably obvious that if there are no colour difference signals the colour signals cannot be different from one another and R = G = B. As a result the colour display can produce only a neutral picture.

FIGURE 4.46

Ideal two-dimensional down-sampled colour difference system. Colour resolution is half of luma resolution, but the eye cannot tell the difference.

The use of colour difference signals is essential for compatibility in both directions between colour and monochrome, but it has a further advantage, which follows from the way in which the eye works. To produce the highest resolution in the fovea, the eye will use signals from all types of cone, regardless of colour. To determine colour the stimuli from three cones must be compared.

There is evidence that the nervous system uses some form of colour difference processing to make this possible. As a result the full acuity of the human eye is available only in monochrome. Detail in colour changes cannot be resolved so well. A further factor is that the lens in the human eye is not achromatic and this means that the ends of the spectrum are not well focused. This is particularly noticeable on blue.

In this case there is no point is expending valuable bandwidth sending high-resolution colour signals. Colour difference working allows the luminance to be sent separately at full bandwidth. This determines the subjective sharpness of the picture. The colour difference information can be sent with considerably reduced resolution, as little as one-quarter that of luma, and the human eye is unable to tell.

The acuity of human vision is axisymmetric. In other words, detail can be resolved equally at all angles. When the human visual system assesses the sharpness of a TV picture, it will measure the quality of the worst axis and the extra information on the better axis is wasted. Consequently the most efficient row-and-column image sampling arrangement is the so-called “square pixel.” Now pixels are dimensionless and so this is meaningless. However, it is understood to mean that the horizontal and vertical spacing between pixels is the same. Thus it is the sampling grid that is square, rather than the pixel.

The square pixel is optimal for luminance and also for colour difference signals. Figure 4.46a shows the ideal. The colour sampling is co-sited with the luminance sampling but the colour sample spacing is twice that of luminance. The colour difference signals after matrixing from RGB have to be low-pass filtered in two dimensions prior to down sampling to prevent aliasing of HF detail. At the display, the down-sampled colour data have to be interpolated in two dimensions to produce colour information in every pixel. In an over-sampling display the colour interpolation can be combined with the display up-sampling stage.

Co-siting the colour and luminance pixels means that the transmitted colour values are displayed unchanged. Only the interpolated values need to be calculated. This minimizes generation loss in the filtering. Down sampling the colour by a factor of 2 in both axes means that the colour data are reduced to one-quarter of their original amount. When viewed by a human this is essentially a lossless process.

References

1. Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J.,27, 379 (1948).

2. Jerri, A.J. The Shannon sampling theorem—its various extensions and applications: a tutorial review. Proc. IEEE, 65, 1565–1596 (1977).

3. Betts, J.A. Signal Processing Modulation and Noise, Chap. 6, Sevenoaks: Stoughton (1970).

4. Harris, S. The effects of sampling clock jitter on Nyquist sampling analog to digital convertors and on oversampling delta-sigma ADCs. J. Audio Eng. Soc., 38, 537–542 (1990).

5. Bennett, W.R. Spectra of quantized signals. Bell Syst. Tech. J., 27, 446–472

6. Widrow, B. Statistical analysis of amplitude quantized sampled-data systems. Trans. AIEE, Part II, 79, 555–568 (1961).

7. Lipshitz, S.P., Wannamaker, R.A., and Vanderkooy, J. Quantization and dither: a theoretical survey. J. Audio Eng. Soc., 40, 355–375 (1992).

8. Maher, R.C. On the nature of granulation noise in uniform quantization systems. J. Audio Eng. Soc., 40, 12–20 (1992).

9. Goodall, W.M. Television by pulse code modulation. Bell Syst. Tech. J., 30, 33–49 (1951).

10. Roberts, L.G. Picture coding using pseudo-random noise. IRE Trans. Inform. Theory, IT-8, 145–154 (1962).

11. Schuchman, L. Dither signals and their effect on quantizing noise. Trans. Commun. Technol., COM-12, 162–165 (1964).

12. Sherwood, Some theorems on quantization and an example using dither. In Conf. Rec., 19th Asilomar Conference on Circuits, Systems, and Computers (Pacific Grove, CA) (1985).

13. Gerzon, M., and Craven, P.G. Optimal noise shaping and dither of digital signals. Presented at the 87th Audio Engineers Society Convention (New York) Preprint No. 2822 (J-1) (1989).

14. Vanderkooy, J., and Lipshitz, S.P. Resolution below the least significant bit in digital systems with dither. J. Audio Eng. Soc., 32, 106–113 (1984).

15. Vanderkooy, J., and Lipshitz, S.P., Digital dither. Presented at the 81st Audio Engineers Society Convention (Los Angeles) Preprint 2412 (C-8) (1986).

16. Vanderkooy, J., and Lipshitz, S.P. Digital dither. In Audio in Digital Times, New York: AES (1989).

17. Hauser, M.W., Principles of oversampling A/D conversion. J. Audio Eng. Soc., 39, 326 (1991).

18. CCIR Recommendation 601–1, Encoding parameters for digital television for studios.

19. EBU Doc. Tech. 3246.

20. SMPTE 125M, Television bit parallel digital interface component video signal 4:2:2.

21. SMPTE 274M Proposed standard—1920 × 1080 image sample structure digital representation and digital timing reference sequences for multiple picture rates.

22. SMPTE 296M 1280 × 720 progressive image sample structure—analog and digital representation and analog interface (2001).