15.3.1 Encoder

The microphone array configuration utilized in ImmACS is depicted in Figure 15.2; Figure 15.3 depicts a block diagram of the ImmACS encoder. The time-domain signals x₁(t), x₂(t), …, x_M(t) are transformed into the short-time Fourier transform (STFT) domain signals X₁(τ, ω ), X₂(τ, ω), …, X_M(τ, ω), where τ denotes the time frame index and ω denotes the radial frequency. The STFT domain signals are fed to the DOA estimation processing block, which returns the estimated number of active sources, , and the vector of DOA estimates. In parallel, the STFT signals are provided to the beamforming and post-filtering block which, given the number and DOAs of the sources, produces separated signals . The separated signals are efficiently downmixed into a single audio channel, e(t), and side information I(τ, ω).

Counting and DOA Estimation of Multiple Audio Sources

For the counting and estimation of the DOAs, ImmACS adopts a relaxed sparsity assumption on the time–frequency (TF) domain of the source signals (Puigt and Deville, 2007; Pavlidi et al., 2012). It is assumed that for each source at least one constant-time analysis zone can be found where the source is isolated, that is, it is dominant over the others. The constant-time analysis zone is defined as a series of Ω frequency-adjacent TF points at time frame index τ, denoted by (τ, Ω), and it is characterized as a single-source zone (SSZ) if it is dominated by one audio source. Such a sparsity assumption allows the signals to overlap in the time–frequency domain. It is therefore a much weaker constraint compared to the W-disjoint orthogonality assumption (Rickard and Yilmaz, 2002), where every TF point is considered to be dominated by one source. In ImmACS, DOA estimation of multiple sources relies on the detection of all SSZs and the derivation of local DOA estimates, deploying a single-source localization algorithm.

To detect an SSZ, the cross- and auto-correlation of the magnitude of the TF transform need to be estimated (in what follows, the time frame index, τ, is dropped for clarity):

(15.1)

The correlation coefficient between a pair of microphone signals (x_i, x_j) is then defined as

(15.2)

A constant-time analysis zone is an SSZ if and only if (Puigt and Deville, 2007)

(15.3)

In practice, every zone that satisfies the inequality

(15.4)

is characterized as an SSZ, where is the average correlation coefficient between pairs of observations of adjacent microphones and ε is a small user-defined threshold.

The SSZ detection stage is followed by the DOA estimation at each SSZ. In general, any suitable single-source DOA estimation algorithm could be applied. In ImmACS, a modified version of the algorithm in Karbasi and Sugiyama (2007) is deployed, mainly because this algorithm is designed specifically for circular apertures, is computationally efficient, and has shown good performance under noisy conditions (Karbasi and Sugiyama, 2007; Pavlidi et al., 2013).

The phase of the cross-power spectrum of a microphone pair is first evaluated over the frequency range of an SSZ as

(15.5)

where the cross-power spectrum is

(15.6)

and * stands for complex conjugate.

The evaluated cross-power spectrum phase is used to estimate the phase rotation factors (Karbasi and Sugiyama, 2007),

(15.7)

where τ_{i → 1}(ϕ)≜τ_{1, 2}(ϕ) − τ_{i, i + 1}(ϕ) is the difference in the relative delay between the signals received at pairs {1, 2} and {i, i + 1}. The time difference of arrival for the microphone pair {i, i + 1} is evaluated according to

(15.8)

where ϕ ∈ [0, 2π), ω ∈ Ω, α and l are the angle and distance between the adjacent microphone pair {i, i + 1} respectively, A is the obtuse angle formed by the chord defined by the microphone pair {1, 2} and the x-axis, and c is the speed of sound (see also Figure 15.2). Since the microphone array is uniform, α, A, and l are given by

(15.9)

where q is the array radius.

Next comes the estimation of the circular integrated cross spectrum (CICS), defined by Karbasi and Sugiyama (2007) as

(15.10)

The DOA associated with the frequency component ω in the SSZ with frequency range Ω is estimated as

(15.11)

In order to enhance the overall DOA estimation, the aforementioned algorithm is applied on “strong” frequency components, that is, those frequencies that correspond to the indices of the d highest peaks in the magnitude of the cross-power spectrum over all microphone pairs in an SSZ. Figure 15.4 illustrates the DOA estimation error versus signal to noise ratio (SNR) for various choices of d. It is clear that using more frequency bins leads in general to a lower estimation error. However, increasing d increases the computational complexity, which has to be taken into consideration for a real-time system. In ImmACS, a standard choice for d is d = 2.

The next step in counting and estimating the final DOAs of active audio sources constitutes the processing of estimated DOAs in SSZs in a block-based manner. To do so, a histogram is formed from the set of estimations in a block of B consecutive time frames, which slides one frame each time. The histogram is smoothed by applying an averaging filter with a window of length h_N, providing

(15.12)

where y(v) is the cardinality of the smoothed histogram at each histogram bin v, V is the number of bins in the histogram, ζ_i is the ith estimate out of N estimates in a block, and w( · ) is the rectangular window of length h_N.

The smoothed histogram is processed following the principles of matching pursuit, that is, it is correlated with a source atom in order to detect the DOA of a possible source, the contribution of which is estimated and removed from the histogram. The process is repeated iteratively until the contribution of a potential source fails to satisfy a user-defined threshold.

A narrower-width pulse is used to accurately locate a peak in the histogram, the index of which denotes the DOA of a source. A wider-width pulse is used to account for the contribution of the source to the overall histogram and to provide better performance at lower SNRs. Each source atom is modeled as a smooth pulse utilizing a Blackman window. The correlation of the histogram with the source atom is performed in a circular manner as the histogram “wraps” from 359° to 0°. Thus, a matrix has to be formed whose rows contain wrapped and shifted versions of the source pulse. We denote by C_N and C_W the matrices for the peak detection (denoted by “N” for narrow) and the contribution estimation (denoted by “W” for wide), respectively. The rows of matrix C_N contain shifted versions of the narrow source atom with width Q_N, and, correspondingly, the rows of matrix C_W contain the shifted wide source atom with width equal to Q_W. This dual-width approach is illustrated in Figure 15.5.

In more detail, at each iteration denoted with loop index j, u = C_Ny_j is formed, where y_j is the smoothed histogram at the current iteration. u denotes the correlation of the histogram with the narrow pulse, and thus the index of its maximum value denotes the DOA estimated at the current iteration j, that is, , where u_i are the elements of u. The contribution of the source located at iteration j is estimated as

(15.13)

Observe that the contribution is actually the row of the matrix C_W that contains a circularly shifted version of the source atom by (i* − 1) elements, that is, c^{(i* − 1)}_W, weighted by . In order to decide if the detected DOA and its contribution correspond to a true source and ultimately estimate the number of active sources, , we create , a length-P_MAX vector whose elements are some predetermined thresholds, representing the relative energy of the jth source. Thus, if then a source is detected and its contribution is removed from the histogram:

(15.14)

The algorithm then proceeds with the next iteration. In the opposite case, the algorithm ceases and the estimated DOAs and their corresponding number are returned.

Beamforming and Downmixing

Based on the estimated number of sources and their corresponding DOAs, ImmACS utilizes a spatial filter that consists of a superdirective beamformer and a post-filter in order to separate the sources’ signals. The superdirective beamformer is designed to maximize the array gain, keeping the signal from the steering direction undistorted, while maintaining a minimum constraint on the white noise gain; the beamformer’s filter coefficients are given by (Cox et al., 1987)

(15.15)

where w(ω, θ) is the M × 1 vector of complex filter coefficients, θ is the beamformer’s steering direction, d(ω, θ) is the steering vector of the array, is the M × M noise coherence matrix, ( · )^H is the Hermitian transpose operation, I is the identity matrix, and ε is used to control the white noise gain constraint. Assuming a spherically isotropic diffuse noise field, the elements for the noise coherence matrix are calculated as

(15.16)

where ω denotes the radial frequency, c is the speed of sound, and d_ij is the distance between the ith and jth microphones. The beamformer is signal independent, facilitating its use in the real-time implementation of ImmACS, as the filter coefficients are calculated offline and stored in the system’s memory.

Hence, for each source s, the microphone signals at frequency ω are filtered with the beamformer filter coefficients w(ω, θ_s) that correspond to the source’s estimated direction θ_s. This procedure results in the beamformer output signals:

(15.17)

A post-filtering operation is then applied to the beamformer output signals; each beamformer output signal is filtered with a binary mask. The binary masks are constructed by comparing the energies of the beamformer output signals for each source at a given frequency. The binary mask for the sth source is given by (Maganti et al., 2007)

(15.18)

The final separated source signals are given by:

(15.19)

From Equation (15.18), it is obvious that the binary masks are orthogonal with respect to each other. This means that if U_s(τ, ω) = 1 for some frequency ω and frame index τ, then for s′ ≠ s, a property which also holds for the final separated source signals. This means that for each frequency only one source is maintained – the one with the highest energy – while the other sources are set to zero. The effect of the post-filtering operation is illustrated with an example of three active sound sources in Figure 15.6: for each frequency, only the source with the highest energy is kept, for example the first source for frequency ω₁, the third source for frequency ω₂, and so on. Looking at the frequency domain representation of the resulting post-filtered signals in Figure 15.6(b), the orthogonality property of the binary masks is evident.

The orthogonality property of the binary masks results in a very efficient downmixing scheme: the sources’ signals can be downmixed by summing them in the frequency domain to create a full-spectrum monophonic audio signal:

(15.20)

The downmix signal is then transformed back to the time-domain signal e(t). To extract the estimated source signals from the downmix, additional information must also be attained, describing to which source each frequency of the downmix belongs. For this reason, ImmACS creates a side-information channel I(τ, ω), which attains the assignment of frequencies to the DOAs of the sources, which also specifies the direction from which each frequency of the downmix signal will be reproduced in the decoder. Hence, the entire soundscape is encoded with the use of one monophonic audio signal and side information. These two streams are transmitted to the decoder in order to recreate and reproduce the soundscape.

ImmACS also offers the potential to include a diffuse part in the encoded soundscape. This is done by setting a cut-off frequency ω_c, which defines the frequencies up to which directional information is extracted: for the frequencies below ω_c, the aforementioned procedure of spatial filtering is applied in order to separate the active sources’ signals. In contrast, the frequencies above ω_c are assumed to be dominated by diffuse sound. For these frequencies the spectrum from an arbitrary microphone of the array is included in the downmix (the dashed line in Figure 15.3), without further processing and without the need to keep any side information for this part, as it represents the part of the soundscape with no prominent direction. Thus, the side-information channel only attains the directional information for the frequencies up to ω_c. Processing only a certain range of frequencies may have several advantages, such as reduction in the computational complexity – especially when the sampling frequency is high – and reduction in the bitrate requirements for the side-information channel. But, more importantly, treating the higher frequencies as diffuse sound offers a sense of immersion to the listeners. Finally, applications like teleconferencing, where the signal content is mostly speech, can tolerate setting ω_c to approximately 4 kHz without degradation in the spatial impression of the recreated soundscape.

To reduce the bitrate requirements, coding schemes can be applied to both the audio and the side-information channels. Since the audio channel consists of a monophonic stream, any monophonic audio encoder can be utilized, such as MP3 (ISO, 1992), or the ultra-low-delay OPUS coder (Valin et al., 2013; Vos et al., 2013) that is used in ImmACS. Encoding the downmix signal with any of the aforementioned codecs has been found not to affect the spatial impression of the recreated soundscape (Alexandridis et al., 2013b).

To encode the side-information channel, lossless coding schemes need to be used in order to attain the assignment of the frequencies to the sources. The side-information channel depends on Equation (15.18), and thus it is sufficient to encode the binary masks. The encoding scheme that ImmACS employs is based on the orthogonality property of the masks, as explained below.

The active sources at a given time frame are sorted in descending order according to the number of frequency bins assigned to them. The binary mask of the first (the most dominant) source is inserted into the bitstream. Given the orthogonality property of the binary masks, it follows that we do not need to encode the mask for the sth source at the frequency bins where at least one of the previous s − 1 masks is equal to one (since the rest of the masks will be zero). These locations can be identified by a simple OR operation between the s − 1 previous masks. Thus, for the second up to the th mask, only the locations where the previous masks are all zero are inserted into the bitstream. The mask of the last source does not need to be encoded, as it contains ones in the frequency bins in which all the previous masks had zeros. A dictionary that associates the sources with their DOAs is also included in the bitstream. The encoding algorithm is presented in Algorithm 15.1 and is illustrated with an example in Figure 15.7. In Algorithm 15.1, find_positions_with_zeros(·) is a function that returns the indices of zero values of the vector taken as argument, and | denotes the concatenation operator. The resulted bitstream is further compressed with Golomb entropy coding (Golomb, 1966) applied on the run lengths of ones and zeros.

15.3.2 Decoder

The decoder recreates the soundscape for the remote participants according to the block diagram depicted in Figure 15.8. Based on the available equipment, ImmACS supports multichannel reproduction using multiple loudspeakers, Figure 15.8(a), or binaural reproduction using headphones, Figure 15.8(b).

In the case where the downmix signal or the side-information channel have been encoded, the respective decoder is first applied. For decoding the side information, the mask of the first source is retrieved first. For the mask of the sth source, the next n bits are read from the bitstream, where n is the number of frequencies for which all the previous s − 1 masks are zero. This can be identified by a simple NOR operation. In terms of complexity, this scheme is computationally efficient, as its main operations are simple OR and NOR operations.

Then, for every incoming frame, the downmix signal e(t) is transformed to the frequency domain and the spectrum is divided into the non-diffuse part and diffuse part, based on the beamformer cutoff frequency ω_c: the frequencies ω ≤ ω_c belong to the non-diffuse part, while the diffuse part—if it exists—includes the frequencies ω > ω_c.

For loudspeaker reproduction, the non-diffuse part is synthesized using vector base amplitude panning (VBAP; Pulkki, 1997) at each frequency, where the panning gains for each loudspeaker are determined according to the DOA at that frequency from the side-information channel I(τ, ω). If a diffuse part is included, it is played back from all loudspeakers (dashed line in Figure 15.8(a)). Assuming a loudspeaker configuration with L loudspeakers, the signal for the lth loudspeaker is thus given by

(15.21)

where g_l(I(τ, ω)) refers to the VBAP gain for the lth loudspeaker in order to play back the time–frequency point (τ, ω) from the direction specified by I(τ, ω).

For binaural reproduction using headphones, ImmACS utilizes an HRTF database in order to position each source at its corresponding direction. Hence, each frequency of the non-diffuse part is filtered with the HRTF of the direction specified by the side information. The optional diffuse part is included in both channels – the dashed line in Figure 15.8(b). Summarizing, the two-channel output is given by

(15.22)

where denotes the left and right channels.

15.4.1 Sound Source Positioning Based on VBAP

Sound source positioning using VBAP (Pulkki, 1997) is traditionally linked to virtual sound source placement and to parametric spatial audio applications, where a DOA θ with respect to some portion of the signal is always defined or estimated. VBAP requires that the loudspeakers are distributed around a circle of fixed radius around the listening area, but their number and direction might be arbitrary, a fact that provides important flexibility in designing the loudspeaker arrangement. Given the incident angle θ, this is mapped to the L loudspeaker gains in accordance with the physical loudspeaker distribution around the listening area. In Figure 15.9 we show an example of how such directivity patterns look for the cases of L = 4 and 8 uniformly distributed loudspeakers in Figure 15.9(a) and Figure 15.9(c), and for L = 5 non-uniform loudspeakers in Figure 15.9(b), assuming that the problem is confined to 2D.

It is worth spending a few words here in order to explain how the aforementioned mapping between incident angle and loudspeaker gain is also useful in the case of the non-parametric approach which we intend to consider. Hacihabiboğlu and Cvetković (2009) explain how the physical loudspeaker angular distribution dictates the optimum directivity patterns for an equal (to the number of loudspeakers) number of directional microphones to capture the directional properties of the sound field. Given such microphones, one could then send the signal captured by each microphone to the corresponding loudspeaker, without further processing, to reproduce a convincing panoramic auditory reality. Respecting the requirements stated in that paper, VBAP dictates optimal loudspeaker panning gains with the following characteristics:

• the panning gains are frequency-invariant;
• the problem of unwanted interchannel crosstalk is efficiently addressed by ensuring that given a single plane wave incident at a certain angle (in the azimuth plane), only two loudspeakers will be activated during reproduction (the two that are adjacent to the estimated angle);
• the sum of the squares of all loudspeaker gains along θ is equal, meaning that there is no information loss.

This implies that we can use the loudspeaker gains provided by VBAP as the desired directivity response for a set of beamformers, which we can then use in order to capture the acoustic environment in different directions. The signal at the output of each beamformer can then be sent directly to the corresponding loudspeaker, a process that is mathematically represented in Equation (15.23). As long as the desired beam patterns are accurately reproduced, the non-parametric approach represents the simplest way to capture and reproduce an acoustic scene, promising accurate spatialization of both the directional and diffuse components of the captured sound field. As will be seen, however, the accurate reconstruction of these patterns using beamforming is not an easy task, and deviations between the desired and actually reproduced directivity patterns are expected to degrade the sense of direction transmitted to the listener.

15.4.2 Non-Parametric Approach

In the time–frequency domain, the process of beamforming can be expressed as

(15.23)

where τ is the time-frame index, ω is the radial frequency, w_l(ω) = [w_1l(ω), …, w_Ml(ω)]^T is the vector with the M complex beamformer weights responsible for the lth loudspeaker channel, X(τ, ω) is the M × 1 vector with the signal at the microphones, and Y_l(τ, ω) represents the lth loudspeaker signal in the time–frequency domain. This process is repeated for all the different loudspeaker channels l = 1, …, L; the signals are transformed to the time domain and sent to the corresponding loudspeakers for playback.

Using the panning gains dictated by VBAP, we demonstrate in what follows an approach for calculating the beamformer weights associated with each loudspeaker channel. Consider a grid of N uniformly distributed directions θ_n in [ − 180°, 180°) with the loudspeaker panning gains g_l(θ_n) prescribed by VBAP. The problem becomes finding the weights w_l(ω) in order to satisfy

(15.24)

Here, G_l = [g_l(θ₁), …, g_l(θ_N)]^T is the desired response provided by VBAP and D(ω) = [d(ω, θ₁), …, d(ω, θ_N)] is the matrix with the array steering vectors which model the array response to a plane wave incident at angle θ_n. Without loss of generality we may consider here that ∀l as the VBAP function does not dictate phase variations across the different channels. Also, for the case of a circular array of radius R the propagation model can be written as (Alexandridis et al., 2013a)

(15.25)

where ϕ_m denotes the angle of the mth sensor with respect to the sensor array center and k is the wavenumber. Assuming that L < N, the linear problem of Equation (15.24) is overdetermined and the solution can be found by minimizing, in the least squares (LSQ) sense, the cost function

(15.26)

However, unconstrained minimization involves inversion of the matrix D(ω)D(ω)^H, which is ill-conditioned at low frequencies as well as at other distinct frequencies. An example of this behavior is shown in Figure 15.10, where we have plotted the condition number of D(ω)D(ω)^H as a function of frequency, considering a circular array of M = 8 uniformly distributed microphones and the reproduction system of Figure 15.9(a). Direct inversion of D(ω)D(ω)^H might thus lead to severe amplification of noise at certain frequencies, which is perceived as unwanted spectral coloration. In order to avoid such a problem, we propose to use Tikhonov regularization by adding a penalty term proportional to the noise response in the previous cost function as

(15.27)

with μ(ω) implying that the value of the regularization parameter varies with frequency. We have observed that this approach achieves a better trade-off between the noise gain and the array gain, as opposed to a constant value of the regularization parameter. In this chapter, we propose a varying value of the regularization parameter of the form

(15.28)

where λ is a fixed scalar and cond(·) represents the condition number of a matrix, that is, the ratio of its largest eigenvalue to its smallest one. The beamformer weights can then be found through LSQ minimization as

(15.29)

where I is the M × M identity matrix.

Finally, we consider an additional normalization step, which aims to ensure unit gain and zero phase shift at the direction of maximum response for each beamformer. Letting θ⁰_l denote this direction for the lth beam at frequency ω, the final weights are calculated as

(15.30)

and the signal for the lth loudspeaker is obtained as in Equation (15.23). The beamformer weights need only be calculated once for each frequency and then stored to be used in the application phase.

In Figure 15.11 we present plots of the actual directivity versus the desired directivity pattern, for the same reproduction system as considered in Figure 15.9. Observe the increment in the amplitude of the side-lobes at 2660 Hz, which is close to a problematic frequency according to Figure 15.10. Also, the subplot corresponding to 7 kHz (in Figure 15.11) is indicative of the spatial aliasing problems that occur at higher frequencies.

15.4.3 Parametric Approach

As shown in Figure 15.11, it is difficult to obtain exactly the desired directivity patterns relying on simple beamforming. Looking, for example, at the sub-figure corresponding to 2660 Hz, we see that a sound source at −135° would also be played back by the loudspeaker at 45°, something that may blur the sense of direction transmitted to the listener. On the other hand, a parametric approach avoids this problem by defining the loudspeaker response as a function of the estimated DOA.

However, it is questionable what type of processing is applicable to the particular type of array that we typically focus on in our work. While the method works sufficiently well for an application with a limited number of speakers, such as the one described in Section 15.3, it is inappropriate for the considered acoustic conditions due to the enormous number of potential sound sources that participate in the sound scene. On the other hand, DirAC does not impose a limitation on the number of sound sources comprising the sound scene, but it is typically implemented with smaller arrays – in terms of both radius and number of sensors – than the one considered in this application (Kuech et al., 2008).

The parametric approach that we present in this section is based on the method introduced by Ito et al. (2010). We exploit this previous study here in order to perform direct-to-diffuse sound field decomposition, at the same time extending it for the purpose of DOA estimation. Assume that a sensor array comprised of M sensors is embedded inside a diffuse noise field and at the same time receives the signal from a single directional source. The observation model for sensor m can be written

(15.31)

where s(τ, ω) is the directional source signal, is the transfer function from the source to the sensor at frequency ω, and U_m(τ, ω) is the diffuse noise component, which is assumed to be uncorrelated with the source signal. The observation interchannel cross-spectrum between sensors m and n can thus be written

(15.32)

(15.33)

where ϕ_ss is the source power spectrum and represents the component of the cross-spectrum due to noise. Assuming that the noise field is isotropic, then , as explained in Ito et al. (2010). This means that the imaginary part of the cross-spectra observation is immune to noise, and therefore depends only on the directional source location. According to this, one may write

(15.34)

where ℑ{ · } denotes the imaginary part of a complex number. We may now use this observation in order to create a model of the imaginary cross-spectra terms through the vector

(15.35)

which contains all the M(M − 1)/2 pairwise sensor combinations, with delays δ_m(θ) and δ_n(θ) defined according to the incident plane wave angle θ. This vector can be seen as an alternative steering vector with the property a(ω, θ ± π) = −a(ω, θ), for θ in radians. Consider now the vector constructed by the vertical concatenation of M(M − 1)/2 imaginary observation cross-spectra terms as

(15.36)

where Φ_mn(τ, ω) corresponds to the element at the mth row and nth column of the signal covariance matrix

(15.37)

Now, z(τ, ω) in Equation (15.36) represents an immune-to-noise observation of the sound field, which should be consistent with respect to the model in Equation (15.35), given that the sound field is a plane wave with an incident angle θ. The most likely DOA at frequency ω and time frame τ can be found by searching for the plane wave signature a(ω, θ) that is most similar to z(τ, ω):

(15.38)

where implies normalization with the Euclidean norm so that all plane wave signatures have the same energy. The source power spectrum at the particular TF point is then found by projecting a(ω, θ_{τ, ω}) onto z(τ, ω) as

(15.39)

On the other hand, an estimation of the total acoustic power can be found by averaging the power across all microphones as ϕ_yy(τ, ω) = tr(Φ)/M, where tr( · ) is the sum of all the diagonal terms of a matrix. The ratio q(τ, ω) = ϕ_ss(τ, ω)/ϕ_yy(τ, ω) then represents a useful metric that can be associated with the diffuseness of the sound field; intuitively, 0 ≤ q(τ, ω) ≤ 1 should hold, with a value close to 1 dictating a purely directional sound field whereas a value close to 0 implies a purely noisy sound field. Furthermore, we may use this metric in order to establish a relation with the so-called diffuseness of the sound field:

(15.40)

where the function min{ · } returns the minimum of a set of numbers and is useful here in order to ensure that the diffuseness does not take negative values, if q(τ, ω) > 1.

The input signal at the lth loudspeaker is then synthesized as Y_l(τ, ω) = Y^dir_l(τ, ω) + Y^{τ, dif}_l(ω), where

(15.41)

(15.42)

Here, X_ref(τ, ω) is the captured signal at a reference microphone (the first microphone), g_l is the VBAP gain responsible for the lth loudspeaker and are the beamformer weights derived in Section 15.4.3. As can be seen, at each TF point, the diffuse channels of this method are nothing but a weighted replica of the loudspeakers signals produced with the parametric technique of Section 15.4.2.

15.4.4 Example Application

Both the parametric and the non-parametric approaches presented above were applied to a real recording produced with the eight-microphone circular array of radius R = 0.05 m shown in Figure 15.12(a). The recording took place in a crowded open stadium during a football match of the Greek Super League. Figure 15.12(b) presents a sketch of the football stadium with the location of the array represented by a white dot. The array was placed at a height of 0.8 m in front of Gates 13 and 14, which were populated with the organized fans of the team which was hosting the game. These fans were cheering and singing constantly throughout the entire duration of the recording, thus providing most of the acoustic information captured by the array.

The reproduction system that we considered for the application phase consisted of four loudspeakers uniformly distributed around the azimuth plane, specifically at 45°, 135°, −135°, and −45° (see Figure 15.12), at a radius of 2.10 m. With respect to the listener’s orientation, these loudspeakers were located at rear right (RR), front right (FR), front left (FL), and rear left (RL). This configuration is of particular interest as it can easily be extended with a 5.1 surround system, adding one more channel to be used for the commentator. The panning gains derived from VBAP for this setup are identical to those depicted in Figure 15.9(a).

The signals were recorded at 44 100 Hz and processed separately with the parametric and the non-parametric approach. For the STFT we used a Hanning window of length 2048 samples and hop size 1024 samples (50% overlap). For the non-parametric method, we used the frequency-varying regularization parameter of Equation (15.28), with λ = 0.003. The variation of μ as a function of the frequency is illustrated with the gray line in Figure 15.10, while the polar plots of Figure 15.11 are illustrative of the deviation between the desired and the actual beamformer directivities.

Avoiding the problem with the imperfect beam shapes of the non-parametric method, one expects the parametric approach to produce better localization of the different acoustic sources. The opinion of the authors, who were also present inside the stadium during the recording, is that the parametric method indeed produced a more convincing sense of direction in comparison to beamforming, and was also more consistent with respect to changes in the orientation of the listener’s head inside the listening area, as well as with reasonable displacements from the sweet spot. On the other hand, the non-parametric method provided a more blurred sense of direction but a much better sense of the reverberation in the stadium. As additional evidence for this contradiction between the two methods, we have plotted in Figure 15.13 the rectified loudspeakers’ signal amplitudes in time, as derived by each technique, for a short-duration segment where the crowd at Gates 13 and 14 was by far the most dominant acoustic source inside the stadium. As the acoustic energy is concentrated at a particular part of the scene, one should expect an uneven distribution of the signal energy across the different channels, which is what we actually observe for the parametric method in Figure 15.13(b). On the other hand, the non-parametric method produces more coherent loudspeaker signals, resulting in reduced separation of different sound sources across the available loudspeaker channels. In Figure 15.13(a) this is observed as an increased contribution from loudspeakers at irrelevant directions with respect to where Gates 13 and 14 actually are.

It is worth at this point spending a few words on the aspect of sound quality. As previously explained, the non-parametric method is subject to spectral coloration caused by the matrix inversion issues explained in Section 15.4.2. We believe, however, that this problem is sufficiently addressed by the regularization method introduced in this chapter, and interested readers are invited to judge for themselves by listening to the sound examples provided online at http://users.ics.forth.gr/nstefana/Stadium2016. On the other hand, the parametric method is prone to musical noise, caused by the rapid variations of the estimated angle across time at each frequency bin. More specifically, at each time frame, at most two loudspeakers reproduce a certain frequency component. As the assignment of frequency components to loudspeakers may change abruptly from one time instant to another, this is perceived as musical noise. However, while this type of degradation is indeed perceived when listening to each channel alone, informal listening tests revealed that in multichannel listening conditions these degradations are not strongly perceived. This is caused by the fact that as the sounds from all channels sum at the center of the loudspeaker array, the components responsible for the phenomenon of musical noise are masked by the superposition of sound from all the available channels. The interested reader may again verify this claim by listening to the sound files provided, but we recommend the use of a surround multichannel system for this purpose.