Title of Invention	AUDIO CODING
Abstract	A spatial decoder unit (23) is arranged for transforming one or more audio channels (s; 1, r) into a pair of binaural output channels (lb, rb). The device comprises a parameter conversion unit (234) for converting the spatial parameters (sp) into binaural parameters (bp) containing binaural information. The device additionally comprises a spatial synthesis unit (232) for transforming the audio channels (L, R) into a pair of binaural signals (Lb, Rb) while using the binaural parameters (bp). The spatial synthesis unit (232) preferably operates in a transform domain, such as the QMF domain. Fig. 6

Title of Invention

AUDIO CODING

Abstract

A spatial decoder unit (23) is arranged for transforming one or more audio channels (s; 1, r) into a pair of binaural output channels (lb, rb). The device comprises a parameter conversion unit (234) for converting the spatial parameters (sp) into binaural parameters (bp) containing binaural information. The device additionally comprises a spatial synthesis unit (232) for transforming the audio channels (L, R) into a pair of binaural signals (Lb, Rb) while using the binaural parameters (bp). The spatial synthesis unit (232) preferably operates in a transform domain, such as the QMF domain. Fig. 6

Full Text	The present invention relates to audio coding. More in particular, the present invention relates to a device for and a method of converting an audio input signal into a binaural output signal, wherein the input signal comprises at least one audio channel and parameters representing additional channels. It is well known to record and reproduce binaural audio signals, that is, audio signals which contain specific directional information to which the human ear is sensitive. Binaural recordings are typically made using two microphones mounted in a dummy human head, so that the recorded sound corresponds to the sound captured by the human ear and includes any influences due to the shape of the head and the ears. Binaural recordings differ from stereo (that is, stereophonic) recordings in that the reproduction of a binaural recording requires a headset, whereas a stereo recording is made for reproduction by loudspeakers. While a binaural recording allows a reproduction of all spatial information using only two channels, a stereo recording would not provide the same spatial perception. Regular dual channel (stereophonic) or multiple channel (e.g. 5.1) recordings may be transformed into binaural recordings by convolving each regular signal with a set of perceptual transfer functions. Such perceptual transfer functions model the influence of the human head, and possibly other objects, on the signal. A well-known type of perceptual transfer function is the so-called Head-Related Transfer Function (HRTF). An alternative type of perceptual transfer function, which also takes into account reflections caused by the walls, ceiling and floor of a room, is the Binaural Room Impulse Response (BRIR). In the case of multiple channel signals, transforming the signals into binaural recording signals with a set of perceptual functions typically implies a convolution of perceptual functions with the signals of all channels. As a typical convolution is computationally demanding, the signals and the HRTF are typically transformed to the frequency (Fourier) domain where the convolution is replaced with a computationally far less demanding multiplication. It is further well known to reduce the number of audio channels to be transmitted or stored by representing the original number of chaimels by a smaller number of channels and parameters indicative of the relationships between the original channels. A set of stereo signals may thus be represented by a single (mono) channel plus a number of associated spatial parameters, while a set of 5.1 signals may be represented by two channels and a set of associated spatial parameters, or even by a single channel plus the associated spatial parameters. This "downmixing" of multiple audio channels in spatial encoders, and the corresponding "upmixing" of audio signals in spatial decoders, is typically carried out in a transform domain or sub-band domain, for example the QMF (Quadrature Mirror Filter) domain. Patent Cooperation Treaty Patent Application WO2004/028204 discloses a system for generating binaural signals using Head Related Transfer Functions; the document "The Reference Model Architecture for MPEG Spatial Audio Coding"; Herre et al; Audio Engineering Society Convention Paper, New York; 28 May 2005; XP009059973 discloses the MPEG reference Model architecture; the document "Synthetic Ambiance in Parametric Stereo Coding", Endegard et al; Preprints of papers presented at the AES Convention, 8 May 2004, pages 1-12, XP008048096 discloses an example of parametric stereo coding; and the document "MP3 Surround: Efficient and Compatible Coding of Multi-Channel Audio"; Herre et al. Audio Engineering Society, Convention Preprint, 8 May 2004; XP002338414 discloses examples of MP3 Surround encoding. When downmixed input channels are to be converted into binaural output charmels, the Prior Art approach is to first upmix the input channels using a spatial decoder to produce upmixed intermediary channels, and then convert these upmixed intermediary channels into binaural channels. This procedure typically produces five or six intermediary channels, which then have to be reduced to two binaural channels. First expanding and then reducing the number of channels is clearly not efficient and increases the computational complexity. In addition, reducing the five or six intermediary channels meant for multiple channel loudspeaker reproduction to only two channels meant for binaural reproduction inevitably introduces artifacts and therefore decreases the sound quality. The QMF domain referred to above is similar, but not identical, to the frequency (Fourier transform) domain. If a spatial decoder is to produce binaural output signals, the downmixed audio signals would first have to be transformed to the QMF domain for upmixing, then be inversely QMF transformed to produce time domain intermediary signals, subsequently be transformed to the frequency domain for multiplication with the (Fourier transformed) HRTF, and finally be inversely transformed to produce time domain output signals. It will be clear that this procedure is not efficient, as several transforms must be performed in succession. The number of computations involved in this Prior Art approach would make it very difficult to design a hand-held consumer device, such as a portable MP3 player, capable of producing binaural output signals from downmixed audio signals. Even if such a device could be implemented, its battery life would be very short due to the required computational load. It is an object of the present invention to overcome these and other problems of the Prior Art and to provide a spatial decoder unit capable of producing a pair of binaural output channels from a set of downmixed audio channels represented by one audio input channels and an associated set of spatial parameters, which decoder has an increased efficiency. Accordingly, the present invention provides a spatial decoder unit for producing a pair of binaural output channels using spatial parameters and a single audio input channels, the device comprising a parameter conversion unit for converting the spatial parameters into binaural parameters using parameterized perceptual transfer functions, the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions; a single transform unit for transforming the single audio input channel into a transformed audio channel; a decorrelation unit for decorrelating the transformed audio channel to generate a transformed decorrelated signal; and a spatial synthesis unit for synthesizing a pair of transformed binaural channels by applying the binaural parameters to the transformed audio channel (S) and the transformed decorrelated signal; and a pair of inverse transform units for inversely transforming the transformed binaural channels into the pair of binaural output channels. By converting the spatial parameters into binaural parameters, the spatial synthesis unit can directly synthesize a pair of binaural channels, without requiring an additional binaural synthesis unit. As no superfluous intermediary signals are produced, the computational requirements are reduced while the introduction of artifacts is substantially eliminated. In the spatial decoder unit of the present invention, the synthesis of the binaural channels can be carried out in the transform domain, for example the QMF domain, without requiring the additional steps of transformation to the frequency domain and the subsequent inverse transformation to the time domain. As two transform steps can be omitted, both the number of computations and the memory requirements are significantly reduced. The spatial decoder unit of the present invention can therefore relatively easily be implemented in a portable consumer device. Furthermore, in the spatial decoder unit of the present invention, binaural channels are produced directly from a downmixed channel, each binaural channel comprising binaural signals for binaural reproduction using a headset or a similar device. The parameter conversion unit derives the binaural parameters used for producing the binaural channels from spatial (that is, upmix) parameters. This derivation of the binaural parameters involves parameterized perceptual transfer functions, such as HRTFs (Head-Related Transfer Functions) and/or Binaural Room Impulse Responses (BRIRs). According to the present invention, therefore, the processing of the perceptual transfer functions is performed in the parameter domain, while in the Prior Art this processing was carried out in the time domain or the frequency domain. This may result in a further reduction of the computational complexity as the resolution in the parameter domain is typically lower than the resolution in the time domain or the frequency domain. It is preferred that the parameter conversion unit is arranged for combining in the parameter domain, in order to determine the binaural parameters, all perceptual transfer function contributions the input (downmix) audio channels would make to the binaural channels. In other words, the spatial parameters and the parameterized perceptual transfer functions are combined in such a manner that the combined parameters result in a binaural output signal having similar statistical properties to those obtained in the Prior Art method involving upmixed intermediary signals. In a preferred embodiment, the spatial decoder unit of the present invention further comprises one or more transform units for transforming the audio input channels into transformed audio input channels, and a pair of inverse transform units for inversely transforming the synthesized binaural channels into the pair of binaural output channels, wherein the spatial synthesis unit is arranged for operating in a transform domain or sub-band domain, preferably the QMF domain. The spatial decoder unit of the present invention may comprise two transform units, the parameter conversion unit being arranged for utilizing perceptual transfer function parameters involving three channels only, two of these three channels incorporating the contributions of composite front and rear channels. In such an embodiment, the parameter coilversion unit may be arranged for processing channel level (e.g. CLD), channel coherence (e.g. ICC), channel prediction (e.g. CPC) and/or phase (e.g. IPD) parameters. The spatial decoder unit of the present invention comprises only a single transform unit, and further comprises a decorrelation unit for decorrelating the transformed single channel output by the single transform unit. In such an embodiment, the parameter conversion unit may be arranged for processing channel level (e.g. CLD), channel coherence (e.g. ICC), and/or phase (e.g. IPD) parameters. The spatial decoder unit of the present invention may additionally comprise a stereo reverberation unit. Such a stereo reverberation unit may be arranged for operating in the time domain or in a transform domain or sub-band (e.g. QMF) domain. The present invention also provides a spatial decoder device for producing a pair of binaural output channels from an input bitstream, the device comprising a demultiplexer unit for demultiplexing the input bitstream into a single downmix channel and signal parameters, a downmix decoder unit for decoding the single downmix channel, and a spatial decoder unit for producing a pair of binaural output channels using the spatial parameters and the single downmix channel. In addition, the present invention provides a consumer device and an audio system comprising a spatial decoder unit and/or spatial decoder device as defined above. The present invention further provides a method of producing a pair of binaural output channels using spatial parameters and a single audio input channels, the method comprising the steps of converting the spatial parameters into binaural parameters using parameterized perceptual transfer functions, the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions; transforming the single audio input channel into a transformed audio channel; decorrelating the transformed audio channel to generate a transformed decorrelated signal; and synthesizing a pair of transformed binaural channels by applying the binaural parameters to the transformed audio channel (S) and the transformed decorrelated signal and inversely transforming the transformed binaural channels into the pair of binaural output channels. . Further aspects of the method according to the present invention will become apparent from the description below. The present invention additionally provides a computer program product for carrying out the method as defined above. A computer program product may comprise a set of computer executable instructions stored on a data carrier, such as a CD or a DVD. The set of computer executable instructions, which allow a programmable computer to carry out the method as defined above, may also be available for downloading from a remote server, for example via the Internet. The present invention will further be explained below with reference to exemplary embodiments illustrated in the accompanying drawings, in which: Fig. 1 schematically shows the application of head-related transfer functions according to the Prior Art. Fig. 2 schematically shows a spatial audio encoder device according to the Prior Art. Fig. 3 schematically shows a spatial audio decoder device according to the Prior Art coupled to a binaural synthesis device. Fig. 4 schematically shows a spatial audio decoder unit according to the Prior Art. Fig. 5 schematically shows an example of a spatial audio decoder unit. Fig. 6 schematically shows a spatial audio decoder device according to the present invention. Fig. 7 schematically shows the spatial audio decoder unit of Fig. 6, provided with a transform domain reverberation unit. Fig. 8 schematically shows the spatial audio decoder unit of Fig. 6, provided with a time domain reverberation unit. Fig. 9 schematically shows a consumer device provided with a spatial audio decoder device according to the present invention. The application of perceptual transfer functions, such as Head-Related Transfer Functions (HRTFs), in accordance with the Prior Art is schematically illustrated in Fig. 1. The binaural synthesis device 3 is shown to comprise six HRTF units 31, each containing the transfer function for a specific combination of an input channel and an output channel. In the example shown, there are three audio input channels chl, ch2 and ch3, which may correspond to the channels 1 (left), c (center) and r (right). The first channel chl is fed to two HRTF units 31 containing HRTF(1,L) and HRTF(1,R) respectively. In this example, HRTF(1,L) is the head-related transfer function which determines the contribution of the first channel to the left binaural signal. Those skilled in the art will know that HRTFs may be determined by making both regular (stereo) recordings and binaural recordings, and deriving a transfer function which represents the shaping of the binaural recording relative to the regular recording. Binaural recordings are made using two microphones mounted in a dummy human head, so that the recorded sound corresponds to the sound captured by the human ear and includes any influences due to the shape of the head and the ears, and even the presence of hair and shoulders. If the HRTF processing takes place in the time domain, the HRTFs are convolved with the (time domain) audio signals of the channels. Typically, however, the HRTFs are transformed to the frequency domain, and the resulting transfer fimctions and the frequency spectra of the audio signals are then multiplied (Fourier transform units and inverse Fourier transform units are not shown in Fig. 1). Suitable Overlap-and-Add (OLA) techniques involving overlapping time frames may be used to accommodate HRTFs having a greater length than the Fast Fourier Transform (FFT) frames. After HRTF processing by the appropriate HRTF unit 31, the resulting left and right signals are added by a respective adder 32 to yield the (time domain) left binaural signal lb and the right binaural signal rb. The exemplary Prior Art binaural synthesis device 3 of Fig. 1 has three input channels. Present-day audio systems often have five or six channels, as is the case in so-called 5.1 systems. However, in order to reduce the amount of data to be transferred and/or stored, the multiple audio channels are typically reduced ("downmixed") to one or two channels. A number of signal parameters indicative of the properties and mutual relationships of the original channels allows an expansion ("upmixing") of the one or two channels to the original number of channels. An exemplary spatial encoder device 1 according to the Prior Art is schematically illustrated in Fig. 2. The spatial encoder device 1 comprises a spatial encoding (SE) unit 11, a downmix encoding (DE) unit 12 and a multiplexer (Mux) 13. The spatial encoding unit 11 receives five audio input channels If (left front), Ir (left rear), rf (right front), rr (right rear) and c (center). The spatial encoding unit 11 downmixes the five input channels to produce two channels 1 (left) and r (right), as well as signal parameters sp (it is noted that the spatial encoding unit 11 may produce a single channel instead of the two channels 1 and r). In the embodiment shown, where five channels are downmixed to two channels (a so-called 5-2-5 configuration), the signal parameters sp may for example comprise: It is noted that "Ife" is an optional low frequency (sub-woofer) channel, and that the "rear" channels are also known as "surround" channels. The two down mix channels 1 and r produced by the spatial encoding unit 11 are fed to the downmix encoding (DE) unit 12, which typically uses a type of coding aimed at reducing the amount of data. The thus encoded downmix channels 1 and r, and the signal parameters sp, are multiplexed by the multiplexer unit 13 to produce an output bit stream bs. In an alternative embodiment (not shown), five (or six) channels are downmixed to a single (mono) channel (a so-called 5-1-5 configuration), and the signal parameters sp may for example comprise: In this alternative embodiment the encoded downmix channel s, as well as the signal parameters sp, are also multiplexed by the multiplexer unit 13 to produce an output bit stream bs. If this bitstream bs were to be used to produce a pair of binaural channels, the Prior Art approach would be to first upmix the two downmix channels 1 and r (or, alternatively, the single downmix channel) to produce the five or six original channels, and then convert these five or six charmels into two binaural channels. An example of this Prior Art approach is illustrated in Fig. 3. The spatial decoder device 2' according to the Prior Art comprises a demultiplexer (Demux) unit 21', a downmix decoding unit 22', and a spatial decoder unit 23'. A binaural synthesis device 3 is coupled to the spatial decoder unit 23' of the spatial decoder device 2'. The demultiplexer unit 21' receives a bitstream bs, which may be identical to the bitstream bs of Fig. 2, and outputs signal parameters sp and two encoded downmix channels. The signal parameters sp are sent to the spatial decoder unit 23', while the encoded downmix channels are first decoded by the downmix decoding unit 22' to produce the decoded downmix charmels 1 and r. The spatial decoder unit 23' essentially carries out the inverse operations of the spatial encoding unit 11 in Fig. 2 and outputs five audio channels. These five audio charmels are fed to the binaural synthesis device 3, which may have a structure similar to the device 3 of Fig. 1 but with additional HRTF units 31 to accommodate all five channels. As in the example of Fig. 1, the binaural synthesis device 3 outputs two binaural channels lb (left binaural) and rb (right binaural). An exemplary structure of the spatial decoder unit 23' of the Prior Art is shown in Fig. 4. The unit 23' of Fig. 4 comprises a two-to-three upmix unit 230', three spatial synthesis (SS) units 232' and three decorrelation (D) units 239'. The two-to-three upmix unit 230' receives the downmix channels 1 & r and the signal parameters sp, and produces three channels 1, r and ce. Each of these charmels is fed to a decorrelator unit 239' which produces a decorrelated version of the respective channel. Each channel 1, r and ce, its respective decorrelated version, and associated signal parameters sp are fed to a respective spatial synthesis (or upmix) unit 232'. The spatial synthesis unit 232' receiving the channel 1, for example, outputs the output channels If (left front) and Ir (left rear). The spatial synthesis units 232' typically perform a matrix multiplication, the parameters of the matrix being determined by the signal parameters sp. It is noted that in the example of Fig. 4 six output channels are produced. In some embodiments, the third decorrelation unit 239' and the third spatial synthesis unit 232' may be omitted, thus producing only five output channels. In all embodiments, however, the spatial synthesis unit 23' of the Prior Art will produce more than two output channels. It is further noted that any (QMF) transform units and inverse (QMF) transform units have been omitted from the merely illustrative example of Fig. 4 for the sake of clarity of the illustration. In actual embodiments the spatial decoding would be carried out in a transform domain, such as the QMF domain. The configuration of Fig. 3 is not efficient. The spatial decoder device 2' converts two downmix channels (1 and r) into five upmixed (intermediary) channels, while the binaural synthesis device 3 then reduces the five upmixed channels to two binaural channels. In addition, the upmixing in the spatial decoder unit 23' is typically carried out in a sub-band domain, such as the QMF (Quadrature Mirror Filter) domain. However, the binaural synthesis device 3 typically processes signals in the frequency (that is, Fourier transform) domain. As these two domains are not identical, the spatial decoder device T first transforms the signals of the downmix channels into the QMF domain, processes the transformed signals, and then transforms the upmixed signals back to the time domain. Subsequently, the binaural synthesis device 3 transforms all (five in the present example) these upmixed signals into the frequency domain, processes the transformed signals, and then transforms the binaural signals back into the time domain. It will be clear that the computational effort involved is considerable, and that a more efficient signal processing is desired, in particular when this processing is to be carried out in a hand-held device. The present invention provides a far more efficient processing by integrating the binaural synthesis device in the spatial decoder device and effectively carrying out the binaural processing in the parameter. An example of a spatial decoder unit is schematically illustrated in Fig. 5, while a combined spatial and binaural decoder device according to the present invention (referred to as spatial decoder device for the sake of brevity) is illustrated in Fig. 6. The spatial decoder unit 23 shown in Fig. 5 comprises transform units 231, a spatial synthesis (SS) unit 232, inverse transform units 233, a parameter conversion (PC) unit 234 and a memory (Mem) unit 235. In the exemple of Fig. 5, the spatial decoder unit 23 comprises two transform units 231, but in other examples only a single transform unit 231 (as in Fig. 6), or more than two transform units 231 may be present, depending on the number of downmix channels. The transform units 231 each receive a downmix channel 1 and r respectively (see also Fig. 3). Each transform unit 231 is arranged for transforming the (signal of the) respective channel into a suitable transform or sub-band domain, in the present example the QMF domain. The QMF transformed channels L and R are fed to the spatial synthesis unit 23'2 which preferably carries out a matrix operation on the signals of the channels L and R to produce the transform domain binaural channels Lb and Rb. Inverse transform units 233 carry out an inverse transform, in the present example an inverse QMF transform, to produce the binaural time domain channels lb and rb. The spatial synthesis unit 232 may be similar or identical to the Prior Art spatial synthesis unit 232' of Fig. 4. However, the parameters used by this unit are different from those used in the Prior Art. More in particular, the parameter conversion unit 234 converts the conventional spatial parameters sp into binaural parameters bp using HRTF parameters hp stored in the memory unit 235. These HRTF parameters hp may comprise: an average level per frequency band for the left transfer function as a function of azimuth (angle in a horizontal plane), elevation (angle in a vertical plane), and distance, an average level per frequency band for the right transfer function as a function of azimuth, elevation and distance, and an average phase or time difference per frequency band as a as a function of azimuth, elevation and distance. In addition, the following parameters may be included: a coherence measure of the left and right transfer functions per HRTF frequency band as a function of azimuth, elevation and distance, and/or absolute phase and/or time parameters for the left and right transfer functions as a function of azimuth, elevation and distance. The actual HRTF parameters used may depend on the particular embodiment. The spatial synthesis unit 232 may determine the binaural channels Lb and Rb using the following formula: where the index k denotes the QMF hybrid (frequency) band index and the index m denotes the QMF slot (time) index. The parameters hij of the matrix Hk are determined by the binaural parameters (bp in Fig. 5). As indicated by the index k, the matrix Hk may depend on the QMF hybrid band. In one example, the parameter conversion unit (234 in Fig. 5) produces the binaural parameters which are then converted into the matrix parameters hij by the spatial synthesis unit 232. In another example, the matrix parameters hij are identical to the binaural parameters produced by the parameter conversion unit (234 in Fig. 5) and can be directly applied by the spatial synthesis unit 232 without being converted. The parameters hij of the matrix Hk may be determined in the following way in the case of two downmix channels (5-2-5 configuration). In the Prior Art spatial decoder unit of Fig. 4, a 2-to-3 decoder unit 230' converts the two (input) downmix channels 1 and r into three (output) channels 1, r, and ce (it will be understood that the output channels 1 and r will typically not be identical to the input channels 1 and r, for this reason the input channels will in the following discussion be labeled lo and ro). In accordance with a examplethe parameter conversion unit (234 in Figs. 5 & 6) is arranged for utilizing perceptual transfer function parameters where the contribution of only three channels only (e.g. 1, r and c) is taken into account, two of these three channels (e.g. 1 and r) comprising composite respective front (If, rf) and rear (Ir, rr) channels. That is, the respective front and rear channels are grouped together to improve the efficiency. The operation of the two-to-three upmix unit 230' can be described by the following matrix operation: with matrix entries mij dependent on the spatial parameters. The relation of spatial parameters and matrix entries is identical to those of a 5.1 MPEG surround decoder. For each of the three resulting signals /, r and c, the effect is determined of the perceptual transfer function (in the present example: HRTF) parameters which correspond to the desired (perceived) position of these sound sources. For the center channel (c), the spatial parameters of the sound source position can be applied directly, resulting in two output signals for center, IB(C) and rB(c); As can be observed from equation (3), the HRTF parameter processing consists of a multiplication of the signal with average power levels Pj and Pr corresponding to the sound source position of the center channel, while the phase difference is distributed symmetrically. This process is performed independently for each QMF band, using the mapping from HRTF parameters to QMF filter bank on the one hand, and mapping from spatial parameters to QMF band on the other hand. For the left (/) channel, the HRTF parameters from the left-front and left-rear channels are combined into a single contribution, using the weights Ml/and wrf . The resulting composite parameters simulate the effect of both the front and rear channels in a statistical sense. The following equations are used to generate the binaural output pair (4, rt) for the left channel: It is noted that the phase modification term is applied to the contra-lateral ear in both cases. Furthermore, since the human auditory system is largely insensitive to binaural phase for frequencies above approx. 2 kHz, the phase modification term only needs to be applied in the lower frequency region. Hence for the remainder of the frequency range, real-valued processing suffices (assuming real-valued mij). It is further noted that the equations above assume incoherent addition of the (HRTF) filtered signals of 1f and Ir, One possible extension would be to include the transmitted Inter-Channel Coherence (ICC) parameters of 1f"and Ir (and of 1f"and rr) in the equations as well to account for front/rear correlation. All processing steps described above can be combined in the parameter domain to result in a single, signal-domain 2x2 matrix: The present invention essentially processes the binaural (that is, HRTF) information in the parameter domain, instead of in the fi-equency or time domain as in the Prior Art. In this way, significant computational savings may be obtained. The spatial decoder device 2 according to the present invention shown merely by way of non-limiting example in Fig. 6 comprises a demultiplexer (Demux) unit 21, a downmix decoding unit 22, and a spatial/binaural decoder unit 23. The demultiplexer unit 21 and the downmix decoding unit 22 may be similar to the Prior Art demultiplexer unit 21' and the downmix decoding unit 22'shown in Fig. 3. The spatial decoder unit 23 of Fig. 6 is identical to the spatial decoder unit 23 of Fig. 5, with the exception of the number of downmix channels and associated transform units. As the spatial decoder device of Fig. 6 is arranged for a single downmix channel s, only a single transform unit 231 is provided while a decorrelation (D) unit 239 has been added for producing a decorrelated version D of the (transform domain) downmix signal S. The binaural parameters bp produced by the parameter conversion unit 234 typically differ from those in the embodiment of Fig, 5, as the signal parameters sp associated with a single downmix channel s typically differ from those associated with two downmix chaimels. In the configuration of Fig. 6, the input of the binaural decoder comprises a mono input signal s accompanied by spatial parameters sp. The binaural synthesis unit generates a stereo output signal with statistical properties that approximate those properties that would result from HRTF processing of the original 5.1 input, which can be described by: Given the spatial parameters which describe statistical properties and inter¬relations of the channels If, rf Ir, rr and c, and the parameters of the HRTF impulse responses, it is possible to estimate the statistical properties (that is, an approximation of the binaural parameters) of the binaural output pair //,, Vb as well. More specifically, the average energy (for each channel), the average phase difference and the coherence can be estimated and subsequently re-instated by means of decorrelation and matrixing of the mono input signal. The binaural parameters comprise a (relative) level change for each of the two binaural output channels (and hence define a Channel Level Difference parameter), an (average) phase difference and a coherence measure (per transform domain time/frequency tile). As a first step, the relative powers (with respect to the power of the mono input signal) of the five (or six) channel (5.1) signal are computed using the transmitted CLD parameters. The relative power of the left-front channel is given by: The expected value of the relative power cji of the left binaural output channel (with respect to the mono input channel), the expected value of the relative power Σ 7R of the right binaural output channel, and the expected value of the cross product LBRB can then be calculated. The coherence of the binaural output (ICCB) is then given by: Finally, the overall (linear) gain of the binaural output compared to the mono input, gB, is given by: The matrix coefficients required to re-instate the IPDB, CLDB, ICCB and gs parameters in the binaural matrix are simply obtained from a conventional parametric stereo decoder, extended with overall gains gn: Further embodiments of the spatial decoder unit of the present invention may contain a reverberation unit. It has been found that adding reverberation improves the perceived distance when binaural sound is produced. For this reason, the spatial decoder unit 23 of Fig. 7 is provided with a stereo reverberation unit 237 connected in parallel with the spatial synthesis unit 232. The stereo reverberation unit 237 of Fig. 7 receives the QMF transform domain single downmix signal S and outputs two reverberation signals, which are added to the transform domain binaural signals (channels Lb and Lr in Fig. 6) by addition units 238. The combined signals are then inversely transformed by inverse transform units 233 before being output. In the embodiment of Fig. 8, the stereo reverberation unit 237 is arranged for producing a reverberation in the time domain and receives the time domain single downmix signal s. The stereo reverberation unit 237 outputs time domain reverberation signals, which are added to the time domain signals of the binaural channels lb and rb by the addition units 238. Either embodiment provides a suitable reverberation. The present invention additionally provides a consumer device, such as a hand-held consumer device, and an audio system comprising a spatial decoder unit or spatial decoder device as defined above. The hand-held consumer device may be constituted by an MP3 player or similar device. A consumer device is schematically illustrated in Fig. 9. The consumer device 50 is shown to comprise a spatial decoder device 2 according to the present invention (see Fig. 6). The present invention is based upon the insight that the computational complexity of a combined spatial decoder device and a binaural synthesis device may be significantly reduced by modifying the spatial parameters in accordance with the binaural information. This allows the spatial decoder device to carry out spatial decoding and perceptual transfer function processing effectively in the same signal processing operation, while avoiding the introduction of any artifacts. It is noted that any terms used in this document should not be construed so as to limit the scope of the present invention. In particular, the words "comprise(s)" and "comprising" are not meant to exclude any elements not specifically stated. Single (circuit) elements may be substituted with multiple (circuit) elements or with their equivalents. It will be understood by those skilled in the art that the present invention is not limited to the embodiments illustrated above and that many modifications and additions may be made without departing from the scope of the invention as defined in the appending claims. CLAIMS: 1. A spatial decoder unit (23) for producing a pair of binaural output channels (lb, rb) using spatial parameters (sp) and a single audio input channel (s), the device comprising: a parameter conversion unit (234) for converting the spatial parameters (sp) into binaural parameters (bp) using parametrized perceptual transfer functions (hp), the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions; a single transform unit (231) for transforming the single audio input channel (s) into a transformed audio channel (S); a decorrelation unit (239) for decorrelating the transformed audio channel (S) to generate a transformed decorrelated signal (D); a spatial synthesis unit (232) for synthesizing a pair of transformed binaural channels (Lb, Rb) by applying the binaural parameters (bp) to the transformed audio channel (S) and the transformed decorrelated signal (D); and a pair of inverse transform units (233) for inversely transforming the transformed binaural channels (Lb, Rb) into the pair of binaural output channels (lb, rb). 2. The spatial decoder unit according to claim 1, wherein the parameter conversion unit (234) is arranged for combining in the parameter domain, in order to determine the binaural parameters, all perceptual transfer function contributions the audio input channels would make to the binaural channels. 3. The spatial decoder unit according to claim 1, wherein the parameter conversion unit (234) is arranged for processing channel level (CLD), channel coherence (ICC) and/or phase (IPD) parameters. 4. The spatial decoder unit according to claim 1, further comprising a stereo reverberation unit (237) arranged for operating in the time domain. 5. The spatial decoder unit according to claim 1, further compnsmg a stereo reverberation unit (237) arranged for operating in a transform domain or sub-band domain, such as the QMF domain. 6. The spatial decoder unit according to claim 1 wherein the parameter conversion unit (234) comprises: means for determining relative powers for a plurality of virtual audio channels in response to the spatial parameters (sp) and the single audio input channel (s); means for estimating statistical properties of the binaural output chaxmels (lb, rb) in response to the relative powers for the plurality of virtual audio channels; and means for determining the binaural parameters (bp) in response to the estimated statistical properties of the binaural output channels (lb, rb). 7. The spatial decoder unit according to claim 6 wherein the statistical properties comprise an average energy of the transformed binaural channels (Lb, Rb), an average phase difference of the transformed binaural channels (Lb, Rb) and a coherence of the transformed binaural channels (Lb, Rb). 8. A spatial decoder device (2) for producing a pair of binaural output channels (lb, rb) from an input bitstream (bs), the device comprising: a demultiplexer unit (21) for demultiplexing the input bitstream into a single downmix channel and signal parameters (sp), a downmix decoder unit (22) for decoding the single downmix channel (s), and a spatial decoder unit (23) according to claim 1. 9. The device according to claim 8, wherein the spatial decoder unit (23) comprises a reverberation unit (237). 10. An audio system, comprising a spatial decoder unit (23) according to claim 1 and/or a spatial decoder device (2) according to claim 8. 11. A consumer device, comprising a spatial decoder unit (23) according to claim 1 and/or a spatial decoder device (2) according to claim 8. 12. A method of producing a pair of binaural output channels (lb, rb) using spatial parameters (sp) and a single audio input channel (s), the method comprising the steps of: converting the spatial parameters (sp) into binaural parameters (bp) using parameterized perceptual transfer functions (hp), the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions; transforming the single audio input channel (s) into a transformed audio channel (S); decorrelating the transformed audio channel (S) to generate a transformed decorrelated signal (D); synthesizing a pair of transformed binaural channels (Lb, Rb) by applying the binaural parameters (bp) to the transformed audio channel (S) and the transformed decorrelated signal (D);and inversely transforming the transformed binaural channels (Lb, Rb) into the pair of binaural output channels (lb, rb). 13. A computer program product for carrying out the method according to claim 12.

Full Text

The present invention relates to audio coding. More in particular, the present invention relates to a device for and a method of converting an audio input signal into a binaural output signal, wherein the input signal comprises at least one audio channel and parameters representing additional channels.
It is well known to record and reproduce binaural audio signals, that is, audio signals which contain specific directional information to which the human ear is sensitive. Binaural recordings are typically made using two microphones mounted in a dummy human head, so that the recorded sound corresponds to the sound captured by the human ear and includes any influences due to the shape of the head and the ears. Binaural recordings differ from stereo (that is, stereophonic) recordings in that the reproduction of a binaural recording requires a headset, whereas a stereo recording is made for reproduction by loudspeakers. While a binaural recording allows a reproduction of all spatial information using only two channels, a stereo recording would not provide the same spatial perception.
Regular dual channel (stereophonic) or multiple channel (e.g. 5.1) recordings may be transformed into binaural recordings by convolving each regular signal with a set of perceptual transfer functions. Such perceptual transfer functions model the influence of the human head, and possibly other objects, on the signal. A well-known type of perceptual transfer function is the so-called Head-Related Transfer Function (HRTF). An alternative type of perceptual transfer function, which also takes into account reflections caused by the walls, ceiling and floor of a room, is the Binaural Room Impulse Response (BRIR).
In the case of multiple channel signals, transforming the signals into binaural recording signals with a set of perceptual functions typically implies a convolution of perceptual functions with the signals of all channels. As a typical convolution is computationally demanding, the signals and the HRTF are typically transformed to the frequency (Fourier) domain where the convolution is replaced with a computationally far less demanding multiplication.

It is further well known to reduce the number of audio channels to be transmitted or stored by representing the original number of chaimels by a smaller number of channels and parameters indicative of the relationships between the original channels. A set of stereo signals may thus be represented by a single (mono) channel plus a number of associated spatial parameters, while a set of 5.1 signals may be represented by two channels and a set of associated spatial parameters, or even by a single channel plus the associated spatial parameters. This "downmixing" of multiple audio channels in spatial encoders, and the corresponding "upmixing" of audio signals in spatial decoders, is typically carried out in a transform domain or sub-band domain, for example the QMF (Quadrature Mirror Filter) domain.
Patent Cooperation Treaty Patent Application WO2004/028204 discloses a system for generating binaural signals using Head Related Transfer Functions; the document "The Reference Model Architecture for MPEG Spatial Audio Coding"; Herre et al; Audio Engineering Society Convention Paper, New York; 28 May 2005; XP009059973 discloses the MPEG reference Model architecture; the document "Synthetic Ambiance in Parametric Stereo Coding", Endegard et al; Preprints of papers presented at the AES Convention, 8 May 2004, pages 1-12, XP008048096 discloses an example of parametric stereo coding; and the document "MP3 Surround: Efficient and Compatible Coding of Multi-Channel Audio"; Herre et al. Audio Engineering Society, Convention Preprint, 8 May 2004; XP002338414 discloses examples of MP3 Surround encoding.
When downmixed input channels are to be converted into binaural output charmels, the Prior Art approach is to first upmix the input channels using a spatial decoder to produce upmixed intermediary channels, and then convert these upmixed intermediary channels into binaural channels. This procedure typically produces five or six intermediary channels, which then have to be reduced to two binaural channels. First expanding and then reducing the number of channels is clearly not efficient and increases the computational complexity. In addition, reducing the five or six intermediary channels meant for multiple channel loudspeaker reproduction to only two channels meant for binaural reproduction inevitably introduces artifacts and therefore decreases the sound quality.
The QMF domain referred to above is similar, but not identical, to the frequency (Fourier transform) domain. If a spatial decoder is to produce binaural output signals, the downmixed audio signals would first have to be transformed to the QMF domain for upmixing, then be inversely QMF transformed to produce time domain intermediary signals, subsequently be transformed to the frequency domain for multiplication with the

(Fourier transformed) HRTF, and finally be inversely transformed to produce time domain output signals. It will be clear that this procedure is not efficient, as several transforms must be performed in succession.
The number of computations involved in this Prior Art approach would make it very difficult to design a hand-held consumer device, such as a portable MP3 player, capable of producing binaural output signals from downmixed audio signals. Even if such a device could be implemented, its battery life would be very short due to the required computational load.
It is an object of the present invention to overcome these and other problems of the Prior Art and to provide a spatial decoder unit capable of producing a pair of binaural output channels from a set of downmixed audio channels represented by one audio input channels and an associated set of spatial parameters, which decoder has an increased efficiency.
Accordingly, the present invention provides a spatial decoder unit for producing a pair of binaural output channels using spatial parameters and a single audio input channels, the device comprising a parameter conversion unit for converting the spatial parameters into binaural parameters using parameterized perceptual transfer functions, the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions; a single transform unit for transforming the single audio input channel into a transformed audio channel; a decorrelation unit for decorrelating the transformed audio channel to generate a transformed decorrelated signal; and a spatial synthesis unit for synthesizing a pair of transformed binaural channels by applying the binaural parameters to the transformed audio channel (S) and the transformed decorrelated signal; and a pair of inverse transform units for inversely transforming the transformed binaural channels into the pair of binaural output channels.
By converting the spatial parameters into binaural parameters, the spatial synthesis unit can directly synthesize a pair of binaural channels, without requiring an additional binaural synthesis unit. As no superfluous intermediary signals are produced, the computational requirements are reduced while the introduction of artifacts is substantially eliminated.
In the spatial decoder unit of the present invention, the synthesis of the binaural channels can be carried out in the transform domain, for example the QMF domain,

without requiring the additional steps of transformation to the frequency domain and the subsequent inverse transformation to the time domain. As two transform steps can be omitted, both the number of computations and the memory requirements are significantly reduced. The spatial decoder unit of the present invention can therefore relatively easily be implemented in a portable consumer device.
Furthermore, in the spatial decoder unit of the present invention, binaural channels are produced directly from a downmixed channel, each binaural channel comprising binaural signals for binaural reproduction using a headset or a similar device. The parameter conversion unit derives the binaural parameters used for producing the binaural channels from spatial (that is, upmix) parameters. This derivation of the binaural parameters involves parameterized perceptual transfer functions, such as HRTFs (Head-Related Transfer Functions) and/or Binaural Room Impulse Responses (BRIRs). According to the present invention, therefore, the processing of the perceptual transfer functions is performed in the parameter domain, while in the Prior Art this processing was carried out in the time domain or the frequency domain. This may result in a further reduction of the computational complexity as the resolution in the parameter domain is typically lower than the resolution in the time domain or the frequency domain.
It is preferred that the parameter conversion unit is arranged for combining in the parameter domain, in order to determine the binaural parameters, all perceptual transfer function contributions the input (downmix) audio channels would make to the binaural channels. In other words, the spatial parameters and the parameterized perceptual transfer functions are combined in such a manner that the combined parameters result in a binaural output signal having similar statistical properties to those obtained in the Prior Art method involving upmixed intermediary signals.
In a preferred embodiment, the spatial decoder unit of the present invention further comprises one or more transform units for transforming the audio input channels into transformed audio input channels, and a pair of inverse transform units for inversely transforming the synthesized binaural channels into the pair of binaural output channels, wherein the spatial synthesis unit is arranged for operating in a transform domain or sub-band domain, preferably the QMF domain.
The spatial decoder unit of the present invention may comprise two transform units, the parameter conversion unit being arranged for utilizing perceptual transfer function parameters involving three channels only, two of these three channels incorporating the contributions of composite front and rear channels. In such an embodiment, the parameter

coilversion unit may be arranged for processing channel level (e.g. CLD), channel coherence (e.g. ICC), channel prediction (e.g. CPC) and/or phase (e.g. IPD) parameters.
The spatial decoder unit of the present invention comprises only a single transform unit, and further comprises a decorrelation unit for decorrelating the transformed single channel output by the single transform unit. In such an embodiment, the parameter conversion unit may be arranged for processing channel level (e.g. CLD), channel coherence (e.g. ICC), and/or phase (e.g. IPD) parameters.
The spatial decoder unit of the present invention may additionally comprise a stereo reverberation unit. Such a stereo reverberation unit may be arranged for operating in the time domain or in a transform domain or sub-band (e.g. QMF) domain.
The present invention also provides a spatial decoder device for producing a pair of binaural output channels from an input bitstream, the device comprising a demultiplexer unit for demultiplexing the input bitstream into a single downmix channel and signal parameters, a downmix decoder unit for decoding the single downmix channel, and a spatial decoder unit for producing a pair of binaural output channels using the spatial parameters and the single downmix channel.
In addition, the present invention provides a consumer device and an audio system comprising a spatial decoder unit and/or spatial decoder device as defined above. The present invention further provides a method of producing a pair of binaural output channels using spatial parameters and a single audio input channels, the method comprising the steps of converting the spatial parameters into binaural parameters using parameterized perceptual transfer functions, the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions; transforming the single audio input channel into a transformed audio channel; decorrelating the transformed audio channel to generate a transformed decorrelated signal; and synthesizing a pair of transformed binaural channels by applying the binaural parameters to the transformed audio channel (S) and the transformed decorrelated signal and inversely transforming the transformed binaural channels into the pair of binaural output channels.
. Further aspects of the method according to the present invention will become apparent from the description below.
The present invention additionally provides a computer program product for carrying out the method as defined above. A computer program product may comprise a set of computer executable instructions stored on a data carrier, such as a CD or a DVD. The set of computer executable instructions, which allow a programmable computer to carry out the

method as defined above, may also be available for downloading from a remote server, for example via the Internet.
The present invention will further be explained below with reference to exemplary embodiments illustrated in the accompanying drawings, in which:
Fig. 1 schematically shows the application of head-related transfer functions according to the Prior Art.
Fig. 2 schematically shows a spatial audio encoder device according to the Prior Art.
Fig. 3 schematically shows a spatial audio decoder device according to the Prior Art coupled to a binaural synthesis device.
Fig. 4 schematically shows a spatial audio decoder unit according to the Prior Art.
Fig. 5 schematically shows an example of a spatial audio decoder unit.
Fig. 6 schematically shows a spatial audio decoder device according to the present invention.
Fig. 7 schematically shows the spatial audio decoder unit of Fig. 6, provided with a transform domain reverberation unit.
Fig. 8 schematically shows the spatial audio decoder unit of Fig. 6, provided with a time domain reverberation unit.
Fig. 9 schematically shows a consumer device provided with a spatial audio decoder device according to the present invention.
The application of perceptual transfer functions, such as Head-Related Transfer Functions (HRTFs), in accordance with the Prior Art is schematically illustrated in Fig. 1. The binaural synthesis device 3 is shown to comprise six HRTF units 31, each containing the transfer function for a specific combination of an input channel and an output channel. In the example shown, there are three audio input channels chl, ch2 and ch3, which may correspond to the channels 1 (left), c (center) and r (right). The first channel chl is fed to two HRTF units 31 containing HRTF(1,L) and HRTF(1,R) respectively. In this example, HRTF(1,L) is the head-related transfer function which determines the contribution of the first channel to the left binaural signal.

Those skilled in the art will know that HRTFs may be determined by making both regular (stereo) recordings and binaural recordings, and deriving a transfer function which represents the shaping of the binaural recording relative to the regular recording. Binaural recordings are made using two microphones mounted in a dummy human head, so that the recorded sound corresponds to the sound captured by the human ear and includes any influences due to the shape of the head and the ears, and even the presence of hair and shoulders.
If the HRTF processing takes place in the time domain, the HRTFs are convolved with the (time domain) audio signals of the channels. Typically, however, the HRTFs are transformed to the frequency domain, and the resulting transfer fimctions and the frequency spectra of the audio signals are then multiplied (Fourier transform units and inverse Fourier transform units are not shown in Fig. 1). Suitable Overlap-and-Add (OLA) techniques involving overlapping time frames may be used to accommodate HRTFs having a greater length than the Fast Fourier Transform (FFT) frames.
After HRTF processing by the appropriate HRTF unit 31, the resulting left and right signals are added by a respective adder 32 to yield the (time domain) left binaural signal lb and the right binaural signal rb.
The exemplary Prior Art binaural synthesis device 3 of Fig. 1 has three input channels. Present-day audio systems often have five or six channels, as is the case in so-called 5.1 systems. However, in order to reduce the amount of data to be transferred and/or stored, the multiple audio channels are typically reduced ("downmixed") to one or two channels. A number of signal parameters indicative of the properties and mutual relationships of the original channels allows an expansion ("upmixing") of the one or two channels to the original number of channels. An exemplary spatial encoder device 1 according to the Prior Art is schematically illustrated in Fig. 2.
The spatial encoder device 1 comprises a spatial encoding (SE) unit 11, a downmix encoding (DE) unit 12 and a multiplexer (Mux) 13. The spatial encoding unit 11 receives five audio input channels If (left front), Ir (left rear), rf (right front), rr (right rear) and c (center). The spatial encoding unit 11 downmixes the five input channels to produce two channels 1 (left) and r (right), as well as signal parameters sp (it is noted that the spatial encoding unit 11 may produce a single channel instead of the two channels 1 and r). In the embodiment shown, where five channels are downmixed to two channels (a so-called 5-2-5 configuration), the signal parameters sp may for example comprise:

It is noted that "Ife" is an optional low frequency (sub-woofer) channel, and that the "rear" channels are also known as "surround" channels.
The two down mix channels 1 and r produced by the spatial encoding unit 11 are fed to the downmix encoding (DE) unit 12, which typically uses a type of coding aimed at reducing the amount of data. The thus encoded downmix channels 1 and r, and the signal parameters sp, are multiplexed by the multiplexer unit 13 to produce an output bit stream bs.
In an alternative embodiment (not shown), five (or six) channels are downmixed to a single (mono) channel (a so-called 5-1-5 configuration), and the signal parameters sp may for example comprise:

In this alternative embodiment the encoded downmix channel s, as well as the signal parameters sp, are also multiplexed by the multiplexer unit 13 to produce an output bit stream bs.

If this bitstream bs were to be used to produce a pair of binaural channels, the Prior Art approach would be to first upmix the two downmix channels 1 and r (or, alternatively, the single downmix channel) to produce the five or six original channels, and then convert these five or six charmels into two binaural channels. An example of this Prior Art approach is illustrated in Fig. 3.
The spatial decoder device 2' according to the Prior Art comprises a demultiplexer (Demux) unit 21', a downmix decoding unit 22', and a spatial decoder unit 23'. A binaural synthesis device 3 is coupled to the spatial decoder unit 23' of the spatial decoder device 2'.
The demultiplexer unit 21' receives a bitstream bs, which may be identical to the bitstream bs of Fig. 2, and outputs signal parameters sp and two encoded downmix channels. The signal parameters sp are sent to the spatial decoder unit 23', while the encoded downmix channels are first decoded by the downmix decoding unit 22' to produce the decoded downmix charmels 1 and r. The spatial decoder unit 23' essentially carries out the inverse operations of the spatial encoding unit 11 in Fig. 2 and outputs five audio channels. These five audio charmels are fed to the binaural synthesis device 3, which may have a structure similar to the device 3 of Fig. 1 but with additional HRTF units 31 to accommodate all five channels. As in the example of Fig. 1, the binaural synthesis device 3 outputs two binaural channels lb (left binaural) and rb (right binaural).
An exemplary structure of the spatial decoder unit 23' of the Prior Art is shown in Fig. 4. The unit 23' of Fig. 4 comprises a two-to-three upmix unit 230', three spatial synthesis (SS) units 232' and three decorrelation (D) units 239'. The two-to-three upmix unit 230' receives the downmix channels 1 & r and the signal parameters sp, and produces three channels 1, r and ce. Each of these charmels is fed to a decorrelator unit 239' which produces a decorrelated version of the respective channel. Each channel 1, r and ce, its respective decorrelated version, and associated signal parameters sp are fed to a respective spatial synthesis (or upmix) unit 232'. The spatial synthesis unit 232' receiving the channel 1, for example, outputs the output channels If (left front) and Ir (left rear). The spatial synthesis units 232' typically perform a matrix multiplication, the parameters of the matrix being determined by the signal parameters sp.
It is noted that in the example of Fig. 4 six output channels are produced. In some embodiments, the third decorrelation unit 239' and the third spatial synthesis unit 232' may be omitted, thus producing only five output channels. In all embodiments, however, the spatial synthesis unit 23' of the Prior Art will produce more than two output channels. It is

further noted that any (QMF) transform units and inverse (QMF) transform units have been omitted from the merely illustrative example of Fig. 4 for the sake of clarity of the illustration. In actual embodiments the spatial decoding would be carried out in a transform domain, such as the QMF domain.
The configuration of Fig. 3 is not efficient. The spatial decoder device 2' converts two downmix channels (1 and r) into five upmixed (intermediary) channels, while the binaural synthesis device 3 then reduces the five upmixed channels to two binaural channels. In addition, the upmixing in the spatial decoder unit 23' is typically carried out in a sub-band domain, such as the QMF (Quadrature Mirror Filter) domain. However, the binaural synthesis device 3 typically processes signals in the frequency (that is, Fourier transform) domain. As these two domains are not identical, the spatial decoder device T first transforms the signals of the downmix channels into the QMF domain, processes the transformed signals, and then transforms the upmixed signals back to the time domain. Subsequently, the binaural synthesis device 3 transforms all (five in the present example) these upmixed signals into the frequency domain, processes the transformed signals, and then transforms the binaural signals back into the time domain. It will be clear that the computational effort involved is considerable, and that a more efficient signal processing is desired, in particular when this processing is to be carried out in a hand-held device.
The present invention provides a far more efficient processing by integrating the binaural synthesis device in the spatial decoder device and effectively carrying out the binaural processing in the parameter. An example of a spatial decoder unit is schematically illustrated in Fig. 5, while a combined spatial and binaural decoder device according to the present invention (referred to as spatial decoder device for the sake of brevity) is illustrated in Fig. 6.
The spatial decoder unit 23 shown in Fig. 5 comprises transform units 231, a spatial synthesis (SS) unit 232, inverse transform units 233, a parameter conversion (PC) unit 234 and a memory (Mem) unit 235. In the exemple of Fig. 5, the spatial decoder unit 23 comprises two transform units 231, but in other examples only a single transform unit 231 (as in Fig. 6), or more than two transform units 231 may be present, depending on the number of downmix channels.
The transform units 231 each receive a downmix channel 1 and r respectively (see also Fig. 3). Each transform unit 231 is arranged for transforming the (signal of the) respective channel into a suitable transform or sub-band domain, in the present example the QMF domain. The QMF transformed channels L and R are fed to the spatial synthesis unit

23'2 which preferably carries out a matrix operation on the signals of the channels L and R to produce the transform domain binaural channels Lb and Rb. Inverse transform units 233 carry out an inverse transform, in the present example an inverse QMF transform, to produce the binaural time domain channels lb and rb.
The spatial synthesis unit 232 may be similar or identical to the Prior Art spatial synthesis unit 232' of Fig. 4. However, the parameters used by this unit are different from those used in the Prior Art. More in particular, the parameter conversion unit 234 converts the conventional spatial parameters sp into binaural parameters bp using HRTF parameters hp stored in the memory unit 235. These HRTF parameters hp may comprise:
an average level per frequency band for the left transfer function as a function of azimuth (angle in a horizontal plane), elevation (angle in a vertical plane), and distance,
an average level per frequency band for the right transfer function as a function of azimuth, elevation and distance, and
an average phase or time difference per frequency band as a as a function of azimuth, elevation and distance.
In addition, the following parameters may be included:
a coherence measure of the left and right transfer functions per HRTF frequency band as a function of azimuth, elevation and distance, and/or
absolute phase and/or time parameters for the left and right transfer functions as a function of azimuth, elevation and distance.
The actual HRTF parameters used may depend on the particular embodiment.
The spatial synthesis unit 232 may determine the binaural channels Lb and Rb using the following formula:

where the index k denotes the QMF hybrid (frequency) band index and the index m denotes the QMF slot (time) index. The parameters hij of the matrix Hk are determined by the binaural parameters (bp in Fig. 5). As indicated by the index k, the matrix Hk may depend on the QMF hybrid band. In one example, the parameter conversion unit (234 in Fig. 5) produces the binaural parameters which are then converted into the matrix parameters hij by the spatial synthesis unit 232. In another example, the matrix parameters hij are identical to the binaural

parameters produced by the parameter conversion unit (234 in Fig. 5) and can be directly applied by the spatial synthesis unit 232 without being converted.
The parameters hij of the matrix Hk may be determined in the following way in the case of two downmix channels (5-2-5 configuration). In the Prior Art spatial decoder unit of Fig. 4, a 2-to-3 decoder unit 230' converts the two (input) downmix channels 1 and r into three (output) channels 1, r, and ce (it will be understood that the output channels 1 and r will typically not be identical to the input channels 1 and r, for this reason the input channels will in the following discussion be labeled lo and ro).
In accordance with a examplethe parameter conversion unit (234 in Figs. 5 & 6) is arranged for utilizing perceptual transfer function parameters where the contribution of only three channels only (e.g. 1, r and c) is taken into account, two of these three channels (e.g. 1 and r) comprising composite respective front (If, rf) and rear (Ir, rr) channels. That is, the respective front and rear channels are grouped together to improve the efficiency.
The operation of the two-to-three upmix unit 230' can be described by the following matrix operation:

with matrix entries mij dependent on the spatial parameters. The relation of spatial parameters and matrix entries is identical to those of a 5.1 MPEG surround decoder. For each of the three resulting signals /, r and c, the effect is determined of the perceptual transfer function (in the present example: HRTF) parameters which correspond to the desired (perceived) position of these sound sources. For the center channel (c), the spatial parameters of the sound source position can be applied directly, resulting in two output signals for center, IB(C) and rB(c);

As can be observed from equation (3), the HRTF parameter processing consists of a multiplication of the signal with average power levels Pj and Pr corresponding to the sound source position of the center channel, while the phase difference is distributed symmetrically. This process is performed independently for each QMF band, using the

mapping from HRTF parameters to QMF filter bank on the one hand, and mapping from spatial parameters to QMF band on the other hand.
For the left (/) channel, the HRTF parameters from the left-front and left-rear channels are combined into a single contribution, using the weights Ml/and wrf . The resulting composite parameters simulate the effect of both the front and rear channels in a statistical sense. The following equations are used to generate the binaural output pair (4, rt) for the left channel:

It is noted that the phase modification term is applied to the contra-lateral ear in both cases. Furthermore, since the human auditory system is largely insensitive to binaural phase for frequencies above approx. 2 kHz, the phase modification term only needs to be applied in the lower frequency region. Hence for the remainder of the frequency range, real-valued processing suffices (assuming real-valued mij).
It is further noted that the equations above assume incoherent addition of the (HRTF) filtered signals of 1f and Ir, One possible extension would be to include the transmitted Inter-Channel Coherence (ICC) parameters of 1f"and Ir (and of 1f"and rr) in the equations as well to account for front/rear correlation.
All processing steps described above can be combined in the parameter domain to result in a single, signal-domain 2x2 matrix:

The present invention essentially processes the binaural (that is, HRTF) information in the parameter domain, instead of in the fi-equency or time domain as in the Prior Art. In this way, significant computational savings may be obtained.
The spatial decoder device 2 according to the present invention shown merely by way of non-limiting example in Fig. 6 comprises a demultiplexer (Demux) unit 21, a downmix decoding unit 22, and a spatial/binaural decoder unit 23. The demultiplexer unit 21 and the downmix decoding unit 22 may be similar to the Prior Art demultiplexer unit 21' and

the downmix decoding unit 22'shown in Fig. 3. The spatial decoder unit 23 of Fig. 6 is identical to the spatial decoder unit 23 of Fig. 5, with the exception of the number of downmix channels and associated transform units. As the spatial decoder device of Fig. 6 is arranged for a single downmix channel s, only a single transform unit 231 is provided while a decorrelation (D) unit 239 has been added for producing a decorrelated version D of the (transform domain) downmix signal S. The binaural parameters bp produced by the parameter conversion unit 234 typically differ from those in the embodiment of Fig, 5, as the signal parameters sp associated with a single downmix channel s typically differ from those associated with two downmix chaimels.
In the configuration of Fig. 6, the input of the binaural decoder comprises a mono input signal s accompanied by spatial parameters sp. The binaural synthesis unit generates a stereo output signal with statistical properties that approximate those properties that would result from HRTF processing of the original 5.1 input, which can be described by:

Given the spatial parameters which describe statistical properties and inter¬relations of the channels If, rf Ir, rr and c, and the parameters of the HRTF impulse responses, it is possible to estimate the statistical properties (that is, an approximation of the binaural parameters) of the binaural output pair //,, Vb as well. More specifically, the average energy (for each channel), the average phase difference and the coherence can be estimated and subsequently re-instated by means of decorrelation and matrixing of the mono input signal.
The binaural parameters comprise a (relative) level change for each of the two binaural output channels (and hence define a Channel Level Difference parameter), an (average) phase difference and a coherence measure (per transform domain time/frequency tile).
As a first step, the relative powers (with respect to the power of the mono input signal) of the five (or six) channel (5.1) signal are computed using the transmitted CLD parameters. The relative power of the left-front channel is given by:

The expected value of the relative power cji of the left binaural output channel (with respect to the mono input channel), the expected value of the relative power Σ 7R of the right binaural output channel, and the expected value of the cross product LBRB can then be calculated. The coherence of the binaural output (ICCB) is then given by:

Finally, the overall (linear) gain of the binaural output compared to the mono input, gB, is given by:

The matrix coefficients required to re-instate the IPDB, CLDB, ICCB and gs parameters in the binaural matrix are simply obtained from a conventional parametric stereo decoder, extended with overall gains gn:

Further embodiments of the spatial decoder unit of the present invention may contain a reverberation unit. It has been found that adding reverberation improves the perceived distance when binaural sound is produced. For this reason, the spatial decoder unit 23 of Fig. 7 is provided with a stereo reverberation unit 237 connected in parallel with the spatial synthesis unit 232. The stereo reverberation unit 237 of Fig. 7 receives the QMF transform domain single downmix signal S and outputs two reverberation signals, which are

added to the transform domain binaural signals (channels Lb and Lr in Fig. 6) by addition units 238. The combined signals are then inversely transformed by inverse transform units 233 before being output.
In the embodiment of Fig. 8, the stereo reverberation unit 237 is arranged for producing a reverberation in the time domain and receives the time domain single downmix signal s. The stereo reverberation unit 237 outputs time domain reverberation signals, which are added to the time domain signals of the binaural channels lb and rb by the addition units 238. Either embodiment provides a suitable reverberation.
The present invention additionally provides a consumer device, such as a hand-held consumer device, and an audio system comprising a spatial decoder unit or spatial decoder device as defined above. The hand-held consumer device may be constituted by an MP3 player or similar device. A consumer device is schematically illustrated in Fig. 9. The consumer device 50 is shown to comprise a spatial decoder device 2 according to the present invention (see Fig. 6).
The present invention is based upon the insight that the computational complexity of a combined spatial decoder device and a binaural synthesis device may be significantly reduced by modifying the spatial parameters in accordance with the binaural information. This allows the spatial decoder device to carry out spatial decoding and perceptual transfer function processing effectively in the same signal processing operation, while avoiding the introduction of any artifacts.
It is noted that any terms used in this document should not be construed so as to limit the scope of the present invention. In particular, the words "comprise(s)" and "comprising" are not meant to exclude any elements not specifically stated. Single (circuit) elements may be substituted with multiple (circuit) elements or with their equivalents.
It will be understood by those skilled in the art that the present invention is not limited to the embodiments illustrated above and that many modifications and additions may be made without departing from the scope of the invention as defined in the appending claims.

CLAIMS:
1. A spatial decoder unit (23) for producing a pair of binaural output channels
(lb, rb) using spatial parameters (sp) and a single audio input channel (s), the device
comprising:
a parameter conversion unit (234) for converting the spatial parameters (sp) into binaural parameters (bp) using parametrized perceptual transfer functions (hp), the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions;
a single transform unit (231) for transforming the single audio input channel (s) into a transformed audio channel (S);
a decorrelation unit (239) for decorrelating the transformed audio channel (S) to generate a transformed decorrelated signal (D);
a spatial synthesis unit (232) for synthesizing a pair of transformed binaural channels (Lb, Rb) by applying the binaural parameters (bp) to the transformed audio channel (S) and the transformed decorrelated signal (D); and
a pair of inverse transform units (233) for inversely transforming the transformed binaural channels (Lb, Rb) into the pair of binaural output channels (lb, rb).
2. The spatial decoder unit according to claim 1, wherein the parameter conversion unit (234) is arranged for combining in the parameter domain, in order to determine the binaural parameters, all perceptual transfer function contributions the audio input channels would make to the binaural channels.
3. The spatial decoder unit according to claim 1, wherein the parameter conversion unit (234) is arranged for processing channel level (CLD), channel coherence (ICC) and/or phase (IPD) parameters.
4. The spatial decoder unit according to claim 1, further comprising a stereo reverberation unit (237) arranged for operating in the time domain.

5. The spatial decoder unit according to claim 1, further compnsmg a stereo reverberation unit (237) arranged for operating in a transform domain or sub-band domain, such as the QMF domain.
6. The spatial decoder unit according to claim 1 wherein the parameter conversion unit (234) comprises:
means for determining relative powers for a plurality of virtual audio channels in response to the spatial parameters (sp) and the single audio input channel (s);
means for estimating statistical properties of the binaural output chaxmels (lb, rb) in response to the relative powers for the plurality of virtual audio channels; and
means for determining the binaural parameters (bp) in response to the estimated statistical properties of the binaural output channels (lb, rb).
7. The spatial decoder unit according to claim 6 wherein the statistical properties comprise an average energy of the transformed binaural channels (Lb, Rb), an average phase difference of the transformed binaural channels (Lb, Rb) and a coherence of the transformed binaural channels (Lb, Rb).
8. A spatial decoder device (2) for producing a pair of binaural output channels (lb, rb) from an input bitstream (bs), the device comprising:
a demultiplexer unit (21) for demultiplexing the input bitstream into a single downmix channel and signal parameters (sp),
a downmix decoder unit (22) for decoding the single downmix channel (s),
and
a spatial decoder unit (23) according to claim 1.
9. The device according to claim 8, wherein the spatial decoder unit (23) comprises a reverberation unit (237).
10. An audio system, comprising a spatial decoder unit (23) according to claim 1 and/or a spatial decoder device (2) according to claim 8.
11. A consumer device, comprising a spatial decoder unit (23) according to claim 1 and/or a spatial decoder device (2) according to claim 8.

12. A method of producing a pair of binaural output channels (lb, rb) using spatial
parameters (sp) and a single audio input channel (s), the method comprising the steps of:
converting the spatial parameters (sp) into binaural parameters (bp) using parameterized perceptual transfer functions (hp), the binaural parameters depending on both the spatial parameters and the parametrized perceptual transfer functions;
transforming the single audio input channel (s) into a transformed audio channel (S);
decorrelating the transformed audio channel (S) to generate a transformed decorrelated signal (D);
synthesizing a pair of transformed binaural channels (Lb, Rb) by applying the binaural parameters (bp) to the transformed audio channel (S) and the transformed decorrelated signal (D);and
inversely transforming the transformed binaural channels (Lb, Rb) into the pair of binaural output channels (lb, rb).
13. A computer program product for carrying out the method according to
claim 12.

Documents:

http://ipindiaonline.gov.in/patentsearch/GrantedSearch/viewdoc.aspx?id=nTfEAQ2RfxksJKBR366Mbw==&loc=egcICQiyoj82NGgGrC5ChA==

« Previous Patent

Next Patent »

Patent Number

269121

Indian Patent Application Number

1240/CHENP/2008

PG Journal Number

41/2015

Publication Date

09-Oct-2015

Grant Date

30-Sep-2015

Date of Filing

12-Mar-2008

Name of Patentee

KONINKLIJKE PHILIPS ELECTRONICS N.V.

Applicant Address

GROENEWOUDSEWEG 1, NL-5621 BA EINDHOVEN, THE NETHERLANDS

Inventors:

#	Inventor's Name	Inventor's Address
1	BREEBAART, DIRK, J	C/O PROF. HOLSTLAAN 6, NL-5656 AA EINDHOVEN, THE NETHERLANDS

PCT International Classification Number

H04S3/00

PCT International Application Number

PCT/IB06/53040

PCT International Filing date

2006-08-31

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1	05108405.1	2005-09-13	EUROPEAN UNION
2	06110231.5	2006-02-21	EUROPEAN UNION