Title of Invention	BINAURAL MULTI-CHANNEL DECODER IN THE CONTEXT OF NON-ENERGY-CONSERVING UPMIX RULES
Abstract	A multi-channel decoder for generating a binaural signal from a downmix signal using upmix rule information on an energy-error introducing upmix rule for calculating a gain factor (180) based on the upmix rule information and characteristics of head related transfer function based filters corresponding to upmix channels. The one or more gain factors are used by a filter processor (182) for filtering the downmix signal so that an energy corrected binaural signal having a left binaural channel and a right binaural channel is obtained.

Title of Invention

BINAURAL MULTI-CHANNEL DECODER IN THE CONTEXT OF NON-ENERGY-CONSERVING UPMIX RULES

Abstract

A multi-channel decoder for generating a binaural signal from a downmix signal using upmix rule information on an energy-error introducing upmix rule for calculating a gain factor (180) based on the upmix rule information and characteristics of head related transfer function based filters corresponding to upmix channels. The one or more gain factors are used by a filter processor (182) for filtering the downmix signal so that an energy corrected binaural signal having a left binaural channel and a right binaural channel is obtained.

Full Text	Field of the Invention The present invention relates to binaural decoding of multi-channel audio signals based on available .downmixed signals and additional control data, by means of HRTF filtering. Background of the Invention and Prior Art Recent development in audio coding has made methods available to recreate a multi-channel representation of an audio signal based on a stereo (or mono) signal and corresponding control data. These methods differ substantially from older matrix based solution such as Dolby Prologic, since additional control data is transmitted to control the re-creation, also referred to as up-mix, of the surround channels based on the transmitted mono or stereo channels. Hence, such a parametric multi-channel audio decoder, e.g. MPEG Surround reconstructs Nchannels based on M transmitted channels, where N>M, and the additional control data. The additional control data represents a significantly lower data rate than that required for transmission of all N channels, making the coding very efficient while at the same time ensuring compatibility with both M channel devices and N channel devices. [J. Breebaart et al. "MPEG spatial audio coding / MPEG Surround: overview and current status", Proc. 119th AES convention, New York, USA, October 2005, Preprint 6447]. These parametric surround coding methods usually comprise a parameterization of the surround signal based on Channel Level Difference (CLD) and Inter-channel coherence/cross- correlation (ICC). These parameters describe power ratios and correlation between channel pairs in the up-mix process. Further Channel Prediction Coefficients (CPC) are also used in prior art to predict intermediate or output channels during the up-mix procedure. Other developments in audio coding have provided means to obtain a multi-channel signal impression over stereo headphones. This is commonly done by downmixing a multi- channel signal to stereo using the original multi-channel signal and HRTF (Head Related Transfer Functions) filters. Alternatively, it would, of course, be useful for computational efficiency reasons and also for audio quality reasons to short-cut the generation of the binaural signal having the left binaural channel and the right binaural channel. However, the question is how the original HRTF filters can be combined. Further a problem arises in a context of an energy-loss-affected upmixing rule, i.e., when the multi- channel decoder input signal includes a downmix signal having, for example, a first downmix channel and a second downmix channel, and further having spatial parameters, which are used for upmixing in a non-energy-conserving way. Such parameters are also known as prediction parameters or CPC parameters. These parameters have, in contrast to channel level difference parameters the property that they are not calculated to reflect the energy distribution between two channels, but they are calculated for performing a best-as-possible waveform matching which automatically results in an energy error (e.g. loss), since, when the prediction parameters are generated, one does not care about energy-conserving properties of an upmix, but one does care about having a good as possible time or subband domain waveform matching of the reconstructed signal compared to the original signal. When one would simply linearly combine HRTF filters based on such transmitted spatial prediction parameters, one will receive artifacts which are especially serious, when the prediction of the channels performs poorly. In that situation, even subtle linear dependencies lead to undesired spectral coloring of the binaural output. It has been found out that this artifact occurs most frequently when the original channels carry signals that are pairwise uncorrelated and have comparable magnitudes. Summary of the Invention It is the object of the present invention to provide an efficient and qualitatively acceptable concept for multi- channel decoding to obtain a binaural signal which can be used, for example, for headphone reproduction of a multi- channel signal. In accordance with the first aspect of the present invention, this object is achieved by a multi-channel decoder for generating a binaural signal from a downmix signal derived from an original multi-channel signal using parameters including an upmix rule information useable for upmixing the downmix signal with an upmix rule, the upmix rule resulting in an energy-error, comprising: a gain factor calculator for calculating at least one gain factor for reducing or eliminating the energy-error, based on the upmix rule information and filter characteristics of a head related transfer function based filters corresponding to upmix channels, and a filter processor for filtering the downmix signal using the at least one gain factor, the filter characteristics and the upmix rule information to obtain an energy-corrected binaural signal. In accordance with a second aspect of this invention, this object is achieved by a method of multi-channel decoding for generating a binaural signal from a downmix signal derived from an original multi-channel signal using parameters including an upmix rule information useable for upmixing the downmix signal with an upmix rule, the upmix rule resulting in an energy-error, comprising: calculating at least one gain factor for deducing or eliminating the energy-error, based on the upmix rule information and filter characteristics of a head related transfer function based filters corresponding to upmix channels; and filtering the downmix signal using the at least one gain factor, the filter characteristics and the upmix rule information to obtain an energy-corrected binaural signal. Further aspects of this invention relate to a computer program having a computer-readable code which implements, when running on a computer, the method of multi-channel decoding. The present invention is based on the finding that one can even advantageously use up-mix rule information on an upmix resulting in an energy error for filtering a downmix signal to obtain a binaural signal without having to fully render the multichannel signal and to subsequently apply a huge number of HRTF filters. Instead, in accordance with the present invention, the upmix rule information relating to an energy-error-affected upmix rule can advantageously be used for short-cutting binaural rendering of a downmix signal, when, in accordance with the present invention, a gain factor is calculated and used when filtering the downmix signal, wherein this gain factor is calculated such that the energy error is reduced or completely eliminated. Particularly, the gain factor not only depends on the information on the upmix rule such as the prediction parameters, but, importantly, also depends on head related transfer function based filters corresponding to upmix channels, for which the upmix rule is given. Particularly, these upmix channels never exist in the preferred embodiment of the present invention, since the binaural channels are calculated without firstly rendering, for example, three intermediate channels. However, one can derive or provide HRTF based filters corresponding to the upmix channels although the upmix channels themselves never exist in the preferred embodiment. It has been found out that the energy error introduced by such an energy-loss- affected upmix rule not only corresponds to the upmix rule information which is transmitted from the encoder to the decoder, but also depends on the HRTF based filters so that, when generating the gain factor, the HRTF based filters also influence the calculation of the gain factor. In view of that, the present invention accounts for the interdependence between upmix rule information such as prediction parameters and the specific appearance of the HRTF based filters for the channels which would be the result of upmixing using the upmix rule. Thus, the present invention provides a solution to the problem of spectral coloring arising from the usage of a predictive upmix in combination with binaural decoding of parametric multi-channel audio. Preferred embodiments of the present invention comprise the following features: an audio decoder for generating a binaural audio signal from M decoded signals and spatial parameters pertinent to the creation of N>M channels, the decoder comprising a gain calculator for estimating, in a multitude of subbands, two compensation gains from P pairs of binaural subband filters and a subset of the spatial parameters pertinent to the creation of P intermediate channels, and a gain adjuster for modifying, in a multitude of subbands, M pairs of binaural subband filters obtained by linear combination of the P pairs of binaural subband filters, the modification consisting of multiplying each of the M pairs with the two gains computed by the gain calculator. Brief Description of the Drawings The present invention will now be described by way of illustrative examples, not limiting the scope or spirit of the invention, with reference to the accompanying drawings, in which: Fig. 1 illustrates binaural synthesis of parametric multichannel signals using HRTF related filters; Fig. 2 illustrates binaural synthesis of parametric multichannel signals using combined filtering; Fig. 3 illustrates the components of the inventive parameter/filter combiner; Fig. 4 illustrates the structure of MPEG Surround spatial decoding; Fig. 5 illustrates the spectrum of a decoded binaural signal without the inventive gain compensation; Fig. 6 illustrates the spectrum of the inventive decoding of a binaural signal. Fig. 7 illustrates a conventional binaural synthesis using HRTFs; Fig. 8 illustrates a MPEG surround encoder; Fig. 9 illustrates cascade of MPEG surround decoder and binaural synthesizer; Fig. 10 illustrates a conceptual 3D binaural decoder for certain configurations; Fig. 11 illustrates a spatial encoder for certain configurations; Fig. 12 illustrates a spatial (MPEG Surround) decoder; Fig. 13 illustrates filtering of two downmix channels using four filters to obtain binaural signals without gain factor correction; Fig. 14 illustrates a spatial setup for explaining different HRTF filters 1-10 in a five channels setup; Fig. 15 illustrates a situation of Fig. 14, when the channels for L, Ls and R, Rs have been combined; Fig. 16a illustrates the setup from Fig. 14 or Fig. 15, when a maximum combination of HRTF filters has been performed and only the four filters of Fig. 13 remain; Fig. 16b illustrates an upmix rule as determined by the Fig. 20 encoder having upmix coefficients resulting in a non-energy-conserving upmix; Fig. 17 illustrates how HRTF filters are combined to finally obtain four HRTF-based filters; Fig. 18 illustrates a preferred embodiment of an inventive multi-channel decoder; Fig. 19a illustrates a first embodiment of the inventive multi-channel decoder having a scaling stage after HRTF-based filtering without gain correction; Fig. 19b illustrates an inventive device having adjusted HRTF-based filters which result in a gain- adjusted filter output signal; and Fig. 20 shows an example for an encoder generating the information for a non-energy-conserving upmix rule. Detailed description of preferred embodiments Before discussing the inventive gain adjusting aspect in detail, a combination of HRTF filters and usage of HRTF- based filters will be discussed in connection with Figs. 7 to 11. In order to better outline the features and advantages of the present invention a more elaborate description is given first. A binaural synthesis algorithm is outlined in Fig. 7. A set of input channels is filtered by a set of HRTFs. Each input signal is split in two signals (a left 'L' , and a right 'R' component); each of these signals is subsequently filtered by an HRTF corresponding to the desired sound source position. All left-ear signals are subsequently summed to generate the left binaural output signal, and the right-ear signals are summed to generate the right binaural output signal. The HRTF convolution can be performed in the time domain, but it is often preferred to perform the filtering in the frequency domain due to computational efficiency. In that case, the summation as shown in Fig. 7 is also performed in the frequency domain. In principle, the binaural synthesis method as outlined in Fig. 7 could be directly used in combination with an MPEG surround encoder/decoder. The MPEG surround encoder is schematically shown in Fig. 8. A multi-channel input signal is analyzed by a spatial encoder, resulting in a mono or stereo down mix signal, combined with spatial parameters. The down mix can be encoded with any conventional mono or stereo audio codec. The resulting down-mix bit stream is combined with the spatial parameters by a multiplexer, resulting in the total output bit stream. A binaural synthesis scheme in combination with an MPEG surround decoder is shown in Fig. 9. The input bit stream is de-multiplexed resulting in spatial parameters and a down-mix bit stream. The latter bit stream is decoded using a conventional mono or stereo decoder. The decoded down mix is decoded by a spatial decoder, which generates a multi- channel output based on the transmitted spatial parameters. Finally, the multi-channel output is processed by a binaural synthesis stage as depicted in Fig. 7, resulting in a binaural output signal. There are however at least three disadvantages of such a cascade of an MPEG surround decoder and a binaural synthesis module: • A multi-channel signal representation is computed as an intermediate step, followed by HRTF convolution and downmixing in the binaural synthesis step. Although HRTF convolution should be performed on a per channel basis, given the fact that each audio channel can have a different spatial position, this is an undesirable situation from a complexity point of view. • The spatial decoder operates in a filterbank (QMF) domain. HRTF convolution, on the other hand, is typically applied in the FFT domain. Therefore, a cascade of a multi-channel QMF synthesis filterbank, a multi-channel DFT transform, and a stereo inverse DFT transform is necessary, resulting in a system with high computational demands. • Coding artifacts created by the spatial decoder to create a multi-channel reconstruction will be audible, and possibly enhanced in the (stereo) binaural output. The spatial encoder is shown in Fig. 11. A multi-channel input signal consisting of Lf, Ls, C, Rf and Rs signals, for the left-front, left-surround, center, right-front and right-surround channels is processed by two 'OTI" units, which both generate a mono down mix and parameters for two input signals. The resulting down-mix signals, combined with the center channel are further processed by a TTT' (Two-To-Three) encoder, generating a stereo down mix and additional spatial parameters. The parameters resulting from the 'TTT' encoder typically consist of a pair of prediction coefficients for each parameter band, or a pair of level differences to describe the energy ratios of the three input signals. The parameters of the 'OTT' encoders consist of level differences and coherence or cross-correlation values between the input signals for each frequency band. In Fig. 12 a MPEG Surround decoder is depicted. The downmix signals 10 and rO are input into a Two-To-Three module, that recreates a center channel, a right side channel and a left side channel. These three channels are further processed by several OTT modules (One-To-Two) yielding the six output channels. The corresponding binaural decoder as seen from a conceptual point of view is shown in Fig. 10. Within the filterbank domain, the stereo input signal {L0, Ro) is processed by a TTT decoder, resulting in three signals L, R and C. These three signals are subject to HRTF parameter processing. The resulting 6 channels are summed to generate the stereo binaural output pair {Lb, Rb) . The TTT decoder can be described as the following matrix operation: with matrix entries mxy dependent on the spatial parameters. The relation of spatial parameters and matrix entries is identical to those relations as in the 5.1- multichannel MPEG surround decoder. Each of the three resulting signals L, R, and C are split in two and processed with HRTF parameters corresponding to the desired (perceived) position of these sound sources. For the center channel (C) , the spatial parameters of the sound source position can be applied directly, resulting in two output signals for center, LB(C) and RB(C): For the left (L) channel, the HRTF parameters from the left-front and left-surround channels are combined into a single HRTF parameter set, using the weights wlf and wrf . The resulting 'composite' HRTF parameters simulate the effect of both the front and surround channels in a statistical sense. The following equations are used to generate the binaural output pair (LB, RB) for the left channel: In a similar fashion, the binaural output for the right channel is obtained according to: Given the above definitions of LB(C), RB(C), LB{L), RB(L), LB(R) and RB(R), the complete LB and RB signals can be derived from a single 2 by 2 matrix given the stereo input signal: with The Hx(Y) filters can be expressed as parametric weighted combinations of parametric versions of the original HRTF filters. In order for this to work, the original HRTF filters are expressed as a • An (average) level per frequency band for the left-ear impulse response; • An (average) level per frequency band for the right- ear impulse response; • An (average) arrival time or phase difference between the left-ear and right-ear impulse response. Hence, the HRTF filters for the left and right ear given the center channel input signal is expressed as: where P,(C) is the average level for a given frequency band for the left ear, and Ø(c)is the phase difference. Hence, the HRTF parameter processing simply consists of a multiplication of the signal with P1and Pr corresponding to the sound source position of the center channel, while the phase difference is distributed symmetrically. This process is performed independently for each QMF band, using the mapping from HRTF parameters to QMF filterbank on the one hand, and mapping from spatial parameters to QMF band on the other hand. Similarly the HRTF filters for the left and right ear given the left channel and right channel are given by: Clearly, the HRTFs are weighted combinations of the levels and phase differences for the parameterized HRTF filters for the six original channels. The weights Wif and Wjs depend on the CLD parameter of the 'OTT' box for Lf and Ls: And the weights wrf and Wrs depend on the CLD parameter of the 'OTT" box for Rf and Rs: The above approach works well for short HRTF filters that sufficiently accurate can be expressed as an average level per frequency band, and an average phase difference per frequency band. However, for long echoic HRTFs this is not the case. The present invention teaches how to extend the approach of a 2 by 2 matrix binaural decoder to handle arbitrary length HRTF filters. In order to achieve this, the present invention comprises the following steps: • Transform the HRTF filter responses to a filterbank domain; • Overall delay difference or phase difference extraction from HRTF filter pairs; •• Morph the responses of the HRTF filter pair as a function of the CLD parameters • Gain adjustment This is achieved by replacing the six complex gainsHY(X) for Y = L0, R0and X = L,R,C with six filters. These filters are derived from the ten filters HY(X) for Y = L0,R0 and X=Lf,Ls,Rf,Rs,C , which describe the given HRTF filter responses in the QMF domain. These QMF representations can be achieved according to the method described below. The morphing of the front and surround channel filters is performed with a complex linear combination according to The phase parameter Øxy can be defined from the main delay time difference rxybetween the front and back HRTF filters and the subband index n of the QMF bank via The role of this phase parameter in the morphing of filters is twofold. First, it realizes a delay compensation of the two filters prior to superposition which leads to a combined response which models a main delay time corresponding to a source position between the front and the back speakers. Second, it makes the necessary gain compensation factor gmuch more stable and slowly varying over frequency than in the case of simple superposition with Øxy=0. The gain factor g is determined by the same incoherent addition power rule as for the parametric HRTF case, andPxy is the real value of the normalized complex cross correlation between the filters In the case of simple superposition withØxy=0, the value of Pxy varies in an erratic and oscillatory manner as a function of frequency, which leads to the need for extensive gain adjustment. In practical implementation it is necessary to limit the value of the gaingand a remaining spectral colorization of the signal cannot be avoided. In contrast, the use of morphing with a delay based phase compensation as taught by the present invention leads to a smooth behavior of pxy as a function of frequency. This value is often even close to one for natural HRTF derived filter pairs since they differ mainly in a delay and amplitude, and the purpose of the phase parameter is to take the delay difference into account in the QMF filterbank domain. An alternative beneficial choice of phase parameter Øxy is given by computing the phase angle of the normalized complex cross correlation between the filters Hy(Xf) and HY(Xs)f and unwrapping the phase values with standard unwrapping techniques as a function of the subband index n of the QMF bank. This choice has the consequence that Pxy is never negative and hence the compensation gain ^satisfies for all subbands. Moreover this choice of phase parameter enables the morphing of the front and surround channel filters in situations where a main delay time difference txy is not available. All signals considered below are subband samples from a modulated filter bank or windowed FFT analysis of discrete time signals or discrete time signals. It is understood that these subbands have to be transformed back to the discrete time domain by corresponding synthesis filter bank operations. Fig. 1 illustrates a procedure for binaural synthesis of parametric multichannel signals using HRTF related filters. A multichannel signal comprising N channels is produced by spatial decoding 101 based on M and transmitted spatial parameters. These TV channels are in turn converted into two output channels intended for binaural listening by means of HRTF filtering. This HRTF filtering 102 superimposes the results of filtering each input channel with one HRTF filter for the left ear and one HRTF filter for the right ear. All in all, this requires 2N filters. Whereas the parametric multichannel signal achieves a high quality listener experience when listened to through N loudspeakers, subtle interdependencies of the N signals will lead to artifacts for the binaural listening. These artifacts are dominated by deviation in spectral content from the reference binaural signal as defined by HRTF filtering of the original N channels prior to coding. A further disadvantage of this concatenation is that the total computational cost for binaural synthesis is the addition of the cost required for each of the components 101 and 102. Fig. 2 illustrates binaural synthesis of parametric multichannel signals by using the combined filtering taught by the present invention. The transmitted spatial parameters are split by 201 into two sets, Set 1 and Set 2. Here, Set 2 comprises parameters pertinent to the creation of P intermediate channels from the M transmitted channels and Set 1 comprises parameters pertinent to the creation of N channels from the P intermediate channels. The prior art precombiner 202 combines selected pairs of the 2N HRTF related subband filters with weights that depend the parameter Set 1 and the selected pairs of filters. The result of this precombination is 2P binaural subband filters which represent a binaural filter pair for each of the P intermediate channels. The inventive combiner 203 combines the 2P binaural subband filters into a set of 2Mbinaural subband filters by applying weights that depend both on the parameter Set 2 and the 2P binaural subband filters. In comparison, a prior art linear combiner would apply weights that depend only on the parameter Set 2. The resulting set of 2M filters consists of a binaural filter pair for each of the Mtransmitted channels. The combined filtering unit 204 obtains a pair of contributions to the two channel output for each of the Mtransmitted channels by filtering with the corresponding filter pair. Subsequently, all the Mcontributions are added up to form a two channel output in the subband domain. Fig. 3 illustrates the components of the inventive combiner 203 for combination of spatial parameters and binaural filters. The linear combiner 301 combines the 2P binaural subband filters into 2M binaural filters by applying weights that are derived from the given spatial parameters, where these spatial parameters are pertinent to the creation of P intermediate channels from the Mtransmitted channels. Specifically, this linear combination simulates the concatenation of an upmix from Mtransmitted channels to P intermediate channels followed by a binaural filtering from P sources. The gain adjuster 303 modifies the 2M binaural filters output from the linear combiner 301 by applying a common left gain to each of the filters that correspond to the left ear output and by applying a common right gain to each of the filters that correspond to the right ear output. Those gains are obtained from gain calculator 302 which derives the gains from the spatial parameters and the 2P binaural filters. The purpose of the gain adjustment of the inventive components 302 and 303 is to compensate for the situation where theP intermediate channels of the spatial decoding carry linear dependencies that lead to unwanted spectral coloring due to the linear combiner 301. The gain calculator 302 taught by the present invention includes means for estimating an energy distribution of the P intermediate channels as a function of the spatial parameters. Fig. 4 illustrates the structure of MPEG Surround spatial decoding in the case of a stereo transmitted signal. The analysis subbands of the M = 2transmitted signals are fed into the 2—>3 box 401 which outputs P = 3 intermediate signals, a combined left, a combined right, and a combined center. This upmix depends on a subset of the transmitted spatial parameters which corresponds to Set 2 on Fig. 2. The three intermediate signals are subsequently fed into three l->2 boxes 402-404 which generate a totality ofN=6 signals 405: lf (left front), ls (left surround), rf (right front) , rs (right surround) , c (center) , and Ife (low frequency extension). This upmix depends on a subset of the transmitted spatial parameters which corresponds to Set 1 on Fig. 2. The final multichannel digital audio output is created by passing the six subband signals into six synthesis filter banks. Fig. 5 illustrates the problem to be solved by the inventive gain compensation. The spectrum of a reference HRTF filtered binaural output for the left ear is depicted as a solid graph. The dashed graph depicts the spectrum of the corresponding decoded signal as generated by the method of Fig. 2, in the case where the combiner 203 consists of the linear combiner 301 only. As it can be seen, there is a substantial spectral energy loss relative to the desired reference spectrum in the frequency intervals 3-4 kHz and 11-13 kHz. There is also a smaller spectral boost around 1 kHz and 10 kHz. Fig. 6 illustrates the benefit of using the inventive gain compensation. The solid graph is the same reference spectrum as in Fig. 5, but now the dashed graph depicts the spectrum of the decoded signal as generated by the method of Fig. 2, in the case where the combiner 203 consists of all the components of Fig. 3. As it can be seen, there is a significantly improved spectral match between the two curves compared to that of the two curves of Fig. 5. In the text which follows, the mathematical description of the inventive gain compensation will be outlined. For discrete complex signals x,y, the complex inner product and squared norm (energy) is defined by where y(k) denotes the complex conjugate signal of y(k). The original multichannel signal consists of TV channels, and each channel has a binaural HRTF related filter pair associated to it. It will however be assumed here that the parametric multichannel signal is created with an intermediate step of predictive upmix from the Mtransmitted channels to P predicted channels. This structure is used in MPEG Surround as described by Fig. 4. It will be assumed that the original set of IN HRTF related filters have been reduced by the prior art precombiner 202 to a filter pair for each of the P predicted channels where M channel signals xp, p = l,2,K ,P, aim at approximating theP signals xp, p = l,2,K ,P, which are derived from the original N channels via partial downmix. In MPEG Surround, these signals are a combined left, a combined right and a combined and scaled center/Ife channel. It is assumed that the HRTF filter pair corresponding to the signal xp is described by a subband filter blp for the left ear and a subband filter b2pfor the right ear. The reference binaural output signal is thus given by the linear superposition of filtered signals for n = l,2, where the star denotes convolution in the time direction. The subband filters can be given in form of finite impulse response (FIR) filters, infinite impulse response (IIR) or derived from a parameterized family of filters. In the encoder, the downmix is formed by the application of a MxP downmix matrix D to a column vector of signals formed by x p = 1,2,K ,Pand the prediction in the decoder is performed by the application of a PxM prediction matrix C to the column vector of signals formed by the M transmitted downmixed channels zm m = l,K ,M, Both matrices are known at the decoder, and ignoring the effects of coding the downmixed channels, the combined effect of prediction can be modeled by where a are the entries of the matrix product A = CD. A straightforward method for producing a binaural output at the decoder is to simply insert the predicted signals xp in (2) resulting in In terms of computations, the binaural filtering is combined with the predictive upmix beforehand such that (5) can be written as with the combined filters defined by This formula describes the action of the linear combiner 301 which combines the coefficients c„m derived from p,tn spatial parameters with the binaural subband domain filters bnp . When the originalP signals^ have a numerical rank essentially bounded by M, the prediction can be designed to perform very well and the approximation jc «JC is valid. This happens for instance if only M of the P channels are active, or if important signal components originate from amplitude panning. In that case the decoded binaural signal (5) is a very good match to the reference (2). On the other hand, in the general case and especially in case the original P signals xp are uncorrelated, there will be a substantial prediction loss and the output from (5) can have an energy that deviates considerably from the energy of (2). As the deviation will be different in different frequency bands, the final audio output suffers from spectral coloring artifacts as described by Fig. 5. The present invention teaches how to circumvent this problem by gain compensating the output according to In terms of computations, the gain compensation is advantageously performed by altering the combined filters according to the gain adjuster 303 , ^(0m(k) = gnhn^(k) . The modified combined filtering then becomes The optimal values of the compensating gains in (8) are The purpose of the gain calculator 302 is to estimate these gains from the information available in the decoder. Several tools for this end will now be outlined. The available information is represented here by the matrix entries apq and the HRTF related subband filters bnp. First, the following approximation will be assumed for the inner product between signals jc^that have been filtered by HRTF related subband filters b,d, This approximation relies on the fact that often most energy of the filters is concentrated in a dominant single tap, which in turn presupposes that the time step of the applied time frequency transform is sufficiently large in comparison to the main delay differences of HRTF filters. Applying the approximation (11) in combination with (2) leads to The next approximation consists of assuming that the original signals are uncorrelated, that is (x ,x\ = 0 for pq. Then (12) reduces to For the decoded energy the result corresponding to (12) is Inserting the predicted signals (4) in (14) and applying the assumption that the original signals are uncorrelated gives What remains in order to be able to calculate the compensation gain given by the quotient (10) is to estimate the energy distribution pi , p = l,2,K ,Pof the original channels up to an arbitrary factor. The present invention teaches to do this by computing, as a function of the energy distribution, the prediction matrix Cmodel corresponding to the assumption that these channels are uncorrelated and that the encoder aims at minimizing the prediction error. The energy distribution is then estimated by solving the nonlinear system of equations Cmodel =C if possible. For prediction parameters that lead to a system of equations without solutions, the gain compensation factors are set to gn=1. This inventive procedure will be detailed in the following section in the most important special case. The computation load imposed by (15) can be reduced in the case where P = M + l by applying the expansion (see for instance PCT/EP2005/011586), where vis a unit vector with components vp such that Dv = 0, and AE is the prediction loss energy, The computation of (15) is then advantageously replaced by the application of (16) in (14), leading to Subsequently, a preferred specialization to prediction of three channels from two channels will be discussed. The case where M = 2and P = 3 is used in MPEG Surround. The signals are a combined left x1=/, a combined right x2=rand a (scaled) combined center/lfe channel x3=c. The downmix matrix is and the prediction matrix is constructed from two transmitted real parameters c1,c2, according to Under the assumption that the original channels are uncorrelated the prediction matrix realizing the minimal prediction error is given by Equating Cmodel=C leads to the (unnormalized) energy distribution taught by the present invention where or = (l-c,)/3, /? = (l-c2)/3, a = a + jd, and p = afi. This holds in the viable range defined by in which case the prediction error can be found in the same scaling from Since P = 3 = 2 + l = M + l, the method outlined by (16)-(18) is applicable. The unit vector is [v,,v2,v3] = [l,l,-l]/v3 and with the definitions the compensation gain for each ear « = 1,2 as computed in a preferred embodiment of the gain calculator 302 can be expressed by Here e>0 is a small number whose purpose is to stabilize the formula near the edge of the viable parameter range and gmax is an upper limit on the applied compensation gain. The gains of (27) are different for the left and right ears, « = 1,2. A variant of the method is to use a common gain go=gl=g, where The inventive correction gain factor can be brought into coexistence with a straight-forward multichannel gain compensation available without any HRTF related issues. In MPEG Surround, compensation for the prediction loss is already applied in the decoder by multiplying the upmix matrix C by a factor 1/p where 0 transmitted spatial parameters. In that case the gains of (27) and (28) have to be replaced by the products pgn and pgrespectively. Such compensation is applied for the binaural decoding studied in Figs. 5 and 6. It is the reason why the prior art decoding of Fig. 5 has boosted parts of the spectrum in comparison to the reference. For the subbands corresponding to those frequency regions, the inventive gain compensation effectively replaces the transmitted parameter gain factor l//?with a smaller value derived from formula (28). In addition, since the case where p = \ corresponds to a successful prediction, a more conservative variant of the gain compensation taught by the present invention will disable the binaural gain compensation forp=l. Furthermore, the present invention is used together with a residual signal. In MPEG Surround, an additional prediction residual signal z3 can be transmitted which makes it possible to reproduce the original P=3 signals xpmore faithfully. In this case the gain compensation is to be replaced by a binaural residual signal addition which will now be outlined. The predictive upmix enhanced by a residual is formed according to where [w„w2,vc3] = [l,l,-l]/3. Substituting #£ for xp in (5) yields the corresponding combined filtering, where the combined filters hn,mare defined by (7) for /w = l,2, and the combined filters for the residual addition are defined by The overall structure of this mode of decoding is therefore also described by Fig. 2 by setting P = M = 3, and by modifying the combiner 203 to perform only the linear combination defined by (7) and (31). Fig. 13 illustrates in a modified representation the result of the linear combiner 301 in Fig. 3. The result of the combiner are four HRTF-based filters hn, hi2, h2i and h22. As will be clearer from the description of Fig. 16a and Fig. 17, these filters correspond to filters indicated by 15, 16, 17 ,18 in Fig. 16a. Fig. 16a shows a head of a listener having a left ear or a left binaural point and having a right ear or a right binaural point. When Fig. 16a would only correspond to a stereo scenario, then filters 15, 16, 17, 18 would be typical head related transfer functions which can be individually measured or obtained via the Internet or in corresponding textbooks for different positions between a listener and the left channel speaker and the right channel speaker. However, since the present invention is directed to a multi-channel binaural decoder, filters illustrated by 15, 16, 17, 18 are not pure HRTF filters, but are HRTF-based filters, which not only reflect HRTF properties but which also depend on the spatial parameters and, particularly, as discussed in connection with Fig. 2, depend on the spatial parameter set 1 and the spatial parameter set 2. Fig. 14 shows the basis for the HRTF-based filters used in Fig. 16a. Particularly, a situation is illustrated where a listener is positioned in a sweet spot between five speakers in a five channel speaker setup which can be found, for example, in typical surround home or cinema entertainment systems. For each channel, there exist two HRTFs which can be converted to channel impulse responses of a filter having the HRTF as the transfer function. Particularly as it is known in the art, an HRTF-based filter accounts for the sound propagation within the head of a person so that, for example, HRTF1 in Fig. 14 accounts for the situation that a sound emitted from speaker Ls meets the right ear after having passed around the head of the listener. Contrary thereto, the sound emitted from the left surround speaker Ls meets the left ear almost directly and is only partly affected by the position of the ear at the head and also the shape of the ear etc. Thus, it becomes clear that the HRTFs 1 and 2 are different from each other. The same is true for the HRTFs 3 and 4 for the left channel, since the relations of both ears to the left channel L are different. This also applies for all other HRTFs, although as becomes clear from Fig. 14, the HRTFs 5 and 6 for the center channel will be almost identical or even completely identical to each other, unless the individual listeners asymmetry is accommodated by the HRTF data. As stated above, these HRTFs have been determined for model heads and can be downloaded for any specific "average head", and loudspeaker setup. Now, as becomes clear at 171 and 172 in Fig. 17, a combination takes place to combine the left channel and the left surround channel to obtain two HRTF-based filters for the left side indicated by L' in Fig. 15. The same procedure is performed for the right side as illustrated by R' in Fig. 15 which results in HRTF 13 and HRTF 14. To this end, reference is also made to item 173 and item 174 in Fig. 17. However, it is to be noted here that, for combining respective HRTFs in items 171, 172, 173 and 174, inter channel level difference parameters reflecting the energy distribution between the L channel and the Ls channel of the original setup or between the R channel and the Rs channel of the original multi-channel setup are accounted for. Particularly, these parameters define a weighting factor when HRTFs are linearly combined. As outlined before, a phase factor can also be applied when combining HRTFs, which phase factor is defined by time delays or unwrapped phase differences between the to be combined HRTFs. However, this phase factor does not depend on the transmitted parameters. Thus, HRTFs 11, 12, 13 and 14 are not true HRTFs filters but are HRTF-based filters, since these filters not only depend from the HRTFs, which are independent from the transmitted signal. Instead, HRTFs 11, 12, 13 and 14 are also dependent on the transmitted signal due to the fact that the channel level difference parameters cldi and cldr are used for calculating these HRTFs 11, 12, 13 and 14. Now, the Fig. 15 situation is obtained, which still has three channels rather than two transmitted channels as included in a preferred down-mix signal. Therefore, a combination of the six HRTFs 11, 12, 5, 6, 13, 14 into four HRTFs 15, 16, 17, 18 as illustrated in Fig. 16a has to be done. To this end, HRTFs 11, 5, 13 are combined using a left upmix rule, which becomes clear from the upmix matrix in Fig. 16b. Particularly the left upmix rule as shown in Fig 16b and as indicated in block 175 includes parameters nin, m21 and m31. This left upmix rule is in the matrix equation of Fig. 16 only for being multiplied by the left channel. Therefore, these three parameters are called the left upmix rule. As outlined in block 176, the same HRTFs 11, 5, 13 are combined, but now using the right upmix rule, i.e., in the Fig. 16b embodiment, the parameters m12, m22 and m32, which all are used for being multiplied by the right channel Ro in Fig. 16b. Thus, HRTF 15 and HRTF 17 are generated. Analogously HRTF 12, HRTF 6 and HRTF 14 of Fig. 15 are combined using the upmix left parameters m11, m21 and m31 to obtain HRTF 16. A corresponding combination is performed using HRTF 12, HRTF, 6 HRTF 14, but now with the upmix right parameters or right upmix rule indicated by m12, m22 and m32 to obtain HRTF 18 of Fig. 16a. Again, it is emphasized that, while original HRTFs in Fig. 14 did not at all depend on the transmitted signal, the new HRTF-based filters 15, 16, 17, 18 now depend on the transmitted signal, since the spatial parameters included in the multi-channel signal were used for calculating these filters 15, 16, 17 and 18. To finally obtain a binaural left channel LB and a binaural right channel RB, the outputs of filters 15 and 17 have to be combined in an adder 130a. Analogously, the output of the filters 16 and 18 have to be combined in an adder 130b. These adders 130a, 130b reflect the superposition of two signals within the human ear. Subsequently, Fig. 18 will be discussed. Fig. 18 shows a preferred embodiment of an inventive multi-channel decoder for generating a binaural signal using a downmix signal derived from an original multi-channel signal. The downmix signal is illustrated at z1 and Z2 or is also indicated by "L" and "R". Furthermore, the downmix signal has parameters associated therewith, which parameters are at least a channel level difference for left and left surround or a channel level difference for right and right surround and information on the upmixing rule. Naturally, when the original multi-channel signal was only a three-channel signal, cld1 or cldr are not transmitted and the only parametric side information will be information on the upmix rule which, as outlined before, is such an upmix rule which results in an energy-error in the upmixed signal. Thus, although the waveforms of the upmixed signals when a non-binaural rendering is performed, match as close as possible the original waveforms, the energy of the upmixed channels is different from the energy of the corresponding original channels. In the preferred embodiment of Fig. 18, the upmix rule information is reflected by two upmix parameters cpci, cpc2. However, any other upmix rule information could be applied and signaled via a certain number of bits. Particularly, one could signal certain upmix scenarios and upmix parameters using a predetermined table at the decoder so that only the table indices have to be transmitted from an encoder to the decoder. Alternatively, one could also use different upmixing scenarios such as an upmix from two to more than three channels. Alternatively, one could also transmit more than two predictive upmix parameters which would then require a corresponding different downmix rule which has to fit to the upmix rule as will be discussed in more detail with respect to Fig. 20. Irrespective of such a preferred embodiment for the upmix rule information, any upmix rule information is sufficient as long as an upmix to generate an energy-loss affected set of upmixed channels is possible, which is waveform-matched to the corresponding set of original signals. The inventive multi-channel decoder includes a gain factor calculator 180 for calculating at least one gain factor gi, gr or g, for reducing or eliminating the energy-error. The gain factor calculator calculates the gain factor based on the upmix rule information and filter characteristics of HRTF-based filters corresponding to upmix channels which would be obtained, when the upmix rule would be applied. However, as outlined before, in the binaural rendering, this upmix does not take place. Nevertheless, as discussed in connection with Fig. 15 and blocks 175, 176, 177, 178 of Fig. 17, HRTF-based filters corresponding to these upmix channels are nevertheless used. As discussed before, the gain factor calculator 180 can calculate different gain factors gi and gr as outlined in equation (27), when, instead of n, 1 or r is inserted. Alternatively, the gain factor calculator could generate a single gain factor for both channels as indicated by equation (28) . Importantly, the inventive gain factor calculator 180 calculates the gain factor based not only on the upmix rule, but also based on the filter characteristics of the HRTF-based filters corresponding to upmix channels. This reflects the situation that the filters themselves also depend on the transmitted signals and are also affected by an energy-error. Thus, the energy-error is not only caused by the upmix rule information such as the prediction parameters CPCi, CPC2, but is also influenced by the filters themselves. Therefore, for obtaining a well-adapted gain correction, the inventive gain factor not only depends on the prediction parameter but also depends on the filters corresponding to the upmix channels as well. The gain factor and the downmix parameters as well as the HRTF-based filters are used in the filter processor 182 for filtering the downmix signal to obtain an energy-corrected binaural signal having a left binaural channel LB and having a right binaural channel RB. In a preferred embodiment, the gain factor depends on a relation between the total energy included in the channel impulse responses of the filters corresponding to upmix channels to a difference between this total energy and an estimated upmix energy error AE. AE can preferably be calculated by combining the channel impulse responses of the filters corresponding to upmix channels and to then calculating the energy of the combined channel impulse response. Since all numbers in the relations for GL and GR in Fig. 18 are positive numbers, which becomes clear from the definitions for AE and E, it is clear that both gain factors are larger than 1. This reflects the experience illustrated in Fig. 5 that, in most times, the energy of the binaural signal is lower than the energy of the original multi-channel signal. It is also to note, that even when the multi-channel gain compensation is applied, i.e., when the factor p is used in most signals, nevertheless an energy-loss is caused. Fig. 19a illustrates a preferred embodiment of the filter processor 182 of Fig. 18. Particularly, Fig. 19a illustrates the situation, when in block 182a the combined filters 15, 16, 17, and 18 of Fig. 16a without gain compensation are used and the filter output signals are added as outlined in Fig. 13. Then, the output of box 182a is input into a scaler box 182b for scaling the output using the gain factor calculated by box 180. Alternatively, the filter processor can be constructed as shown in Fig. 19b. Here, HRTFs 15 to 18 are calculated as illustrated in box 182c. Thus, the calculator 182c performs the HRTF combination without any gain adjustment. Then, a filter adjuster 182d is provided, which uses the inventively calculated gain factor. The filter adjuster results in adjusted filters as shown in block 180e, where block 180e performs the filtering using the adjusted filter and performs the subsequent adding of the corresponding filter output as shown in Fig. 13. Thus, no post-scaling as in Fig 19a is necessary to obtain gain-corrected binaural channels LB and RB. Generally, as has been outlined in connection with equation 16, equation 17 and equation 18, the gain calculation takes place using the estimated upmix error AE. This approximation is especially useful for the case where the number of upmix channels is equal to the number of downmix channels +1. Thus, in case of two downmix channels, this approximation works well for three upmix channels. Alternatively, when one would have three downmix channels, this approximation would also work well in a scenario in which there are four upmix channels. However, it is to be noted that the calculation of the gain factor based on an estimation of the upmix error can also be performed for scenarios in which for example, five channels are predicted using three downmix channels. Alternatively, one could also use a prediction-based upmix from two downmix channels to four upmix channels. Regarding the estimated upmix energy-error AE, one can not only directly calculate this estimated error as indicated in equation (25) for the preferred case, but one could also transmit some information on the actually occurred upmix error in a bit stream. Nevertheless, even in other cases than the special case as illustrated in connection with equations (25) to (28), one could then calculate the value EnB based on the HRTF-based filters for the upmix channels using prediction parameters. When equation (26) is considered, it becomes clear that this equation can also easily be applied to a 2/4 prediction upmix scheme, when the weighting factors for the energies of the HRTF-based filter impulse responses are correspondingly adapted. In view of that, it becomes clear that the general structure of equation (27), i.e., calculating the gain factor based on relation of EB/(EB - AEB) also applies for other scenarios. Subsequently, Fig. 20 will be discussed to show a schematic implementation of a prediction-based encoder which could be used for generating the downmix signal L, R and the upmix rule information transmitted to a decoder so that the decoder can perform the gain compensation in the context of the binaural filter processor. A downmixer 191 receives five original channels or, alternatively, three original channels as illustrated by (Ls and Rs) . The downmixer 191 can work based on a pre- determined downmix rule. In that case, the downmix rule indication as illustrated by line 192 is not required. Naturally, the error- minimizer 193 could vary the downmix rule as well in order to minimize the error between reconstructed channels at the output of an upmixer 194 with respect to the corresponding original input channels. Thus, the error-minimizer 193 can vary the downmix rule 192 or the upmixer rule 196 so that the reconstructed channels have a minimum prediction loss AE. This optimization problem is solved by any of the well-known algorithms within the error-minimizer 193, which preferably operates in a subband-wise way to minimize the difference between the reconstruction channels and the input channels. As stated before, the input channels can be original channels L, Ls, R, Rs, C. Alternatively the input channels can only be three channels L, R, C, wherein, in this context, the input channels L, R, can be derived by corresponding OTT boxes illustrated in Fig. 11. Alternatively, when the original signal only has channels L, R, C, then these channels can also be termed as "original channels". Fig. 20 furthermore illustrates that any upmix rule information can be used besides the transmission of two prediction parameters as long as a decoder is in the position to perform an upmix using this upmix rule information. Thus, the upmix rule information can also be an entry into a lookup table or any other upmix related information. The present invention therefore, provides an efficient way of performing binaural decoding of multi-channel audio signals based on available downmixed signals and additional control data by means of HRTF filtering. The present invention provides a solution to the problem of spectral coloring arising from the combination of predictive upmix with binaural decoding. Depending on certain implementation requirements of the inventive methods, the inventive methods can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, in particular a disk, DVD or a CD having electronically readable control signals stored thereon, which cooperate with a programmable computer system such that the inventive methods are performed. Generally, the present invention is, therefore, a computer program product with a program code stored on a machine readable carrier, the program code being operative for performing the inventive methods when the computer program product runs on a computer. In other words, the . inventive methods are, therefore, a computer program having a program code for performing at least one of the inventive methods when the computer program runs on a computer. While the foregoing has been particularly shown and described with reference to particular embodiments thereof, it will be understood by those skilled in the art that various other changes in the form and details may be made without departing from the spirit and scope thereof. It is to be understood that various changes may be made in adapting to different embodiments without departing from the broader concepts disclosed herein and comprehended by the claims that follow. Claims 1. Multi-channel decoder for generating a binaural signal from a downmix signal derived from an original multi- channel signal using parameters including an upmix rule information useable for upmixing the downmix signal with an upmix rule, the upmix rule resulting in an energy-error, comprising: a gain factor calculator for calculating at least one gain factor for reducing or eliminating the energy- error, based on the upmix rule information and filter characteristics of a head related transfer function based filters corresponding to upmix channels; and a filter processor for filtering the downmix signal using the at least one gain factor, the filter characteristics and the upmix rule information to obtain an energy-corrected binaural signal. 2. Multi-channel decoder of claim 1, in which the filter processor is operative to calculate filter coefficients for two gain adjusted filters for each channel of the downmix signal and to filter the downmix channel using each of the two gain adjusted filters. 3. Multi-channel decoder of claim 1, in which the filter processor is operative to calculate filter coefficients for two filters for each channel of the downmix channel without using the gain factor and to filter the downmix channels and to gain adjust subsequent to filtering the downmix channel. 4. Multi-channel decoder of claim 1, in which the gain factor calculator is operative to calculate the gain factor based on an energy of a combined impulse response of the filter characteristics, the combined impulse response being calculated by adding or subtracting individual filter impulse responses. 5. Multi-channel decoder of claim 1, in which the gain factor calculator is operative to calculate the gain factor based on a combination of powers of individual filter impulse responses. 6. Multi-channel decoder of claim 5, in which the gain factor calculator is operative to calculate the gain factor based on a weighted addition of powers of individual filter impulse responses, wherein weighting coefficients used in the weighted addition depend on the upmix rule information. 7. Multi-channel decoder of claim 1, in which the gain factor calculator is operative to calculate the gain factor based on an expression having a numerator and a denominator, the numerator having a combination of powers of individual filter impulse filter responses, and the denominator having a weighted addition of powers of individual filter impulse responses, wherein weighting coefficients used in the weighted addition depend on the upmix rule information. 8. Multi-channel decoder of claim 1, in which the gain factor calculator is operative to calculate the gain factor based on the following equation: wherein gn is the gain factor for the first channel, when n is set to 1, wherein g2 is the gain factor of a second channel, when n is set to 2, wherein EnB is a weighted addition energy calculated by weighting energies of channel impulse responses using weighting parameters, and wherein AEn is an estimate for the energy error introduced by the upmix rule, wherein a, (3 and y are upmix rule dependent parameters, and wherein s is a number greater than or equal to zero. 9. Multi-channel decoder of claim 8, in which the gain factor calculator is operative to calculate En and AEn based on the following equation: in which bn, 1 is an HRTF-based filter corresponding to first upmix channel and a nth binaural channel, wherein bn,2 is a HRTF-based filter impulse response corresponding to a second upmix channel and a nth binaural channel, wherein bn,3 is a HRTF-based filter impulse response corresponding to a third upmix channel for a nth binaural channel, wherein the following definitions are valid wherein c1 is a first prediction parameter, C2 is a second prediction parameter, and wherein the first prediction parameter and the second prediction parameter constitute the upmix rule information. 10. Multi-channel decoder of claim 1, in which the gain factor calculator is operative to calculate a common gain factor for a left binaural channel and a right binaural channel. 11. Multi-channel decoder of claim 1, in which the filter processor is operative to use, as the filter characteristics, the head related transfer function based filters for the left binaural channel and the right binaural channel for virtual center, left and right positions or to use filter characteristics derived by combining HRTF filters for a virtual left front position and a virtual left surround position or by combining HRTF filters for a virtual right front position and a virtual right surround position. 12. Multi-channel decoder of claim 11, in which parameters relating to original left and left surround channels or original right and right surround channels are included in a decoder input signal, and wherein the filter processor is operative to use the parameters for combining the head related transfer function filters. 13. Multi-channel decoder of claim 1, in which the gain factor calculator is operative to calculate the gain factor for a binaural channel based on a ratio of a weighted linear combination of energies of channel impulse responses of the HRTF-based filters for the binaural channel and a value obtained by subtracting an estimated energy-error from the weighted linear combination. 14. Multi-channel decoder of claim 13, in which the gain factor calculator is operative to use information on the upmix rule for determining the weighting factor. 15. Multi-channel decoder of claim 14, in which the upmix rule information includes at least two prediction parameters usable for constructing an upmix matrix resulting in output channels having the energy-error with respect to corresponding three input channels. 16. Multi-channel decoder of claim 1, in which the filter processor is operative to have, as filter characteristics, a first filter for filtering a left downmix channel for obtaining a first left binaural output, a second filter for filtering a right downmix channel for obtaining a second left binaural output, a third filter for filtering a left downmix channel for obtaining a first right binaural output, a fourth filter for filtering a right downmix channel for obtaining a second right binaural output, an adder for adding the first left binaural output and the second left binaural output to obtain a left binaural channel and for adding the first right binaural output and the second right binaural output to obtain a right binaural channel, wherein the filter processor is operative to apply a gain factor for the left binaural channel to the first and the second filters or to the left binaural output before or after adding and to apply the gain factor for the right binaural channel to the third filter and to the fourth filter or to the right binaural output before or after adding. 17. Multi-channel decoder of claim 1, in which the upmix rule information includes upmix parameters usable for constructing an upmix matrix resulting in an upmix from two to three channels. 18. Multi-channel decoder of claim 17, in which the upmix rule is defined as follows: wherein L is a first upmix channel, R is a second upmix channel, and C is a third upmix channel, L0 is a first downmix channel, R0 is a second downmix channel, and mij are upmix rule information parameters. 19. Multi-channel decoder of claim 1, in which a prediction loss parameter is included in a multi- channel decoder input signal, and in which a filter processor is operative to scale the gain factor using the prediction loss parameter. 20. Multi-channel decoder of claim 1, in which the gain calculator is operative to calculate the gain factor subband-wise, and in which the filter processor is operative to apply the gain factor subband-wise- 21. Multi-channel decoder of claim 11, in which the filter processor is operative to combine HRTF filters associated with two channels by adding weighted or phase shifted versions of channel impulse responses of the HRTF filters, wherein weighting factors for weighting the channel impulse responses is of the HRTF filters depend on a level difference between the channels, and an applied phase shift depends on a time delay between the channel impulse responses of the HRTF filters. 22. Multi-channel decoder of claim 1, in which filter characteristics of HRTF-based filters or HRTF filters are complex subband filters obtained by filtering a real-valued filter impulse response of an HRTF filter using a complex-exponential modulated filterbank. . 23. Method of multi-channel decoding for generating a binaural signal from a downmix signal derived from an original multi-channel signal using parameters including an upmix rule information useable for upmixing the downmix signal with an upmix rule, the upmix rule resulting in an energy-error, comprising: calculating at least one gain factor for reducing or eliminating the energy-error, based on the upmix rule information and filter characteristics of a head related transfer function based filters corresponding to upmix channels; and filtering the downmix signal using the at least one gain factor, the filter characteristics and the upmix rule information to obtain an energy-corrected binaural signal. 24. Computer program having a program code for performing the method in accordance with claim 23, when the computer program runs on a computer. A multi-channel decoder for generating a binaural signal from a downmix signal using upmix rule information on an energy-error introducing upmix rule for calculating a gain factor (180) based on the upmix rule information and characteristics of head related transfer function based filters corresponding to upmix channels. The one or more gain factors are used by a filter processor (182) for filtering the downmix signal so that an energy corrected binaural signal having a left binaural channel and a right binaural channel is obtained.

Full Text

Field of the Invention
The present invention relates to binaural decoding of
multi-channel audio signals based on available .downmixed
signals and additional control data, by means of HRTF
filtering.
Background of the Invention and Prior Art
Recent development in audio coding has made methods
available to recreate a multi-channel representation of an
audio signal based on a stereo (or mono) signal and
corresponding control data. These methods differ
substantially from older matrix based solution such as
Dolby Prologic, since additional control data is
transmitted to control the re-creation, also referred to
as up-mix, of the surround channels based on the
transmitted mono or stereo channels.
Hence, such a parametric multi-channel audio decoder, e.g.
MPEG Surround reconstructs Nchannels based on M
transmitted channels, where N>M, and the additional
control data. The additional control data represents a
significantly lower data rate than that required for
transmission of all N channels, making the coding very
efficient while at the same time ensuring compatibility
with both M channel devices and N channel devices. [J.
Breebaart et al. "MPEG spatial audio coding / MPEG
Surround: overview and current status", Proc. 119th AES
convention, New York, USA, October 2005, Preprint 6447].
These parametric surround coding methods usually comprise
a parameterization of the surround signal based on Channel

Level Difference (CLD) and Inter-channel coherence/cross-
correlation (ICC). These parameters describe power ratios
and correlation between channel pairs in the up-mix
process. Further Channel Prediction Coefficients (CPC) are
also used in prior art to predict intermediate or output
channels during the up-mix procedure.
Other developments in audio coding have provided means to
obtain a multi-channel signal impression over stereo
headphones. This is commonly done by downmixing a multi-
channel signal to stereo using the original multi-channel
signal and HRTF (Head Related Transfer Functions) filters.
Alternatively, it would, of course, be useful for
computational efficiency reasons and also for audio quality
reasons to short-cut the generation of the binaural signal
having the left binaural channel and the right binaural
channel.
However, the question is how the original HRTF filters can
be combined. Further a problem arises in a context of an
energy-loss-affected upmixing rule, i.e., when the multi-
channel decoder input signal includes a downmix signal
having, for example, a first downmix channel and a second
downmix channel, and further having spatial parameters,
which are used for upmixing in a non-energy-conserving way.
Such parameters are also known as prediction parameters or
CPC parameters. These parameters have, in contrast to
channel level difference parameters the property that they
are not calculated to reflect the energy distribution
between two channels, but they are calculated for
performing a best-as-possible waveform matching which
automatically results in an energy error (e.g. loss),
since, when the prediction parameters are generated, one
does not care about energy-conserving properties of an
upmix, but one does care about having a good as possible
time or subband domain waveform matching of the
reconstructed signal compared to the original signal.

When one would simply linearly combine HRTF filters based
on such transmitted spatial prediction parameters, one will
receive artifacts which are especially serious, when the
prediction of the channels performs poorly. In that
situation, even subtle linear dependencies lead to
undesired spectral coloring of the binaural output. It has
been found out that this artifact occurs most frequently
when the original channels carry signals that are pairwise
uncorrelated and have comparable magnitudes.
Summary of the Invention
It is the object of the present invention to provide an
efficient and qualitatively acceptable concept for multi-
channel decoding to obtain a binaural signal which can be
used, for example, for headphone reproduction of a multi-
channel signal.
In accordance with the first aspect of the present
invention, this object is achieved by a multi-channel
decoder for generating a binaural signal from a downmix
signal derived from an original multi-channel signal using
parameters including an upmix rule information useable for
upmixing the downmix signal with an upmix rule, the upmix
rule resulting in an energy-error, comprising: a gain
factor calculator for calculating at least one gain factor
for reducing or eliminating the energy-error, based on the
upmix rule information and filter characteristics of a head
related transfer function based filters corresponding to
upmix channels, and a filter processor for filtering the
downmix signal using the at least one gain factor, the
filter characteristics and the upmix rule information to
obtain an energy-corrected binaural signal.
In accordance with a second aspect of this invention, this
object is achieved by a method of multi-channel decoding

for generating a binaural signal from a downmix signal
derived from an original multi-channel signal using
parameters including an upmix rule information useable for
upmixing the downmix signal with an upmix rule, the upmix
rule resulting in an energy-error, comprising: calculating
at least one gain factor for deducing or eliminating the
energy-error, based on the upmix rule information and
filter characteristics of a head related transfer function
based filters corresponding to upmix channels; and
filtering the downmix signal using the at least one gain
factor, the filter characteristics and the upmix rule
information to obtain an energy-corrected binaural signal.
Further aspects of this invention relate to a computer
program having a computer-readable code which implements,
when running on a computer, the method of multi-channel
decoding.
The present invention is based on the finding that one can
even advantageously use up-mix rule information on an upmix
resulting in an energy error for filtering a downmix signal
to obtain a binaural signal without having to fully render
the multichannel signal and to subsequently apply a huge
number of HRTF filters. Instead, in accordance with the
present invention, the upmix rule information relating to
an energy-error-affected upmix rule can advantageously be
used for short-cutting binaural rendering of a downmix
signal, when, in accordance with the present invention, a
gain factor is calculated and used when filtering the
downmix signal, wherein this gain factor is calculated such
that the energy error is reduced or completely eliminated.
Particularly, the gain factor not only depends on the
information on the upmix rule such as the prediction
parameters, but, importantly, also depends on head related
transfer function based filters corresponding to upmix
channels, for which the upmix rule is given. Particularly,
these upmix channels never exist in the preferred

embodiment of the present invention, since the binaural
channels are calculated without firstly rendering, for
example, three intermediate channels. However, one can
derive or provide HRTF based filters corresponding to the
upmix channels although the upmix channels themselves never
exist in the preferred embodiment. It has been found out
that the energy error introduced by such an energy-loss-
affected upmix rule not only corresponds to the upmix rule
information which is transmitted from the encoder to the
decoder, but also depends on the HRTF based filters so
that, when generating the gain factor, the HRTF based
filters also influence the calculation of the gain factor.
In view of that, the present invention accounts for the
interdependence between upmix rule information such as
prediction parameters and the specific appearance of the
HRTF based filters for the channels which would be the
result of upmixing using the upmix rule.
Thus, the present invention provides a solution to the
problem of spectral coloring arising from the usage of a
predictive upmix in combination with binaural decoding of
parametric multi-channel audio.
Preferred embodiments of the present invention comprise the
following features: an audio decoder for generating a
binaural audio signal from M decoded signals and spatial
parameters pertinent to the creation of N>M channels,
the decoder comprising a gain calculator for estimating, in
a multitude of subbands, two compensation gains from P
pairs of binaural subband filters and a subset of the
spatial parameters pertinent to the creation of P
intermediate channels, and a gain adjuster for modifying,
in a multitude of subbands, M pairs of binaural subband
filters obtained by linear combination of the P pairs of
binaural subband filters, the modification consisting of
multiplying each of the M pairs with the two gains
computed by the gain calculator.

Brief Description of the Drawings
The present invention will now be described by way of
illustrative examples, not limiting the scope or spirit of
the invention, with reference to the accompanying
drawings, in which:
Fig. 1 illustrates binaural synthesis of parametric
multichannel signals using HRTF related filters;
Fig. 2 illustrates binaural synthesis of parametric
multichannel signals using combined filtering;
Fig. 3 illustrates the components of the inventive
parameter/filter combiner;
Fig. 4 illustrates the structure of MPEG Surround
spatial decoding;
Fig. 5 illustrates the spectrum of a decoded binaural
signal without the inventive gain compensation;
Fig. 6 illustrates the spectrum of the inventive
decoding of a binaural signal.
Fig. 7 illustrates a conventional binaural synthesis
using HRTFs;
Fig. 8 illustrates a MPEG surround encoder;
Fig. 9 illustrates cascade of MPEG surround decoder and
binaural synthesizer;
Fig. 10 illustrates a conceptual 3D binaural decoder for
certain configurations;

Fig. 11 illustrates a spatial encoder for certain
configurations;
Fig. 12 illustrates a spatial (MPEG Surround) decoder;
Fig. 13 illustrates filtering of two downmix channels
using four filters to obtain binaural signals
without gain factor correction;
Fig. 14 illustrates a spatial setup for explaining
different HRTF filters 1-10 in a five channels
setup;
Fig. 15 illustrates a situation of Fig. 14, when the
channels for L, Ls and R, Rs have been combined;
Fig. 16a illustrates the setup from Fig. 14 or Fig. 15,
when a maximum combination of HRTF filters has
been performed and only the four filters of Fig.
13 remain;
Fig. 16b illustrates an upmix rule as determined by the
Fig. 20 encoder having upmix coefficients
resulting in a non-energy-conserving upmix;
Fig. 17 illustrates how HRTF filters are combined to
finally obtain four HRTF-based filters;
Fig. 18 illustrates a preferred embodiment of an
inventive multi-channel decoder;
Fig. 19a illustrates a first embodiment of the inventive
multi-channel decoder having a scaling stage
after HRTF-based filtering without gain
correction;

Fig. 19b illustrates an inventive device having adjusted
HRTF-based filters which result in a gain-
adjusted filter output signal; and
Fig. 20 shows an example for an encoder generating the
information for a non-energy-conserving upmix
rule.
Detailed description of preferred embodiments
Before discussing the inventive gain adjusting aspect in
detail, a combination of HRTF filters and usage of HRTF-
based filters will be discussed in connection with Figs. 7
to 11.
In order to better outline the features and advantages of
the present invention a more elaborate description is
given first. A binaural synthesis algorithm is outlined in
Fig. 7. A set of input channels is filtered by a set of
HRTFs. Each input signal is split in two signals (a left
'L' , and a right 'R' component); each of these signals is
subsequently filtered by an HRTF corresponding to the
desired sound source position. All left-ear signals are
subsequently summed to generate the left binaural output
signal, and the right-ear signals are summed to generate
the right binaural output signal.
The HRTF convolution can be performed in the time domain,
but it is often preferred to perform the filtering in the
frequency domain due to computational efficiency. In that
case, the summation as shown in Fig. 7 is also performed in
the frequency domain.
In principle, the binaural synthesis method as outlined in
Fig. 7 could be directly used in combination with an MPEG
surround encoder/decoder. The MPEG surround encoder is
schematically shown in Fig. 8. A multi-channel input signal

is analyzed by a spatial encoder, resulting in a mono or
stereo down mix signal, combined with spatial parameters.
The down mix can be encoded with any conventional mono or
stereo audio codec. The resulting down-mix bit stream is
combined with the spatial parameters by a multiplexer,
resulting in the total output bit stream.
A binaural synthesis scheme in combination with an MPEG
surround decoder is shown in Fig. 9. The input bit stream
is de-multiplexed resulting in spatial parameters and a
down-mix bit stream. The latter bit stream is decoded using
a conventional mono or stereo decoder. The decoded down mix
is decoded by a spatial decoder, which generates a multi-
channel output based on the transmitted spatial parameters.
Finally, the multi-channel output is processed by a
binaural synthesis stage as depicted in Fig. 7, resulting
in a binaural output signal.
There are however at least three disadvantages of such a
cascade of an MPEG surround decoder and a binaural
synthesis module:
• A multi-channel signal representation is computed as
an intermediate step, followed by HRTF convolution and
downmixing in the binaural synthesis step. Although
HRTF convolution should be performed on a per channel
basis, given the fact that each audio channel can have
a different spatial position, this is an undesirable
situation from a complexity point of view.
• The spatial decoder operates in a filterbank (QMF)
domain. HRTF convolution, on the other hand, is
typically applied in the FFT domain. Therefore, a
cascade of a multi-channel QMF synthesis filterbank, a
multi-channel DFT transform, and a stereo inverse DFT
transform is necessary, resulting in a system with
high computational demands.

• Coding artifacts created by the spatial decoder to
create a multi-channel reconstruction will be audible,
and possibly enhanced in the (stereo) binaural output.
The spatial encoder is shown in Fig. 11. A multi-channel
input signal consisting of Lf, Ls, C, Rf and Rs signals,
for the left-front, left-surround, center, right-front and
right-surround channels is processed by two 'OTI" units,
which both generate a mono down mix and parameters for two
input signals. The resulting down-mix signals, combined
with the center channel are further processed by a *TTT'
(Two-To-Three) encoder, generating a stereo down mix and
additional spatial parameters.
The parameters resulting from the 'TTT' encoder typically
consist of a pair of prediction coefficients for each
parameter band, or a pair of level differences to describe
the energy ratios of the three input signals. The
parameters of the 'OTT' encoders consist of level
differences and coherence or cross-correlation values
between the input signals for each frequency band.
In Fig. 12 a MPEG Surround decoder is depicted. The downmix
signals 10 and rO are input into a Two-To-Three module,
that recreates a center channel, a right side channel and a
left side channel. These three channels are further
processed by several OTT modules (One-To-Two) yielding the
six output channels.
The corresponding binaural decoder as seen from a
conceptual point of view is shown in Fig. 10. Within the
filterbank domain, the stereo input signal {L0, Ro) is
processed by a TTT decoder, resulting in three signals L, R
and C. These three signals are subject to HRTF parameter
processing. The resulting 6 channels are summed to generate
the stereo binaural output pair {Lb, Rb) .

The TTT decoder can be described as the following matrix
operation:

with matrix entries mxy dependent on the spatial
parameters. The relation of spatial parameters and matrix
entries is identical to those relations as in the 5.1-
multichannel MPEG surround decoder. Each of the three
resulting signals L, R, and C are split in two and
processed with HRTF parameters corresponding to the desired
(perceived) position of these sound sources. For the center
channel (C) , the spatial parameters of the sound source
position can be applied directly, resulting in two output
signals for center, LB(C) and RB(C):

For the left (L) channel, the HRTF parameters from the
left-front and left-surround channels are combined into a
single HRTF parameter set, using the weights wlf and wrf .
The resulting 'composite' HRTF parameters simulate the
effect of both the front and surround channels in a
statistical sense. The following equations are used to
generate the binaural output pair (LB, RB) for the left
channel:

In a similar fashion, the binaural output for the right
channel is obtained according to:

Given the above definitions of LB(C), RB(C), LB{L), RB(L),
LB(R) and RB(R), the complete LB and RB signals can be
derived from a single 2 by 2 matrix given the stereo input
signal:

with

The Hx(Y) filters can be expressed as parametric weighted
combinations of parametric versions of the original HRTF
filters. In order for this to work, the original HRTF
filters are expressed as a
• An (average) level per frequency band for the left-ear
impulse response;
• An (average) level per frequency band for the right-
ear impulse response;
• An (average) arrival time or phase difference between
the left-ear and right-ear impulse response.
Hence, the HRTF filters for the left and right ear given
the center channel input signal is expressed as:

where P,(C) is the average level for a given frequency band
for the left ear, and Ø(c)is the phase difference.
Hence, the HRTF parameter processing simply consists of a
multiplication of the signal with P1and Pr corresponding
to the sound source position of the center channel, while
the phase difference is distributed symmetrically. This
process is performed independently for each QMF band, using
the mapping from HRTF parameters to QMF filterbank on the
one hand, and mapping from spatial parameters to QMF band
on the other hand.
Similarly the HRTF filters for the left and right ear given
the left channel and right channel are given by:

Clearly, the HRTFs are weighted combinations of the levels
and phase differences for the parameterized HRTF filters
for the six original channels.
The weights Wif and Wjs depend on the CLD parameter of the
'OTT' box for Lf and Ls:

And the weights wrf and Wrs depend on the CLD parameter of
the 'OTT" box for Rf and Rs:

The above approach works well for short HRTF filters that
sufficiently accurate can be expressed as an average level
per frequency band, and an average phase difference per
frequency band. However, for long echoic HRTFs this is not
the case.
The present invention teaches how to extend the approach of
a 2 by 2 matrix binaural decoder to handle arbitrary length
HRTF filters. In order to achieve this, the present
invention comprises the following steps:
• Transform the HRTF filter responses to a filterbank
domain;
• Overall delay difference or phase difference
extraction from HRTF filter pairs;
•• Morph the responses of the HRTF filter pair as a
function of the CLD parameters
• Gain adjustment
This is achieved by replacing the six complex gainsHY(X)
for Y = L0, R0and X = L,R,C with six filters. These filters
are derived from the ten filters HY(X) for Y = L0,R0 and
X=Lf,Ls,Rf,Rs,C , which describe the given HRTF filter
responses in the QMF domain. These QMF representations can
be achieved according to the method described below.
The morphing of the front and surround channel filters is
performed with a complex linear combination according to

The phase parameter Øxy can be defined from the main delay
time difference rxybetween the front and back HRTF filters
and the subband index n of the QMF bank via

The role of this phase parameter in the morphing of
filters is twofold. First, it realizes a delay
compensation of the two filters prior to superposition
which leads to a combined response which models a main
delay time corresponding to a source position between the
front and the back speakers. Second, it makes the
necessary gain compensation factor gmuch more stable and
slowly varying over frequency than in the case of simple
superposition with Øxy=0.
The gain factor g is determined by the same incoherent
addition power rule as for the parametric HRTF case,

andPxy is the real value of the normalized complex cross
correlation between the filters

In the case of simple superposition withØxy=0, the value
of Pxy varies in an erratic and oscillatory manner as a
function of frequency, which leads to the need for
extensive gain adjustment. In practical implementation it

is necessary to limit the value of the gaingand a
remaining spectral colorization of the signal cannot be
avoided.
In contrast, the use of morphing with a delay based phase
compensation as taught by the present invention leads to a
smooth behavior of pxy as a function of frequency. This
value is often even close to one for natural HRTF derived
filter pairs since they differ mainly in a delay and
amplitude, and the purpose of the phase parameter is to
take the delay difference into account in the QMF
filterbank domain.
An alternative beneficial choice of phase parameter Øxy is
given by computing the phase angle of the normalized
complex cross correlation between the filters
Hy(Xf) and HY(Xs)f
and unwrapping the phase values with standard unwrapping
techniques as a function of the subband index n of the QMF
bank. This choice has the consequence that Pxy is never
negative and hence the compensation gain ^satisfies
for all subbands. Moreover this choice of phase
parameter enables the morphing of the front and surround
channel filters in situations where a main delay time
difference txy is not available.
All signals considered below are subband samples from a
modulated filter bank or windowed FFT analysis of discrete

time signals or discrete time signals. It is understood
that these subbands have to be transformed back to the
discrete time domain by corresponding synthesis filter
bank operations.
Fig. 1 illustrates a procedure for binaural synthesis of
parametric multichannel signals using HRTF related filters.
A multichannel signal comprising N channels is produced by
spatial decoding 101 based on M and transmitted spatial parameters. These TV channels are in
turn converted into two output channels intended for
binaural listening by means of HRTF filtering. This HRTF
filtering 102 superimposes the results of filtering each
input channel with one HRTF filter for the left ear and one
HRTF filter for the right ear. All in all, this requires
2N filters. Whereas the parametric multichannel signal
achieves a high quality listener experience when listened
to through N loudspeakers, subtle interdependencies of the
N signals will lead to artifacts for the binaural
listening. These artifacts are dominated by deviation in
spectral content from the reference binaural signal as
defined by HRTF filtering of the original N channels prior
to coding. A further disadvantage of this concatenation is
that the total computational cost for binaural synthesis is
the addition of the cost required for each of the
components 101 and 102.
Fig. 2 illustrates binaural synthesis of parametric
multichannel signals by using the combined filtering taught
by the present invention. The transmitted spatial
parameters are split by 201 into two sets, Set 1 and Set 2.
Here, Set 2 comprises parameters pertinent to the creation
of P intermediate channels from the M transmitted channels
and Set 1 comprises parameters pertinent to the creation of
N channels from the P intermediate channels. The prior art
precombiner 202 combines selected pairs of the 2N HRTF
related subband filters with weights that depend the
parameter Set 1 and the selected pairs of filters. The

result of this precombination is 2P binaural subband
filters which represent a binaural filter pair for each of
the P intermediate channels. The inventive combiner 203
combines the 2P binaural subband filters into a set of
2Mbinaural subband filters by applying weights that depend
both on the parameter Set 2 and the 2P binaural subband
filters. In comparison, a prior art linear combiner would
apply weights that depend only on the parameter Set 2. The
resulting set of 2M filters consists of a binaural filter
pair for each of the Mtransmitted channels. The combined
filtering unit 204 obtains a pair of contributions to the
two channel output for each of the Mtransmitted channels
by filtering with the corresponding filter pair.
Subsequently, all the Mcontributions are added up to form
a two channel output in the subband domain.
Fig. 3 illustrates the components of the inventive
combiner 203 for combination of spatial parameters and
binaural filters. The linear combiner 301 combines the 2P
binaural subband filters into 2M binaural filters by
applying weights that are derived from the given spatial
parameters, where these spatial parameters are pertinent
to the creation of P intermediate channels from the
Mtransmitted channels. Specifically, this linear
combination simulates the concatenation of an upmix from
Mtransmitted channels to P intermediate channels
followed by a binaural filtering from P sources. The gain
adjuster 303 modifies the 2M binaural filters output from
the linear combiner 301 by applying a common left gain to
each of the filters that correspond to the left ear output
and by applying a common right gain to each of the filters
that correspond to the right ear output. Those gains are
obtained from gain calculator 302 which derives the gains
from the spatial parameters and the 2P binaural filters.
The purpose of the gain adjustment of the inventive
components 302 and 303 is to compensate for the situation
where theP intermediate channels of the spatial decoding
carry linear dependencies that lead to unwanted spectral

coloring due to the linear combiner 301. The gain
calculator 302 taught by the present invention includes
means for estimating an energy distribution of the P
intermediate channels as a function of the spatial
parameters.
Fig. 4 illustrates the structure of MPEG Surround spatial
decoding in the case of a stereo transmitted signal. The
analysis subbands of the M = 2transmitted signals are fed
into the 2—>3 box 401 which outputs P = 3 intermediate
signals, a combined left, a combined right, and a combined
center. This upmix depends on a subset of the transmitted
spatial parameters which corresponds to Set 2 on Fig. 2.
The three intermediate signals are subsequently fed into
three l->2 boxes 402-404 which generate a totality ofN=6
signals 405: lf (left front), ls (left surround), rf (right
front) , rs (right surround) , c (center) , and Ife (low
frequency extension). This upmix depends on a subset of
the transmitted spatial parameters which corresponds to
Set 1 on Fig. 2. The final multichannel digital audio
output is created by passing the six subband signals into
six synthesis filter banks.
Fig. 5 illustrates the problem to be solved by the
inventive gain compensation. The spectrum of a reference
HRTF filtered binaural output for the left ear is depicted
as a solid graph. The dashed graph depicts the spectrum of
the corresponding decoded signal as generated by the
method of Fig. 2, in the case where the combiner 203
consists of the linear combiner 301 only. As it can be
seen, there is a substantial spectral energy loss relative
to the desired reference spectrum in the frequency
intervals 3-4 kHz and 11-13 kHz. There is also a smaller
spectral boost around 1 kHz and 10 kHz.
Fig. 6 illustrates the benefit of using the inventive gain
compensation. The solid graph is the same reference
spectrum as in Fig. 5, but now the dashed graph depicts

the spectrum of the decoded signal as generated by the
method of Fig. 2, in the case where the combiner 203
consists of all the components of Fig. 3. As it can be
seen, there is a significantly improved spectral match
between the two curves compared to that of the two curves
of Fig. 5.
In the text which follows, the mathematical description of
the inventive gain compensation will be outlined. For
discrete complex signals x,y, the complex inner product
and squared norm (energy) is defined by

where y(k) denotes the complex conjugate signal of y(k).
The original multichannel signal consists of TV channels,
and each channel has a binaural HRTF related filter pair
associated to it. It will however be assumed here that the
parametric multichannel signal is created with an
intermediate step of predictive upmix from the
Mtransmitted channels to P predicted channels. This
structure is used in MPEG Surround as described by Fig. 4.
It will be assumed that the original set of IN HRTF
related filters have been reduced by the prior art
precombiner 202 to a filter pair for each of the P
predicted channels where M

channel signals xp, p = l,2,K ,P, aim at approximating theP
signals xp, p = l,2,K ,P, which are derived from the original
N channels via partial downmix. In MPEG Surround, these
signals are a combined left, a combined right and a
combined and scaled center/Ife channel. It is assumed that
the HRTF filter pair corresponding to the signal xp is

described by a subband filter blp for the left ear and a
subband filter b2pfor the right ear. The reference binaural
output signal is thus given by the linear superposition of
filtered signals for n = l,2,

where the star denotes convolution in the time direction.
The subband filters can be given in form of finite impulse
response (FIR) filters, infinite impulse response (IIR) or
derived from a parameterized family of filters.
In the encoder, the downmix is formed by the application
of a MxP downmix matrix D to a column vector of signals
formed by x p = 1,2,K ,Pand the prediction in the decoder is
performed by the application of a PxM prediction matrix
C to the column vector of signals formed by the M
transmitted downmixed channels zm m = l,K ,M,

Both matrices are known at the decoder, and ignoring the
effects of coding the downmixed channels, the combined
effect of prediction can be modeled by

where a are the entries of the matrix product A = CD.
A straightforward method for producing a binaural output
at the decoder is to simply insert the predicted signals
xp in (2) resulting in

In terms of computations, the binaural filtering is
combined with the predictive upmix beforehand such that
(5) can be written as

with the combined filters defined by

This formula describes the action of the linear combiner
301 which combines the coefficients c„m derived from
p,tn
spatial parameters with the binaural subband domain
filters bnp . When the originalP signals^ have a
numerical rank essentially bounded by M, the prediction
can be designed to perform very well and the approximation
jc «JC is valid. This happens for instance if only M of the
P channels are active, or if important signal components
originate from amplitude panning. In that case the decoded
binaural signal (5) is a very good match to the reference
(2). On the other hand, in the general case and
especially in case the original P signals xp are
uncorrelated, there will be a substantial prediction loss
and the output from (5) can have an energy that deviates
considerably from the energy of (2). As the deviation will
be different in different frequency bands, the final audio
output suffers from spectral coloring artifacts as
described by Fig. 5. The present invention teaches how to
circumvent this problem by gain compensating the output
according to

In terms of computations, the gain compensation is
advantageously performed by altering the combined filters
according to the gain adjuster 303 , ^(0m(k) = gnhn^(k) . The
modified combined filtering then becomes

The optimal values of the compensating gains in (8) are

The purpose of the gain calculator 302 is to estimate
these gains from the information available in the decoder.
Several tools for this end will now be outlined. The
available information is represented here by the matrix
entries apq and the HRTF related subband filters bnp.
First, the following approximation will be assumed for the
inner product between signals jc^that have been filtered
by HRTF related subband filters b,d,

This approximation relies on the fact that often most
energy of the filters is concentrated in a dominant single
tap, which in turn presupposes that the time step of the
applied time frequency transform is sufficiently large in
comparison to the main delay differences of HRTF filters.
Applying the approximation (11) in combination with (2)
leads to

The next approximation consists of assuming that the
original signals are uncorrelated, that is (x ,x\ = 0 for
p*q. Then (12) reduces to

For the decoded energy the result corresponding to (12) is

Inserting the predicted signals (4) in (14) and applying
the assumption that the original signals are uncorrelated
gives

What remains in order to be able to calculate the
compensation gain given by the quotient (10) is to
estimate the energy distribution pi , p = l,2,K ,Pof the
original channels up to an arbitrary factor. The present
invention teaches to do this by computing, as a function
of the energy distribution, the prediction matrix Cmodel
corresponding to the assumption that these channels are
uncorrelated and that the encoder aims at minimizing the
prediction error. The energy distribution is then
estimated by solving the nonlinear system of
equations Cmodel =C if possible. For prediction parameters
that lead to a system of equations without solutions, the
gain compensation factors are set to gn=1. This inventive

procedure will be detailed in the following section in the
most important special case.
The computation load imposed by (15) can be reduced in the
case where P = M + l by applying the expansion (see for
instance PCT/EP2005/011586),

where vis a unit vector with components vp such that
Dv = 0, and AE is the prediction loss energy,

The computation of (15) is then advantageously replaced by
the application of (16) in (14), leading to

Subsequently, a preferred specialization to prediction of
three channels from two channels will be discussed. The
case where M = 2and P = 3 is used in MPEG Surround. The
signals are a combined left x1=/, a combined right x2=rand
a (scaled) combined center/lfe channel x3=c. The downmix
matrix is

and the prediction matrix is constructed from two
transmitted real parameters c1,c2, according to

Under the assumption that the original channels are
uncorrelated the prediction matrix realizing the minimal
prediction error is given by

Equating Cmodel=C leads to the (unnormalized) energy
distribution taught by the present invention

where or = (l-c,)/3, /? = (l-c2)/3, a = a + jd, and p = afi. This
holds in the viable range defined by

in which case the prediction error can be found in the
same scaling from

Since P = 3 = 2 + l = M + l, the method outlined by (16)-(18) is
applicable. The unit vector is [v,,v2,v3] = [l,l,-l]/v3 and with
the definitions

the compensation gain for each ear « = 1,2 as computed in a
preferred embodiment of the gain calculator 302 can be
expressed by

Here e>0 is a small number whose purpose is to stabilize
the formula near the edge of the viable parameter range
and gmax is an upper limit on the applied compensation
gain. The gains of (27) are different for the left and
right ears, « = 1,2. A variant of the method is to use a
common gain go=gl=g, where

The inventive correction gain factor can be brought into
coexistence with a straight-forward multichannel gain
compensation available without any HRTF related issues.

In MPEG Surround, compensation for the prediction loss is
already applied in the decoder by multiplying the upmix
matrix C by a factor 1/p where 0

transmitted spatial parameters. In that case the gains of
(27) and (28) have to be replaced by the products pgn and
pgrespectively. Such compensation is applied for the
binaural decoding studied in Figs. 5 and 6. It is the
reason why the prior art decoding of Fig. 5 has boosted
parts of the spectrum in comparison to the reference. For
the subbands corresponding to those frequency regions, the
inventive gain compensation effectively replaces the
transmitted parameter gain factor l//?with a smaller value
derived from formula (28).
In addition, since the case where p = \ corresponds to a
successful prediction, a more conservative variant of the
gain compensation taught by the present invention will
disable the binaural gain compensation forp=l.
Furthermore, the present invention is used together with a
residual signal. In MPEG Surround, an additional prediction
residual signal z3 can be transmitted which makes it
possible to reproduce the original P=3 signals xpmore
faithfully. In this case the gain compensation is to be
replaced by a binaural residual signal addition which will
now be outlined. The predictive upmix enhanced by a
residual is formed according to

where [w„w2,vc3] = [l,l,-l]/3. Substituting #£ for xp in (5)
yields the corresponding combined filtering,

where the combined filters hn,mare defined by (7) for
/w = l,2, and the combined filters for the residual addition
are defined by

The overall structure of this mode of decoding is
therefore also described by Fig. 2 by setting P = M = 3, and
by modifying the combiner 203 to perform only the linear
combination defined by (7) and (31).
Fig. 13 illustrates in a modified representation the result
of the linear combiner 301 in Fig. 3. The result of the
combiner are four HRTF-based filters hn, hi2, h2i and h22.
As will be clearer from the description of Fig. 16a and
Fig. 17, these filters correspond to filters indicated by
15, 16, 17 ,18 in Fig. 16a.
Fig. 16a shows a head of a listener having a left ear or a
left binaural point and having a right ear or a right
binaural point. When Fig. 16a would only correspond to a
stereo scenario, then filters 15, 16, 17, 18 would be
typical head related transfer functions which can be
individually measured or obtained via the Internet or in
corresponding textbooks for different positions between a
listener and the left channel speaker and the right channel
speaker.
However, since the present invention is directed to a
multi-channel binaural decoder, filters illustrated by 15,
16, 17, 18 are not pure HRTF filters, but are HRTF-based
filters, which not only reflect HRTF properties but which
also depend on the spatial parameters and, particularly, as
discussed in connection with Fig. 2, depend on the spatial
parameter set 1 and the spatial parameter set 2.

Fig. 14 shows the basis for the HRTF-based filters used in
Fig. 16a. Particularly, a situation is illustrated where a
listener is positioned in a sweet spot between five
speakers in a five channel speaker setup which can be
found, for example, in typical surround home or cinema
entertainment systems. For each channel, there exist two
HRTFs which can be converted to channel impulse responses
of a filter having the HRTF as the transfer function.
Particularly as it is known in the art, an HRTF-based
filter accounts for the sound propagation within the head
of a person so that, for example, HRTF1 in Fig. 14 accounts
for the situation that a sound emitted from speaker Ls
meets the right ear after having passed around the head of
the listener. Contrary thereto, the sound emitted from the
left surround speaker Ls meets the left ear almost directly
and is only partly affected by the position of the ear at
the head and also the shape of the ear etc. Thus, it
becomes clear that the HRTFs 1 and 2 are different from
each other.
The same is true for the HRTFs 3 and 4 for the left
channel, since the relations of both ears to the left
channel L are different. This also applies for all other
HRTFs, although as becomes clear from Fig. 14, the HRTFs 5
and 6 for the center channel will be almost identical or
even completely identical to each other, unless the
individual listeners asymmetry is accommodated by the HRTF
data.
As stated above, these HRTFs have been determined for model
heads and can be downloaded for any specific "average
head", and loudspeaker setup.
Now, as becomes clear at 171 and 172 in Fig. 17, a
combination takes place to combine the left channel and the
left surround channel to obtain two HRTF-based filters for
the left side indicated by L' in Fig. 15. The same

procedure is performed for the right side as illustrated by
R' in Fig. 15 which results in HRTF 13 and HRTF 14. To this
end, reference is also made to item 173 and item 174 in
Fig. 17. However, it is to be noted here that, for
combining respective HRTFs in items 171, 172, 173 and 174,
inter channel level difference parameters reflecting the
energy distribution between the L channel and the Ls
channel of the original setup or between the R channel and
the Rs channel of the original multi-channel setup are
accounted for. Particularly, these parameters define a
weighting factor when HRTFs are linearly combined.
As outlined before, a phase factor can also be applied when
combining HRTFs, which phase factor is defined by time
delays or unwrapped phase differences between the to be
combined HRTFs. However, this phase factor does not depend
on the transmitted parameters.
Thus, HRTFs 11, 12, 13 and 14 are not true HRTFs filters
but are HRTF-based filters, since these filters not only
depend from the HRTFs, which are independent from the
transmitted signal. Instead, HRTFs 11, 12, 13 and 14 are
also dependent on the transmitted signal due to the fact
that the channel level difference parameters cldi and cldr
are used for calculating these HRTFs 11, 12, 13 and 14.
Now, the Fig. 15 situation is obtained, which still has
three channels rather than two transmitted channels as
included in a preferred down-mix signal. Therefore, a
combination of the six HRTFs 11, 12, 5, 6, 13, 14 into four
HRTFs 15, 16, 17, 18 as illustrated in Fig. 16a has to be
done.
To this end, HRTFs 11, 5, 13 are combined using a left
upmix rule, which becomes clear from the upmix matrix in
Fig. 16b. Particularly the left upmix rule as shown in Fig
16b and as indicated in block 175 includes parameters nin,
m21 and m31. This left upmix rule is in the matrix equation

of Fig. 16 only for being multiplied by the left channel.
Therefore, these three parameters are called the left upmix
rule.
As outlined in block 176, the same HRTFs 11, 5, 13 are
combined, but now using the right upmix rule, i.e., in the
Fig. 16b embodiment, the parameters m12, m22 and m32, which
all are used for being multiplied by the right channel Ro
in Fig. 16b.
Thus, HRTF 15 and HRTF 17 are generated. Analogously HRTF
12, HRTF 6 and HRTF 14 of Fig. 15 are combined using the
upmix left parameters m11, m21 and m31 to obtain HRTF 16. A
corresponding combination is performed using HRTF 12, HRTF,
6 HRTF 14, but now with the upmix right parameters or right
upmix rule indicated by m12, m22 and m32 to obtain HRTF 18 of
Fig. 16a.
Again, it is emphasized that, while original HRTFs in Fig.
14 did not at all depend on the transmitted signal, the new
HRTF-based filters 15, 16, 17, 18 now depend on the
transmitted signal, since the spatial parameters included
in the multi-channel signal were used for calculating these
filters 15, 16, 17 and 18.
To finally obtain a binaural left channel LB and a binaural
right channel RB, the outputs of filters 15 and 17 have to
be combined in an adder 130a. Analogously, the output of
the filters 16 and 18 have to be combined in an adder 130b.
These adders 130a, 130b reflect the superposition of two
signals within the human ear.
Subsequently, Fig. 18 will be discussed. Fig. 18 shows a
preferred embodiment of an inventive multi-channel decoder
for generating a binaural signal using a downmix signal
derived from an original multi-channel signal. The downmix
signal is illustrated at z1 and Z2 or is also indicated by
"L" and "R". Furthermore, the downmix signal has parameters

associated therewith, which parameters are at least a
channel level difference for left and left surround or a
channel level difference for right and right surround and
information on the upmixing rule.
Naturally, when the original multi-channel signal was only
a three-channel signal, cld1 or cldr are not transmitted
and the only parametric side information will be
information on the upmix rule which, as outlined before, is
such an upmix rule which results in an energy-error in the
upmixed signal. Thus, although the waveforms of the upmixed
signals when a non-binaural rendering is performed, match
as close as possible the original waveforms, the energy of
the upmixed channels is different from the energy of the
corresponding original channels.
In the preferred embodiment of Fig. 18, the upmix rule
information is reflected by two upmix parameters cpci,
cpc2. However, any other upmix rule information could be
applied and signaled via a certain number of bits.
Particularly, one could signal certain upmix scenarios and
upmix parameters using a predetermined table at the decoder
so that only the table indices have to be transmitted from
an encoder to the decoder. Alternatively, one could also
use different upmixing scenarios such as an upmix from two
to more than three channels. Alternatively, one could also
transmit more than two predictive upmix parameters which
would then require a corresponding different downmix rule
which has to fit to the upmix rule as will be discussed in
more detail with respect to Fig. 20.
Irrespective of such a preferred embodiment for the upmix
rule information, any upmix rule information is sufficient
as long as an upmix to generate an energy-loss affected set
of upmixed channels is possible, which is waveform-matched
to the corresponding set of original signals.

The inventive multi-channel decoder includes a gain factor
calculator 180 for calculating at least one gain factor gi,
gr or g, for reducing or eliminating the energy-error. The
gain factor calculator calculates the gain factor based on
the upmix rule information and filter characteristics of
HRTF-based filters corresponding to upmix channels which
would be obtained, when the upmix rule would be applied.
However, as outlined before, in the binaural rendering,
this upmix does not take place. Nevertheless, as discussed
in connection with Fig. 15 and blocks 175, 176, 177, 178 of
Fig. 17, HRTF-based filters corresponding to these upmix
channels are nevertheless used.
As discussed before, the gain factor calculator 180 can
calculate different gain factors gi and gr as outlined in
equation (27), when, instead of n, 1 or r is inserted.
Alternatively, the gain factor calculator could generate a
single gain factor for both channels as indicated by
equation (28) .
Importantly, the inventive gain factor calculator 180
calculates the gain factor based not only on the upmix
rule, but also based on the filter characteristics of the
HRTF-based filters corresponding to upmix channels. This
reflects the situation that the filters themselves also
depend on the transmitted signals and are also affected by
an energy-error. Thus, the energy-error is not only caused
by the upmix rule information such as the prediction
parameters CPCi, CPC2, but is also influenced by the
filters themselves.
Therefore, for obtaining a well-adapted gain correction,
the inventive gain factor not only depends on the
prediction parameter but also depends on the filters
corresponding to the upmix channels as well.
The gain factor and the downmix parameters as well as the
HRTF-based filters are used in the filter processor 182 for

filtering the downmix signal to obtain an energy-corrected
binaural signal having a left binaural channel LB and
having a right binaural channel RB.
In a preferred embodiment, the gain factor depends on a
relation between the total energy included in the channel
impulse responses of the filters corresponding to upmix
channels to a difference between this total energy and an
estimated upmix energy error AE. AE can preferably be
calculated by combining the channel impulse responses of
the filters corresponding to upmix channels and to then
calculating the energy of the combined channel impulse
response. Since all numbers in the relations for GL and GR
in Fig. 18 are positive numbers, which becomes clear from
the definitions for AE and E, it is clear that both gain
factors are larger than 1. This reflects the experience
illustrated in Fig. 5 that, in most times, the energy of
the binaural signal is lower than the energy of the
original multi-channel signal. It is also to note, that
even when the multi-channel gain compensation is applied,
i.e., when the factor p is used in most signals,
nevertheless an energy-loss is caused.
Fig. 19a illustrates a preferred embodiment of the filter
processor 182 of Fig. 18. Particularly, Fig. 19a
illustrates the situation, when in block 182a the combined
filters 15, 16, 17, and 18 of Fig. 16a without gain
compensation are used and the filter output signals are
added as outlined in Fig. 13. Then, the output of box 182a
is input into a scaler box 182b for scaling the output
using the gain factor calculated by box 180.
Alternatively, the filter processor can be constructed as
shown in Fig. 19b. Here, HRTFs 15 to 18 are calculated as
illustrated in box 182c. Thus, the calculator 182c performs
the HRTF combination without any gain adjustment. Then, a
filter adjuster 182d is provided, which uses the
inventively calculated gain factor. The filter adjuster

results in adjusted filters as shown in block 180e, where
block 180e performs the filtering using the adjusted filter
and performs the subsequent adding of the corresponding
filter output as shown in Fig. 13. Thus, no post-scaling as
in Fig 19a is necessary to obtain gain-corrected binaural
channels LB and RB.
Generally, as has been outlined in connection with equation
16, equation 17 and equation 18, the gain calculation takes
place using the estimated upmix error AE. This
approximation is especially useful for the case where the
number of upmix channels is equal to the number of downmix
channels +1. Thus, in case of two downmix channels, this
approximation works well for three upmix channels.
Alternatively, when one would have three downmix channels,
this approximation would also work well in a scenario in
which there are four upmix channels.
However, it is to be noted that the calculation of the gain
factor based on an estimation of the upmix error can also
be performed for scenarios in which for example, five
channels are predicted using three downmix channels.
Alternatively, one could also use a prediction-based upmix
from two downmix channels to four upmix channels. Regarding
the estimated upmix energy-error AE, one can not only
directly calculate this estimated error as indicated in
equation (25) for the preferred case, but one could also
transmit some information on the actually occurred upmix
error in a bit stream. Nevertheless, even in other cases
than the special case as illustrated in connection with
equations (25) to (28), one could then calculate the value
EnB based on the HRTF-based filters for the upmix channels
using prediction parameters. When equation (26) is
considered, it becomes clear that this equation can also
easily be applied to a 2/4 prediction upmix scheme, when
the weighting factors for the energies of the HRTF-based
filter impulse responses are correspondingly adapted.

In view of that, it becomes clear that the general
structure of equation (27), i.e., calculating the gain
factor based on relation of EB/(EB - AEB) also applies for
other scenarios.
Subsequently, Fig. 20 will be discussed to show a schematic
implementation of a prediction-based encoder which could be
used for generating the downmix signal L, R and the upmix
rule information transmitted to a decoder so that the
decoder can perform the gain compensation in the context of
the binaural filter processor.
A downmixer 191 receives five original channels or,
alternatively, three original channels as illustrated by
(Ls and Rs) . The downmixer 191 can work based on a pre-
determined downmix rule. In that case, the downmix rule
indication as illustrated by line 192 is not required.
Naturally, the error- minimizer 193 could vary the downmix
rule as well in order to minimize the error between
reconstructed channels at the output of an upmixer 194 with
respect to the corresponding original input channels.
Thus, the error-minimizer 193 can vary the downmix rule 192
or the upmixer rule 196 so that the reconstructed channels
have a minimum prediction loss AE. This optimization
problem is solved by any of the well-known algorithms
within the error-minimizer 193, which preferably operates
in a subband-wise way to minimize the difference between
the reconstruction channels and the input channels.
As stated before, the input channels can be original
channels L, Ls, R, Rs, C. Alternatively the input channels
can only be three channels L, R, C, wherein, in this
context, the input channels L, R, can be derived by
corresponding OTT boxes illustrated in Fig. 11.
Alternatively, when the original signal only has channels
L, R, C, then these channels can also be termed as
"original channels".

Fig. 20 furthermore illustrates that any upmix rule
information can be used besides the transmission of two
prediction parameters as long as a decoder is in the
position to perform an upmix using this upmix rule
information. Thus, the upmix rule information can also be
an entry into a lookup table or any other upmix related
information.
The present invention therefore, provides an efficient way
of performing binaural decoding of multi-channel audio
signals based on available downmixed signals and additional
control data by means of HRTF filtering. The present
invention provides a solution to the problem of spectral
coloring arising from the combination of predictive upmix
with binaural decoding.
Depending on certain implementation requirements of the
inventive methods, the inventive methods can be
implemented in hardware or in software. The implementation
can be performed using a digital storage medium, in
particular a disk, DVD or a CD having electronically
readable control signals stored thereon, which cooperate
with a programmable computer system such that the
inventive methods are performed. Generally, the present
invention is, therefore, a computer program product with a
program code stored on a machine readable carrier, the
program code being operative for performing the inventive
methods when the computer program product runs on a
computer. In other words, the . inventive methods are,
therefore, a computer program having a program code for
performing at least one of the inventive methods when the
computer program runs on a computer.
While the foregoing has been particularly shown and
described with reference to particular embodiments thereof,
it will be understood by those skilled in the art that
various other changes in the form and details may be made

without departing from the spirit and scope thereof. It is
to be understood that various changes may be made in
adapting to different embodiments without departing from
the broader concepts disclosed herein and comprehended by
the claims that follow.

Claims
1. Multi-channel decoder for generating a binaural signal
from a downmix signal derived from an original multi-
channel signal using parameters including an upmix
rule information useable for upmixing the downmix
signal with an upmix rule, the upmix rule resulting in
an energy-error, comprising:
a gain factor calculator for calculating at least one
gain factor for reducing or eliminating the energy-
error, based on the upmix rule information and filter
characteristics of a head related transfer function
based filters corresponding to upmix channels; and
a filter processor for filtering the downmix signal
using the at least one gain factor, the filter
characteristics and the upmix rule information to
obtain an energy-corrected binaural signal.
2. Multi-channel decoder of claim 1, in which the filter
processor is operative to calculate filter
coefficients for two gain adjusted filters for each
channel of the downmix signal and to filter the
downmix channel using each of the two gain adjusted
filters.
3. Multi-channel decoder of claim 1, in which the filter
processor is operative to calculate filter
coefficients for two filters for each channel of the
downmix channel without using the gain factor and to
filter the downmix channels and to gain adjust
subsequent to filtering the downmix channel.
4. Multi-channel decoder of claim 1, in which the gain
factor calculator is operative to calculate the gain
factor based on an energy of a combined impulse
response of the filter characteristics, the combined

impulse response being calculated by adding or
subtracting individual filter impulse responses.
5. Multi-channel decoder of claim 1, in which the gain
factor calculator is operative to calculate the gain
factor based on a combination of powers of individual
filter impulse responses.
6. Multi-channel decoder of claim 5, in which the gain
factor calculator is operative to calculate the gain
factor based on a weighted addition of powers of
individual filter impulse responses, wherein weighting
coefficients used in the weighted addition depend on
the upmix rule information.
7. Multi-channel decoder of claim 1, in which the gain
factor calculator is operative to calculate the gain
factor based on an expression having a numerator and a
denominator, the numerator having a combination of
powers of individual filter impulse filter responses,
and the denominator having a weighted addition of
powers of individual filter impulse responses, wherein
weighting coefficients used in the weighted addition
depend on the upmix rule information.
8. Multi-channel decoder of claim 1, in which the gain
factor calculator is operative to calculate the gain
factor based on the following equation:

wherein gn is the gain factor for the first channel,
when n is set to 1, wherein g2 is the gain factor of a
second channel, when n is set to 2, wherein EnB is a
weighted addition energy calculated by weighting

energies of channel impulse responses using weighting
parameters, and wherein AEn is an estimate for the
energy error introduced by the upmix rule, wherein a,
(3 and y are upmix rule dependent parameters, and
wherein s is a number greater than or equal to zero.
9. Multi-channel decoder of claim 8, in which the gain
factor calculator is operative to calculate En and AEn
based on the following equation:

in which bn, 1 is an HRTF-based filter corresponding to
first upmix channel and a nth binaural channel,
wherein bn,2 is a HRTF-based filter impulse response
corresponding to a second upmix channel and a nth
binaural channel, wherein bn,3 is a HRTF-based filter
impulse response corresponding to a third upmix
channel for a nth binaural channel,
wherein the following definitions are valid

wherein c1 is a first prediction parameter, C2 is a
second prediction parameter, and wherein the first
prediction parameter and the second prediction
parameter constitute the upmix rule information.
10. Multi-channel decoder of claim 1, in which the gain
factor calculator is operative to calculate a common
gain factor for a left binaural channel and a right
binaural channel.

11. Multi-channel decoder of claim 1, in which the filter
processor is operative to use, as the filter
characteristics, the head related transfer function
based filters for the left binaural channel and the
right binaural channel for virtual center, left and
right positions or to use filter characteristics
derived by combining HRTF filters for a virtual left
front position and a virtual left surround position or
by combining HRTF filters for a virtual right front
position and a virtual right surround position.
12. Multi-channel decoder of claim 11, in which parameters
relating to original left and left surround channels
or original right and right surround channels are
included in a decoder input signal, and
wherein the filter processor is operative to use the
parameters for combining the head related transfer
function filters.
13. Multi-channel decoder of claim 1, in which the gain
factor calculator is operative to calculate the gain
factor for a binaural channel based on a ratio of a
weighted linear combination of energies of channel
impulse responses of the HRTF-based filters for the
binaural channel and a value obtained by subtracting
an estimated energy-error from the weighted linear
combination.
14. Multi-channel decoder of claim 13, in which the gain
factor calculator is operative to use information on
the upmix rule for determining the weighting factor.
15. Multi-channel decoder of claim 14, in which the upmix
rule information includes at least two prediction
parameters usable for constructing an upmix matrix
resulting in output channels having the energy-error
with respect to corresponding three input channels.

16. Multi-channel decoder of claim 1, in which the filter
processor is operative to have, as filter
characteristics,
a first filter for filtering a left downmix channel
for obtaining a first left binaural output,
a second filter for filtering a right downmix channel
for obtaining a second left binaural output,
a third filter for filtering a left downmix channel
for obtaining a first right binaural output,
a fourth filter for filtering a right downmix channel
for obtaining a second right binaural output,
an adder for adding the first left binaural output and
the second left binaural output to obtain a left
binaural channel and for adding the first right
binaural output and the second right binaural output
to obtain a right binaural channel,
wherein the filter processor is operative to apply a
gain factor for the left binaural channel to the first
and the second filters or to the left binaural output
before or after adding and to apply the gain factor
for the right binaural channel to the third filter and
to the fourth filter or to the right binaural output
before or after adding.
17. Multi-channel decoder of claim 1, in which the upmix
rule information includes upmix parameters usable for
constructing an upmix matrix resulting in an upmix
from two to three channels.
18. Multi-channel decoder of claim 17, in which the upmix
rule is defined as follows:

wherein L is a first upmix channel, R is a second
upmix channel, and C is a third upmix channel, L0 is a
first downmix channel, R0 is a second downmix channel,
and mij are upmix rule information parameters.
19. Multi-channel decoder of claim 1, in which a
prediction loss parameter is included in a multi-
channel decoder input signal, and
in which a filter processor is operative to scale the
gain factor using the prediction loss parameter.
20. Multi-channel decoder of claim 1, in which the gain
calculator is operative to calculate the gain factor
subband-wise, and
in which the filter processor is operative to apply
the gain factor subband-wise-
21. Multi-channel decoder of claim 11, in which the filter
processor is operative to combine HRTF filters
associated with two channels by adding weighted or
phase shifted versions of channel impulse responses of
the HRTF filters, wherein weighting factors for
weighting the channel impulse responses is of the HRTF
filters depend on a level difference between the
channels, and an applied phase shift depends on a time
delay between the channel impulse responses of the
HRTF filters.
22. Multi-channel decoder of claim 1, in which filter
characteristics of HRTF-based filters or HRTF filters

are complex subband filters obtained by filtering a
real-valued filter impulse response of an HRTF filter
using a complex-exponential modulated filterbank. .
23. Method of multi-channel decoding for generating a
binaural signal from a downmix signal derived from an
original multi-channel signal using parameters
including an upmix rule information useable for
upmixing the downmix signal with an upmix rule, the
upmix rule resulting in an energy-error, comprising:
calculating at least one gain factor for reducing or
eliminating the energy-error, based on the upmix rule
information and filter characteristics of a head
related transfer function based filters corresponding
to upmix channels; and
filtering the downmix signal using the at least one
gain factor, the filter characteristics and the upmix
rule information to obtain an energy-corrected
binaural signal.
24. Computer program having a program code for performing
the method in accordance with claim 23, when the
computer program runs on a computer.

A multi-channel decoder for generating a binaural signal
from a downmix signal using upmix rule information on an
energy-error introducing upmix rule for calculating a gain
factor (180) based on the upmix rule information and
characteristics of head related transfer function based
filters corresponding to upmix channels. The one or more
gain factors are used by a filter processor (182) for
filtering the downmix signal so that an energy corrected
binaural signal having a left binaural channel and a right
binaural channel is obtained.

Documents:

http://ipindiaonline.gov.in/patentsearch/GrantedSearch/viewdoc.aspx?id=cCbcMOpToteogdpjafNBSQ==&loc=wDBSZCsAt7zoiVrqcFJsRw==

« Previous Patent

Next Patent »

Patent Number

270243

Indian Patent Application Number

4739/KOLNP/2008

PG Journal Number

49/2015

Publication Date

04-Dec-2015

Grant Date

04-Dec-2015

Date of Filing

21-Nov-2008

Name of Patentee

DOLBY INTERNATIONAL AB

Applicant Address

APOLLO BUILDING, 3E HERIKERBERGWEG 1-35, 1101 CN AMSTERDAM ZUID-OOST, NETHERLANDS

Inventors:

#	Inventor's Name	Inventor's Address
1	VILLEMOES, LARS	MANDOLINVAGEN 22 175 56 JARFALLA

PCT International Classification Number

G10L 19/00,H04S 7/00

PCT International Application Number

PCT/EP2006/008613

PCT International Filing date

2006-09-04

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1	60/803,819	2006-06-02	U.S.A.