Title of Invention	APPARATUS AND METHOD FOR SYNTHESIZING THREE OUTPUT CHANNELS USING TWO INPUT CHANNELS
Abstract	For synthesizing at least three output channels using two stereo input channels, the stereo input channels are analyzed (15) to detect signal components occurring in both input channels. A signal generator (16) is operative to introduce at least a part of the detected signal components into the second channel (12b) associated with a second speaker in an intended speaker scheme, which is positioned between a first and a third speaker in the speaker scheme. When, however, feeding of the complete detected signal components would result in a clipping situation, then only a part of the detected signal components is fed into the second channel as a real center channel and the remainder is located in the first and third channels as a phantom center channel.

Title of Invention

APPARATUS AND METHOD FOR SYNTHESIZING THREE OUTPUT CHANNELS USING TWO INPUT CHANNELS

Abstract

For synthesizing at least three output channels using two stereo input channels, the stereo input channels are analyzed (15) to detect signal components occurring in both input channels. A signal generator (16) is operative to introduce at least a part of the detected signal components into the second channel (12b) associated with a second speaker in an intended speaker scheme, which is positioned between a first and a third speaker in the speaker scheme. When, however, feeding of the complete detected signal components would result in a clipping situation, then only a part of the detected signal components is fed into the second channel as a real center channel and the remainder is located in the first and third channels as a phantom center channel.

Full Text	Apparatus and method for synthesizing three output channels using two input channels Specification The present invention is related to multi-channel synthesiz- ers and, particularly, to devices generating three or more output channels using two stereo input channels. Multi-channel audio material is becoming more and more popu- lar also in the consumer home environment. This is mainly due to the fact that movies on DVD offer 5.1 multi-channel sound and therefore even home users frequently install audio play- back systems, which are capable of reproducing multi-channel audio. Such a setup consists e.g. of 3 speakers L, C, R in the front, 2 speakers Ls, Rs in the back and a low frequency enhancement channel LFE and provides several well-known ad- vantages over 2-channel stereo reproduction, e.g.: improved front image stability even outside of the op- timal central listening position due to the Center channel (larger "sweet-spot" = optimum listening posi- tion) increased sense of listener "involvement" created by the rear speakers. Nevertheless, there exists a huge amount of legacy audio con- tent, which consists only of two ("stereo") audio channels, e.g. on Compact Discs (CDs). To play back two-channel legacy audio material over a 5.1 multi-channel setup there are two basic options: 1. Reproduce the left and right channel stereo signals over the L and R speakers, respectively, i.e., play it back in the legacy way. This solution does not take advantage of the extended loudspeaker setup (Center and rear loudspeakers). 2. One may use a method to convert the two channels of the content material to a multi-channel signal (this may happen "on the fly" or by means of preprocessing) that makes use of all the 5.1 speakers and in this way benefits from the previously discussed advantages of the multi-channel setup. Solution #2 clearly has advantages over #1, but also contains some problems especially with respect to the conversion of the two front channels (Left and Right = LR) to three front channels (Multi-channel Left, Center and Right = L'C'R'). A good LR to L'C'R' conversion solution should fulfill the following requirements: 1) To recreate a similar, but more stable front image in the L'C'R' than in the LR playback case, The Center channel shall reproduce all the sound events which usu- ally are perceived to come from the middle between the Left and Right loudspeaker, if the listener is in the "sweet spot". Furthermore, signals in left front posi- tions shall be reproduced by L'C, and signals in the right front positions shall be reproduced by R'C, re- spectively (see J.M. Jot and C. Avendano, "Spatial En- hancement of Audio Recordings", AES 23rd Conference, Copenhagen, 2003). 2) The sum of the acoustical energy emitted by the chan- nels L'C'R' should be equal to the sum of the acousti- cal energy of the source channels LR in order to achieve an equally loud sound impression for L'C'R as for LR. Assuming equal characteristics in all reproduc- tion channels, this translates into "the sum of the electrical energy of the channels L'C'R' should be equal to the sum of the electrical energy of the source channels LR." Due to requirement #1 the signals of the Left and Right chan- nels may be mixed into one (single) center channel. This is particularly true, if the Left and the Right channel signals are near identical, i.e. they represent a phantom sound source in the middle of the front sound stage. This phantom image is now replaced by a "real" image generated by the Cen- ter speaker. Due to requirement #2, this Center signal shall carry the sum of the Left and the Right energy. If the level of the Left or the Right channel signals is close to the maximum amplitude that can be transmitted by the channel (= 0 dBFS; dBFS = dB Full Scale), the sum of the levels of both channels will exceed the maximum level, which can be repre- sented by the channel/system. This usually results in the un- desirable effect of "clipping". The clipping situation is shown in Fig. 6. Fig. 6 illustrates a time waveform of a signal 60 processed by a processor hav- ing a maximum positive threshold 61a and a maximum negative threshold 61b. Depending on the capability of the digital processor processing the digital signal, the maximum positive threshold and the maximum negative thresholds may be +1 and - 1. Alternatively, when a digital processor is used represent- ing the numbers in integers, the maximum positive threshold will be 32768 corresponding to 215, and the maximum negative threshold will be -32768 corresponding to -215. Since a time waveform signal is represented by a sequence of samples, each sample being a digital number between -32768 and +32768, it is easily clear that higher numbers can be ob- tained, when, for a certain time instance, the first channel has a quite high value and the second channel also has a quite high value, and when these quite high values are added together. Theoretically, the maximum number obtained by this adding together of two channels can be 65536. However, the digital signal processor is not able to represent this high number. Instead, the digital processor will only represent numbers equal to the maximum positive threshold or the maxi- mum negative threshold. Therefore, the digital signal proces- sor performs clipping in that a number higher or equal to the maximum positive threshold or the maximum negative threshold is replaced by a number equal to the maximum positive thresh- old and the maximum negative threshold so that, with regard to Fig. 6., the illustrated situation appears. Within a clip- ping time portion 62, the waveform 60 does not have its natu- ral (sine) shape, but is flattened or clipped. When this clipped waveform is evaluated from a spectral point of view, it becomes clear that this time domain clipping results in strong harmonic components caused by a high gradient magni- tude at the beginning and the end of the clipping time por- tion 62. This "digital clipping" is not related to the replay setup, i.e., the amplifier and the loudspeakers used for rendering the audio signal. However, each amplifier/loudspeaker combi- nation also has only a limited linear range, and, when this linear range is exceeded by a processed signal, also a kind of clipping takes place, which can be avoided using the in- ventive concept. In any case, the occurrence of clipping introduces heavy dis- tortions in the audio signal, which degrade the perceived sound quality very much. Thus, the occurrence of clipping has to be avoided. This is even more due to the fact that the sound improvement by rendering a stereo signal by a multi- channel setup such as a 5.1 speaker system is small compared to the very annoying clipping distortions. Therefore, when one cannot guaranty that clipping does not occur, one would prefer to only use the left and the right speakers of a multi-channel setup for rendering a stereo signal. There exist prior art solutions to overcome this clipping problem. A simple solution to overcome this problem is to scale down all channels equally to a level where none of the channel signal (especially the Center signal) exceeds the 0 dBFS limit. This can be done statically by a predefined fixed value. In this case the fixed value must also be valid for worst case situations, where the Left and Right channel have maximum levels. For the average LR to L'C'R' conversion this leads to a significantly quieter L'C'R' version than the original stereo LR, which is undesirable, especially when us- ers are switching between stereo and multi-channel reproduc- tion. This behavior can be observed at commercially available matrix decoders (Dolby ProLogicII and Logic7 Decoder) that can be used as LR to L'C'R' converters. See Dolby Publica- tion: "Dolby Surround Pro Logic II Decoder - Principles of Operation", htp://www.dolby.com/assets/pdf/tech_library/209 _Dolby_Surround_Pro_Logic_II_Decoder_Principles_of_Operation. pdf or Griesinger, D.: "Multichannel Matrix Surround Decoders for Two-Eared Listeners", 101st AES Convention, Los Angeles, USA, 1996, Preprint 4402. Another simple solution is to use dynamic range compression in order to dynamically (depending on the signal) limit the peak signal, sometimes also called a "limiter". A disadvan- tage of this approach is that the true dynamic range of the audio program is not reproduced but subjected to compression (see Digital Audio Effects DAFX; Udo Zolzer, Editor; 2002; Wiley & Sons; p. 99ff: "Limiter"). The downscaling problem is undesirable, since it reduces the level or volume of a sound signal compared to the level of the original signal. In order to completely avoid any even theoretical occurrence of clipping, one would have to down- scale all channels by a scaling factor equal to 0.5. This re- sults in a strongly reduced output level of the multi-channel signal compared to the original signal. When one only listens to this downscaled multi-channel signal, one can compensate for this level reduction by increasing the amplification of the sound amplifier. However, when one switches between sev- eral sources, the (legacy) stereo signal will appear to a listener very loud, when it is replayed using the same ampli- fication setting of the amplifier a set for the multichannel reproduction. Thus, a user would have to think about reducing the amplifi- cation setting of its amplifier before switching from a multi-channel representation of a stereo signal to a true stereo representation of the stereo signal in order to not damage her or his ears or equipment. The other prior art method using dynamic range compression effectively avoids clipping. However, the audio signal itself is changed. Thus, the dynamic compression leads to a non- authentic audio signal, which, even when the introduced arti- facts are not too annoying, is questionable from the authen- ticity point of view. It is an object of the present invention to provide an im- proved concept for multi-channel synthesis using two input channels. This object is achieved by an apparatus for synthesizing in accordance with claim 1, a method of synthesizing in accor- dance with claim 14, a computer program in accordance with claim 15 or a three channel representation in accordance with claim 16. The present invention is based on the finding that, for over- coming the clipping problem and for nevertheless achieving the advantages incurred by replaying a stereo signal using three or more channels of a multi-channel setup, the center channel is generated as usual, i.e., receives sound events located in the middle between the left and the right loud- speakers, which is also called a "real center" rendering. However, when the real center would come into the clipping range, only a portion of the energy of the signal components representing the events in the middle of the audio setup are fed into the center channel. The remainder of the energy of these sound events is fed back into the first and third (or left and right) channels or remains there from the beginning. Thus, for a time frame, where clipping may occur, when the two/three upmix procedure is performed without modifications, the center channel is scaled down the level below or equal to the maximum level possible without clipping. Nevertheless, the missing part/energy of the signal, which cannot be ren- dered by the center channel is reproduced with the left chan- nel and the right channel as a "virtual center" or "phantom center". The signal of the real center and the virtual center is then acoustically combined during playback recreating an intended center without clipping. This "mixing" of the real center and the virtual center results in an improved more stable front image of a stereo audio signal, i.e., in an increased sweet spot, although the sweet spot is not as large as when there would not be a phantom center at all. However, the inventive process does not have any clipping artifacts, since the re- mainder of the energy not being processable within the second channel due to the clipping problem is not lost but is ren- dered by the original left and right channels. It is noted here that, for any situations, the energy of the left and right channels in the multi-channel setup is lower than the energy in the original left and right channels, since the energy of the center channel is drawn from the left and right channels. Therefore, even when, in accordance with the present invention, a remaining part of the energy is fed back to the left and right output channels, there will never exist a clipping problem within these channels. A further advantage of the present invention is that the in- ventive signal generation is performed in a way that, in a preferred embodiment, the total electrical or acoustical en- ergy of the generated three output channels (and optionally generated additional output channels such as Ls, Rs, Cs, LFE, ...) is preserved with respect to the energy of the original stereo signal. The same overall loudness irrespective of the way of rendering the signal, i.e., whether the signal is ren- dered using a stereo setup having only two speakers or whether the signal is rendered using a multi-channel setup having more than two speakers, can be guaranteed. Furthermore, the inventive signal generation and distribution of sound energy to the center channel and the left and right channels is dynamically applied only if clipping would be un- avoidable, i.e., the second center channel is completely un- changed in situations, which are not effected by clipping, i.e., when sampling values of the second channel remain below or are only equal to the maximum threshold. Furthermore, the resulting acoustic combination of the "real center" and the "phantom center" produces a signal which is much closer to the optimal three channel configuration, i.e., three channels without clipping or three channels in which sampling values without any min/max threshold are allowable. The inventive sound image is, therefore, in preferred embodi- ments neither different in level compared to the stereo input signal nor non-authentic as would be the case when using a limiter or a simple clipper. Preferred embodiments of the present invention are subse- quently explained with respect to the accompanying drawings, in which: Fig. 1 illustrates an apparatus for synthesizing the upper channels in accordance with the preferred embodi- ment of the present invention; Fig. 2a a preferred embodiment of the signal generator of Fig. 1 having a post processor; Fig. 2b a preferred implementation of the post processor of Fig. 2a; Fig. 3 a further embodiment of the inventive signal gen- erator having an iterative upmixer control; Fig. 4 a further embodiment of the inventive signal gen- erator completely operating in the parameter do- main; Fig. 5 an example for a 5.1 sound system optionally also having a surround center channel Cs; Fig. 6 an illustration of a clipped waveform; Fig. 7 a schematic illustration of the energy situation of the original two-channel input signal and the three-channel output signal before and after clip- ping; and Fig. 8 illustrates a preferred input channels analyzer. Fig. 1 illustrates a preferred embodiment of an inventive ap- paratus for synthesizing three output channels using two in- put channels, wherein a second channel of the three output channels is intended for a speaker in an audio replay setup, which is positioned between two speakers, which are intended to receive the first output channel and the third output channel. The input channels are indicated by 10a, which chan- nel can be for example the left channel L, and 10b for the second channel, which can be the right channel R. The output channels are indicated as 12a for the right channel, 12b for center channel and 12c for the left channel. Additional out- put channels can be generated such as a left surround output channel 14a, a right surround output channel 14b and a low frequency enhancement channel 14c. The arrangement of the corresponding speakers for these channels is shown in Fig. 5. In the middle of these speakers 12a, 12b, 12c, 14a, 14b is a sweet spot 50. When a listener is positioned within the sweet spot, then he or she will have an optimum sound impression. Additionally, one might add a center surround channel 51 Cs, which is positioned between the left surround channel 14a and the right surround channel 14b. The signal for the center surround channel 51 can be calculated using the same process as calculating the signal for the center channel 12b. Addi- tionally, the inventive methods can, therefore, also be ap- plied to the calculation of the center surround channel in order to avoid clipping in the center surround channel. It is to be noted that the inventive process is usable for each audio channel constellation, in which two input channels intended for two different spatial positions in a replay setup are used and in which three output channels are gener- ated using these two input channels, wherein the second chan- nel of the three channels is located between two additional speakers in the replay setup, which are provided with the first and the third input channel signals. The inventive synthesizer apparatus of Fig. 1 includes an in- put channel analyzer 15 for analyzing the two input channels in order to determine signal components which occur in both input channels. These signal components which occur in both input channels can be used to build the real center channel, i.e. can be rendered via the center channel C shown in Fig. 5. Typically, a stereo signal includes a lot of such mono- phonic signal components such as a speaker person or, when music signals are considered, a singer or a solo instrument positioned in front of an orchestra and, therefore, posi- tioned in front of the audience. The inventive synthesizer apparatus additionally includes a time and frequency selective and, furthermore signal depend- ent signal generator 16 for generating the three output chan- nels 12a, 12b, 12c using the two input channels 10a, 10b and information on detected signal components occurring in both input channels as provided via line 13. Particularly, the in- ventive signal generator is operative to feed detected signal components at least partly into the second channel. Further- more, the generator is operative to only feed a portion of the detected signal components in the second channel, when there exists a situation, in which a complete feeding of the detected signal components would result in exceeding the maximum threshold. Thus, the second output channel has a time portion, which only includes a part of the detected signal components to avoid clipping, while in a different portion of the second output channel, the complete detected signal components have been fed into the second output channel. The remainder of the detected signal components are included in the first and third output channels and, therefore, form the "phantom cen- ter" when these channels are rendered via the speaker setup for example shown in Fig. 5. Depending on the implementation of the inventive concept, the "portion" of the detected signal components located in the second channel, and the remainder of the detected signal com- ponents located in the first and third channels can be an en- ergy portion or frequency portion or any other portion, so that the second channel only includes a portion of the de- tected signal components and will not have any value above the maximum threshold and will, therefore, not induce any clipping distortions. Fig. 2a illustrates a preferred embodiment of the inventive signal analyzer 16 of Fig. 1. Particularly, in the Fig. 2a embodiment, the signal analyzer includes a 2-3-upmixer 16 performing an upmixing process controlled by the input chan- nels analyzer 15 of Fig. 1. The output of the 2-3-upmixer L, R, C are upmixed channels. However, channel C might be sub- ject to clipping, since channel C is generated using an add- ing process, in which signal components from the left channel and from the right channel are added together. The center channel C is input into a clipping detector 16d, which feeds a post processor 16c, which also receives infor- mation on detected signal components. Particularly, the clip- ping detector 16b is operative to examine the time wave form of the center channel 12c. Depending on the implementation, the clipping detector can be constructed in different ways. When it is assumed that the Fig. 2a signal generator can process numbers having a magni- tude being higher than a predetermined maximum threshold, then the clipping detector 16b simply examines the time wave- form to see, whether there are higher numbers than the maxi- mum threshold of the subsequent processing stage. When such a situation is detected, the post processor 16c is activated via activation line 16d to start post processing such that the energy of the center channel is reduced and the energy of the left and right channels is increased so that the three output channels 12a, 12b, 12c are finally output by the post processor 16c. Thus, in accordance with the Fig. 2a embodi- ment, the LR to LCR conversion process is done as usual. The internal first-stage center channel signal 20b is analyzed to check, whether clipping would occur if it has to be output as an external signal such as in an AES/EBU or as SPDIF format. When this happens, a part of the signal 20b is removed in the post processor 16c resulting in a modified center channel signal 12b and distributed instead to the intermediate left and right channels 20a, 20c as a "phantom center" contribu- tion. After the postprocessing, the center channel signal 12b is again below 0 dBFS. A preferred embodiment of the post processor 16c is shown in Fig. 2b. The center channel 20b after the upmixer 16a is in- put into a part extractor 25. The part extractor receives in- formation 13 on detected signal components and a control sig- nal via line 16d from the clipping detector, which may also include an indication of an amount of extraction. Alterna- tively, the amount of extraction per iteration step may be fixed independent of any occurring clipping, and an iterative trial/error process can be applied to extract increasing amounts of the detected signal components in a step-by-step fashion until the clipping detector 16b does not detect any clipping anymore. Then, the modified center channel 12b is output by the part extractor, and the remainder of the de- tected signal components corresponding to the extracted part have to be re-distributed to the left and right channels 20c, 20a output by the upmixer after multiplying by 0.5. To this end, the post processor includes two multipliers 26 in each branch or a single multiplier before branching, and a left adder 27a and a right adder 27b. When the detection of the signal components occurring in both input channels has been perfect, then the left and right channels 20a, 20c do not include any "phantom center". How- ever, by adding the extracted components (after multiplica- tion by 0.5) to these channels, a phantom center is added to the left and right channels. Subsequently, a further embodiment of the present invention and, particularly, of the signal generator 16 of Fig. 1 is discussed in connection with Fig. 3. The input channels are input into a controllable 2-3-upmixer receiving information on detected signal components for generating three output channels in a first iteration step controlled by an iteration controller 30. The first step will be equal to the upmixer operation in Fig. 2a, i.e., the center channel 20b can have clipping problems. Such a clipping situation will be detected by a clipping detector 16b. In contrast to the Fig. 2a em- bodiment, the clipping detector 16b controls the upmixer 16a in a feed-back way via the upmixer control line 31 to change the upmixing rule in a certain way so that the generated cen- ter channel 20b receives, after one or more iteration steps as controlled by the iteration controller 30, only an allowed portion of the detected signal components so that no clipping occurs anymore. Thus, the Fig. 3 embodiment illustrates an iterative process. In a first pass of the iterative process, the up-mixer opera- tion is done as usual. At the output, a detector 16b checks, whether clipping occurs. When clipping is detected, this time frame is processed again, now using the re-mapping process and using re-routing of a part of the center signal energy to the left and right channels as a phantom center contribution. The Fig. 4 embodiment completely operates in the parameter domain. To this end, an up-mixer parameter calculator 40 is provided, which is connected to a parameter changer 41. Addi- tionally, a clipping detector 42 is provided, which is opera- tive to examine the original left and right channels or the calculated up-mixer parameters to find out, whether clipping will occur or not after a straight forward up-mix process. When the clipping detector 42 detects a clipping danger, it controls a parameter change 41 via a control line 44 to pro- vide changed up-mix parameters, which are then provided to a straight-forward up-mixer 16a, which then generates the first, second, and third output channels so that no clipping occurs in the second channel and, for a time frame, in which the clipping detector 42 has originally detected a clipping problem, the left and right channels 12c, 12a, have a phantom center contribution. In contrast to the Fig. 2 and Fig. 3 embodiments, the inven- tive process is carried out based on processing parameters that are used for deriving the output signals 20a, 20b, 20c, or 12a, 12b, 12c from the input stereo signals. Thus, in or- der to provide implementations with still lower computational complexity, also the clipping detection and the manipulation of signal levels or part of it are based on the processing parameters. This is in contrast to the Fig. 2 and 3 embodi- ments, in which the inventive process is carried out on ac- tual audio channel signals that were already created for the center channel after a possible clipping could be detected. The inventive clipping detection/control can be performed by a post-processing. Thus, the intended conversion parameters are analyzed and modified according to the inventive concept to provide clipping after the synthesis of the actual output audio signals. An alternative way to control the parameter change 41 is via an iterative way. Intended conversion pa- rameters are analyzed. When, after the synthesis of the real audio signal, clipping may occur, the conversion parameters are modified. Then, the process is again started and finally, the output channel signals are synthesized without any clip- ping and with real center and phantom center contributions in the corresponding channels. Subsequently, a preferred implementation of the input chan- nels analyzer will be discussed. To this end, reference is made to Fig. 8, which illustrates such a preferred input channels analyzer 15. First of all, subsequent or overlapping frames following each other are generated using a windowing block 80 so that, at the output of block 80, there is, on line 81a, a block of values of the left channel and, on line 81b, a block of values of the right channel. Then, a frequency analysis is performed for each block individually. To this end, a frequency analyzer 82 is provided for each channel. The frequency analyzer can be any device for generating a frequency domain representation of a time domain signal. Such a frequency analyzer can include a short-time Fourier trans- form, an FFT algorithm, or an MDCT transform or any other transform device. Alternatively, the frequency analyzer block 82 may also include a subband filter bank for generat- ing for example 32 subband channels or a higher or lower num- ber of subband channels from a block of input signal values. Depending on the implementation of the subband filter bank, the functionality of the framing device 80 and the frequency analysis block 82 can be implemented in a single digitally implemented subband filter bank. Then, a band-wise cross correlation is performed as indicated by device 84. Thus, the cross-correlator determines a cross correlation measure between corresponding bands, i.e., bands having the same frequency index. The cross correlation meas- ure determined by block 84 can have a value between 0 and 1, wherein 0 indicates no correlation, and wherein 1 indicates full correlation. When the device 84 outputs a low cross cor- relation measure, this means that the left and right signal components in the respective band are different from each other so that this band does not include signal components occurring in both bands, which should be inserted into a cen- ter channel. When, however, the cross correlation measure is high, indicating that the signals in both bands are very similar to each other, then this band has a signal component occurring in the left and right channels so that this band should be inserted into the center channel. A further criterion for deciding whether signals in bands are similar to each other is the signal energy. Therefore, the preferred embodiment of the inventive input channels analyzer includes a band-wise energy calculator 85, which calculates the energy in each band and which outputs an energy similar- ity measure indicating, whether the energies in the corre- sponding bands are similar to each other or different from each other. The energy similarity measure output by device 85 and the cross correlation measure output by device 84 are both input into a final decision stage 86, which comes to a conclusion that, in a certain frame, a certain band i occurs in both channels or not. When the decision stage 86 determines that the signal occurs in both channels, then this signal portion is fed into the center channel to generate a "real center". Fig. 8 shows an embodiment for implementing the input chan- nels analyzer. Additional embodiments are known in the art and, for example, illustrated in "Spatial enhancement of au- dio recordings", Jot and Avendano, 23rd International AES Conference, Copenhagen, Denmark, May 23-25, 2003. Particu- larly, other methods of analyzing two channels to find signal components in these channels include statistical or analyti- cal analyzing methods such as the principle component analy- sis or the independent subspace analysis or other methods known in the art of audio analysis. All these methods have in common that they detect signal components occurring in both channels, which should be fed into a center channel to gener- ate a real center. Subsequently, reference is made to Fig. 7 to illustrate an energy situation before and after a two-three upmix process has been implemented by the two-three upmixer 16a in the Fig- ures. A left input channel L illustrated at 70 in Fig. 7 has a certain energy. In this example, the right input channel of the two stereo input channels has a different (lower) energy as illustrated at 71. It is assumed that the channel analyzer has found out that there are signal components occurring in both channels. These signal components occurring in both channels have an energy as illustrated at 72 in Fig. 7. When the whole energy 72 would be fed into the center channel as shown at 73, the energy of the center channel would be above an energy limit, wherein the energy limit at least roughly illustrates that the signal having such a high energy has am- plitude values above the amplitude maximum threshold. There- fore, only a portion of the energy 72 is input into the real center, while the exceeding portion is equally (re-) distrib- uted to the synthesized left and right channels L' and R' as illustrated by arrows 76. In this context, it is to be noted that there are different ways of redistributing energy from the center channel back to the left and right channels or for introducing a correct amount of energy from an original left channel and an origi- nal right channel into the center channel. One could, for ex- ample, scale down all detected signal components by a certain downscaling factor and introduce the downscaled signal into the center channel. This would have equal consequences for the signal components in each band, when a frequency- selective analysis was applied. Alternatively, one could also perform a band-wise energy control. This means that when there have been detected e.g. 10 bands having detected signal components, one could introduce only 5 bands into the center channel and leave the remaining 5 bands in the left and right channels in order to reduce the energy in the center channel. Depending on certain implementation requirements of the in- ventive methods, the inventive method can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, in particular a disk or a CD having electronically readable control signals stored thereon, which can cooperate with a programmable computer system such that the inventive method is performed. Gener- ally, the present invention is, therefore, a computer program product with a program code stored on a machine-readable car- rier, the program code being configured for performing the inventive method, when the computer program product runs on a computer. In other words, the invention is also a computer program having a program code for performing the inventive method, when the computer program runs on a computer. Claims 1. Apparatus for synthesizing three output channels (12a, 12b, 12c) using two input channels (10a, 10b), wherein a second channel of the three output channels is feedable to a speaker in an intended audio rendering scheme, which is positioned between two speakers being feedable with the first output channel and the third output channel, comprising: an analyzer (15) for analyzing the two input channels for detecting signal components occurring in both input channels to obtain detected signal components; and a signal generator (16) for generating the three output channels (12a, 12b, 12c) using the two input channels (10a, 10b), wherein the signal generator comprises: a two-three up-mixer (16a) for generating at least a second intermediate channel (12b) including at least a portion of the detected signal components; a clipping detector (16b) for detecting a portion of the second channel having an amplitude above the maximum threshold; and a two-three up-mixer controller (30, 31) for controlling the two-three up-mixer (16a) so that only a portion of the detected signal components is fed to the second channel and a remainder of the signal components remains positioned in the first and the third output channels, when a complete feeding of the detected signal compo nents would result in exceeding a maximum threshold (61a, 61b) for the second channel. 2. Apparatus in accordance with claim 1, in which the signal generator (16) is operative to generate the three output channels such that, for a certain time period, a total energy of the three output channels and potentially generated additional output channels is equal to an electrical or acoustical energy of the two input channels. 3. Apparatus in accordance with claim 1 or 2, in which the signal generator (16) is operative to generate the second output channel such that the portion of the detected signal components fed into the second channel is as large as possible so that an energy of the second output channel, which includes only the portion of the detected signal components always has a maximum amplitude below or equal to the maximum threshold (61a, 61b). 4. Apparatus in accordance with any one of the preceding claims, in which the signal generator (16) is adapted so that a remainder (73) of the detected signal components, which is not in the second channel, is included in the first and the third channels. 5. Apparatus in accordance with any one of the preceding claims, in which the maximum threshold (61a, 61b) is a full-scale amplitude determined by the apparatus for synthesizing or a digital or an analog processing device connected to the apparatus for synthesizing. 6. Apparatus in accordance with claim 5, in which the maximum threshold (61a, 61b) is equal to a maximum allowable positive or negative sampling value of a time domain waveform of a signal. 7. Apparatus in accordance with any one of the preceding claims, in which the analyzer (15) is operative to determine (84) a measure for a cross-correlation between at least a portion of the first input channel and the second input channel and to detect (86) a portion having a cross-correlation measure above a similarity threshold. 8. Apparatus in accordance with any one of the preceding claims, in which the analyzer (15) is operative to detect (85) an energy of a portion of the first, channel and a portion of the second channel and to detect (86) portions of the channels having energies being equal or differing by less than an equality threshold. 9. Apparatus in accordance with any one of the preceding claims, in which the analyzer (15) and the signal generator (16) are operative to perform a frequency selective or time selective analysis and synthesis. 10. Apparatus in accordance with any one of the preceding claims, in which the first and the second channels are a left channel (L) and a right channel (R) of a stereo representation of an audio signal, and in which the three output channels are a front-left channel (1'), a center channel (C ) , and a front-right channel (R' ), or a rear-left channel (Ls) , a rear-center channel (Cs) , and a rear-right channel (Cs) . 11. Method of synthesizing three output channels (12a, 12b, 12c) using two input channels (10a, 10b), wherein a second channel (12b) of the three output channels is feed- able to a speaker in an intended audio rendering scheme, which is positioned between two speakers being feedable with the first output channel and the third output channel, comprising: analyzing (15) the two input channels for detecting signal components occurring in both input channels; and generating (16) the three output channels using the two input channels, wherein the step of generating comprises : generating (16a) at least a second intermediate channel (12b) including at least a portion of the detected signal components; detecting (16b) a portion of the second channel having an amplitude above the maximum threshold; and controlling (30, 31) the step of generating so that only a portion of the detected signal components is fed to the second channel and a remainder of the signal components remains positioned in the first and the third output channels, when a complete feeding of the detected signal components would result in exceeding a maximum threshold for the second channel. . Apparatus for synthesizing three output channels (12a, 12b, 12c) using two input channels (10a, 10b), wherein a second channel of the three output channels is feedable to a speaker in an intended audio rendering scheme, which is positioned between two speakers being feedable with the first output channel and the third output channel, comprising: an analyzer (15) for analyzing the two input channels for detecting signal components occurring in both input channels to obtain detected signal components; and a signal generator (16) for generating the three output channels (12a, 12b, 12c) using the two input channels (10a, 10b), wherein the signal generator comprises: a clipping detector (42) for determining a portion of the input channels, in which there is a clipping probability; a two-three up-mixer (16a) for generating three intermediate channels, wherein a second intermediate channel includes at least a portion of the detected signal components; and a controller (41) for controlling the two-three up- mixer (16a) so that a generation parameter for up- mixing the portion determined by the clipping detector is controlled such that the second channel always has an amplitude below or equal to the maximum threshold. L3. Method of synthesizing three output channels (12a, 12b, 12c) using two input channels (10a, 10b), wherein a second channel (12b) of the three output channels is feed- able to a speaker in an intended audio rendering scheme, which is positioned between two speakers being feedable with the first output channel and the third output channel, comprising: analyzing (15) the two input channels for detecting siq- nal components occurring in both input channels; and generating (16) the three output channels using the two input channels, wherein the step of generating comprises : determining (42) a portion of the input channels, in which there is a clipping probability; generating (16a) three intermediate channels, wherein a second intermediate channel includes at least a portion of the detected signal components; and controlling (41) the step of generating (16a) so that a generation parameter for up-mixing the portion determined by the clipping detector is controlled such that the second channel always has an amplitude below or equal to the maximum threshold. 14. Computer program for performing, when running on a computer, a method of synthesizing in accordance with claim 11 or 13. For synthesizing at least three output channels using two stereo input channels, the stereo input channels are analyzed (15) to detect signal components occurring in both input channels. A signal generator (16) is operative to introduce at least a part of the detected signal components into the second channel (12b) associated with a second speaker in an intended speaker scheme, which is positioned between a first and a third speaker in the speaker scheme. When, however, feeding of the complete detected signal components would result in a clipping situation, then only a part of the detected signal components is fed into the second channel as a real center channel and the remainder is located in the first and third channels as a phantom center channel.

Full Text

Apparatus and method for synthesizing three output channels
using two input channels
Specification
The present invention is related to multi-channel synthesiz-
ers and, particularly, to devices generating three or more
output channels using two stereo input channels.
Multi-channel audio material is becoming more and more popu-
lar also in the consumer home environment. This is mainly due
to the fact that movies on DVD offer 5.1 multi-channel sound
and therefore even home users frequently install audio play-
back systems, which are capable of reproducing multi-channel
audio. Such a setup consists e.g. of 3 speakers L, C, R in
the front, 2 speakers Ls, Rs in the back and a low frequency
enhancement channel LFE and provides several well-known ad-
vantages over 2-channel stereo reproduction, e.g.:
improved front image stability even outside of the op-
timal central listening position due to the Center
channel (larger "sweet-spot" = optimum listening posi-
tion)
increased sense of listener "involvement" created by
the rear speakers.
Nevertheless, there exists a huge amount of legacy audio con-
tent, which consists only of two ("stereo") audio channels,
e.g. on Compact Discs (CDs).
To play back two-channel legacy audio material over a 5.1
multi-channel setup there are two basic options:
1. Reproduce the left and right channel stereo signals
over the L and R speakers, respectively, i.e., play

it back in the legacy way. This solution does not
take advantage of the extended loudspeaker setup
(Center and rear loudspeakers).
2. One may use a method to convert the two channels of
the content material to a multi-channel signal (this
may happen "on the fly" or by means of preprocessing)
that makes use of all the 5.1 speakers and in this
way benefits from the previously discussed advantages
of the multi-channel setup.
Solution #2 clearly has advantages over #1, but also contains
some problems especially with respect to the conversion of
the two front channels (Left and Right = LR) to three front
channels (Multi-channel Left, Center and Right = L'C'R').
A good LR to L'C'R' conversion solution should fulfill the
following requirements:
1) To recreate a similar, but more stable front image in
the L'C'R' than in the LR playback case, The Center
channel shall reproduce all the sound events which usu-
ally are perceived to come from the middle between the
Left and Right loudspeaker, if the listener is in the
"sweet spot". Furthermore, signals in left front posi-
tions shall be reproduced by L'C, and signals in the
right front positions shall be reproduced by R'C, re-
spectively (see J.M. Jot and C. Avendano, "Spatial En-
hancement of Audio Recordings", AES 23rd Conference,
Copenhagen, 2003).
2) The sum of the acoustical energy emitted by the chan-
nels L'C'R' should be equal to the sum of the acousti-
cal energy of the source channels LR in order to
achieve an equally loud sound impression for L'C'R as
for LR. Assuming equal characteristics in all reproduc-
tion channels, this translates into "the sum of the

electrical energy of the channels L'C'R' should be
equal to the sum of the electrical energy of the source
channels LR."
Due to requirement #1 the signals of the Left and Right chan-
nels may be mixed into one (single) center channel. This is
particularly true, if the Left and the Right channel signals
are near identical, i.e. they represent a phantom sound
source in the middle of the front sound stage. This phantom
image is now replaced by a "real" image generated by the Cen-
ter speaker. Due to requirement #2, this Center signal shall
carry the sum of the Left and the Right energy. If the level
of the Left or the Right channel signals is close to the
maximum amplitude that can be transmitted by the channel (= 0
dBFS; dBFS = dB Full Scale), the sum of the levels of both
channels will exceed the maximum level, which can be repre-
sented by the channel/system. This usually results in the un-
desirable effect of "clipping".
The clipping situation is shown in Fig. 6. Fig. 6 illustrates
a time waveform of a signal 60 processed by a processor hav-
ing a maximum positive threshold 61a and a maximum negative
threshold 61b. Depending on the capability of the digital
processor processing the digital signal, the maximum positive
threshold and the maximum negative thresholds may be +1 and -
1. Alternatively, when a digital processor is used represent-
ing the numbers in integers, the maximum positive threshold
will be 32768 corresponding to 215, and the maximum negative
threshold will be -32768 corresponding to -215.
Since a time waveform signal is represented by a sequence of
samples, each sample being a digital number between -32768
and +32768, it is easily clear that higher numbers can be ob-
tained, when, for a certain time instance, the first channel
has a quite high value and the second channel also has a
quite high value, and when these quite high values are added
together. Theoretically, the maximum number obtained by this

adding together of two channels can be 65536. However, the
digital signal processor is not able to represent this high
number. Instead, the digital processor will only represent
numbers equal to the maximum positive threshold or the maxi-
mum negative threshold. Therefore, the digital signal proces-
sor performs clipping in that a number higher or equal to the
maximum positive threshold or the maximum negative threshold
is replaced by a number equal to the maximum positive thresh-
old and the maximum negative threshold so that, with regard
to Fig. 6., the illustrated situation appears. Within a clip-
ping time portion 62, the waveform 60 does not have its natu-
ral (sine) shape, but is flattened or clipped. When this
clipped waveform is evaluated from a spectral point of view,
it becomes clear that this time domain clipping results in
strong harmonic components caused by a high gradient magni-
tude at the beginning and the end of the clipping time por-
tion 62.
This "digital clipping" is not related to the replay setup,
i.e., the amplifier and the loudspeakers used for rendering
the audio signal. However, each amplifier/loudspeaker combi-
nation also has only a limited linear range, and, when this
linear range is exceeded by a processed signal, also a kind
of clipping takes place, which can be avoided using the in-
ventive concept.
In any case, the occurrence of clipping introduces heavy dis-
tortions in the audio signal, which degrade the perceived
sound quality very much. Thus, the occurrence of clipping has
to be avoided. This is even more due to the fact that the
sound improvement by rendering a stereo signal by a multi-
channel setup such as a 5.1 speaker system is small compared
to the very annoying clipping distortions. Therefore, when
one cannot guaranty that clipping does not occur, one would
prefer to only use the left and the right speakers of a
multi-channel setup for rendering a stereo signal.

There exist prior art solutions to overcome this clipping
problem.
A simple solution to overcome this problem is to scale down
all channels equally to a level where none of the channel
signal (especially the Center signal) exceeds the 0 dBFS
limit. This can be done statically by a predefined fixed
value. In this case the fixed value must also be valid for
worst case situations, where the Left and Right channel have
maximum levels. For the average LR to L'C'R' conversion this
leads to a significantly quieter L'C'R' version than the
original stereo LR, which is undesirable, especially when us-
ers are switching between stereo and multi-channel reproduc-
tion. This behavior can be observed at commercially available
matrix decoders (Dolby ProLogicII and Logic7 Decoder) that
can be used as LR to L'C'R' converters. See Dolby Publica-
tion: "Dolby Surround Pro Logic II Decoder - Principles of
Operation", htp://www.dolby.com/assets/pdf/tech_library/209
_Dolby_Surround_Pro_Logic_II_Decoder_Principles_of_Operation.
pdf or Griesinger, D.: "Multichannel Matrix Surround Decoders
for Two-Eared Listeners", 101st AES Convention, Los Angeles,
USA, 1996, Preprint 4402.
Another simple solution is to use dynamic range compression
in order to dynamically (depending on the signal) limit the
peak signal, sometimes also called a "limiter". A disadvan-
tage of this approach is that the true dynamic range of the
audio program is not reproduced but subjected to compression
(see Digital Audio Effects DAFX; Udo Zolzer, Editor; 2002;
Wiley & Sons; p. 99ff: "Limiter").
The downscaling problem is undesirable, since it reduces the
level or volume of a sound signal compared to the level of
the original signal. In order to completely avoid any even
theoretical occurrence of clipping, one would have to down-
scale all channels by a scaling factor equal to 0.5. This re-
sults in a strongly reduced output level of the multi-channel

signal compared to the original signal. When one only listens
to this downscaled multi-channel signal, one can compensate
for this level reduction by increasing the amplification of
the sound amplifier. However, when one switches between sev-
eral sources, the (legacy) stereo signal will appear to a
listener very loud, when it is replayed using the same ampli-
fication setting of the amplifier a set for the multichannel
reproduction.
Thus, a user would have to think about reducing the amplifi-
cation setting of its amplifier before switching from a
multi-channel representation of a stereo signal to a true
stereo representation of the stereo signal in order to not
damage her or his ears or equipment.
The other prior art method using dynamic range compression
effectively avoids clipping. However, the audio signal itself
is changed. Thus, the dynamic compression leads to a non-
authentic audio signal, which, even when the introduced arti-
facts are not too annoying, is questionable from the authen-
ticity point of view.
It is an object of the present invention to provide an im-
proved concept for multi-channel synthesis using two input
channels.
This object is achieved by an apparatus for synthesizing in
accordance with claim 1, a method of synthesizing in accor-
dance with claim 14, a computer program in accordance with
claim 15 or a three channel representation in accordance with
claim 16.
The present invention is based on the finding that, for over-
coming the clipping problem and for nevertheless achieving
the advantages incurred by replaying a stereo signal using
three or more channels of a multi-channel setup, the center
channel is generated as usual, i.e., receives sound events

located in the middle between the left and the right loud-
speakers, which is also called a "real center" rendering.
However, when the real center would come into the clipping
range, only a portion of the energy of the signal components
representing the events in the middle of the audio setup are
fed into the center channel. The remainder of the energy of
these sound events is fed back into the first and third (or
left and right) channels or remains there from the beginning.
Thus, for a time frame, where clipping may occur, when the
two/three upmix procedure is performed without modifications,
the center channel is scaled down the level below or equal to
the maximum level possible without clipping. Nevertheless,
the missing part/energy of the signal, which cannot be ren-
dered by the center channel is reproduced with the left chan-
nel and the right channel as a "virtual center" or "phantom
center".
The signal of the real center and the virtual center is then
acoustically combined during playback recreating an intended
center without clipping. This "mixing" of the real center and
the virtual center results in an improved more stable front
image of a stereo audio signal, i.e., in an increased sweet
spot, although the sweet spot is not as large as when there
would not be a phantom center at all. However, the inventive
process does not have any clipping artifacts, since the re-
mainder of the energy not being processable within the second
channel due to the clipping problem is not lost but is ren-
dered by the original left and right channels.
It is noted here that, for any situations, the energy of the
left and right channels in the multi-channel setup is lower
than the energy in the original left and right channels,
since the energy of the center channel is drawn from the left
and right channels. Therefore, even when, in accordance with
the present invention, a remaining part of the energy is fed

back to the left and right output channels, there will never
exist a clipping problem within these channels.
A further advantage of the present invention is that the in-
ventive signal generation is performed in a way that, in a
preferred embodiment, the total electrical or acoustical en-
ergy of the generated three output channels (and optionally
generated additional output channels such as Ls, Rs, Cs, LFE,
...) is preserved with respect to the energy of the original
stereo signal. The same overall loudness irrespective of the
way of rendering the signal, i.e., whether the signal is ren-
dered using a stereo setup having only two speakers or
whether the signal is rendered using a multi-channel setup
having more than two speakers, can be guaranteed.
Furthermore, the inventive signal generation and distribution
of sound energy to the center channel and the left and right
channels is dynamically applied only if clipping would be un-
avoidable, i.e., the second center channel is completely un-
changed in situations, which are not effected by clipping,
i.e., when sampling values of the second channel remain below
or are only equal to the maximum threshold.
Furthermore, the resulting acoustic combination of the "real
center" and the "phantom center" produces a signal which is
much closer to the optimal three channel configuration, i.e.,
three channels without clipping or three channels in which
sampling values without any min/max threshold are allowable.
The inventive sound image is, therefore, in preferred embodi-
ments neither different in level compared to the stereo input
signal nor non-authentic as would be the case when using a
limiter or a simple clipper.
Preferred embodiments of the present invention are subse-
quently explained with respect to the accompanying drawings,
in which:

Fig. 1 illustrates an apparatus for synthesizing the upper
channels in accordance with the preferred embodi-
ment of the present invention;
Fig. 2a a preferred embodiment of the signal generator of
Fig. 1 having a post processor;
Fig. 2b a preferred implementation of the post processor of
Fig. 2a;
Fig. 3 a further embodiment of the inventive signal gen-
erator having an iterative upmixer control;
Fig. 4 a further embodiment of the inventive signal gen-
erator completely operating in the parameter do-
main;
Fig. 5 an example for a 5.1 sound system optionally also
having a surround center channel Cs;
Fig. 6 an illustration of a clipped waveform;
Fig. 7 a schematic illustration of the energy situation of
the original two-channel input signal and the
three-channel output signal before and after clip-
ping; and
Fig. 8 illustrates a preferred input channels analyzer.
Fig. 1 illustrates a preferred embodiment of an inventive ap-
paratus for synthesizing three output channels using two in-
put channels, wherein a second channel of the three output
channels is intended for a speaker in an audio replay setup,
which is positioned between two speakers, which are intended
to receive the first output channel and the third output
channel. The input channels are indicated by 10a, which chan-
nel can be for example the left channel L, and 10b for the

second channel, which can be the right channel R. The output
channels are indicated as 12a for the right channel, 12b for
center channel and 12c for the left channel. Additional out-
put channels can be generated such as a left surround output
channel 14a, a right surround output channel 14b and a low
frequency enhancement channel 14c. The arrangement of the
corresponding speakers for these channels is shown in Fig. 5.
In the middle of these speakers 12a, 12b, 12c, 14a, 14b is a
sweet spot 50. When a listener is positioned within the sweet
spot, then he or she will have an optimum sound impression.
Additionally, one might add a center surround channel 51 Cs,
which is positioned between the left surround channel 14a and
the right surround channel 14b. The signal for the center
surround channel 51 can be calculated using the same process
as calculating the signal for the center channel 12b. Addi-
tionally, the inventive methods can, therefore, also be ap-
plied to the calculation of the center surround channel in
order to avoid clipping in the center surround channel.
It is to be noted that the inventive process is usable for
each audio channel constellation, in which two input channels
intended for two different spatial positions in a replay
setup are used and in which three output channels are gener-
ated using these two input channels, wherein the second chan-
nel of the three channels is located between two additional
speakers in the replay setup, which are provided with the
first and the third input channel signals.
The inventive synthesizer apparatus of Fig. 1 includes an in-
put channel analyzer 15 for analyzing the two input channels
in order to determine signal components which occur in both
input channels. These signal components which occur in both
input channels can be used to build the real center channel,
i.e. can be rendered via the center channel C shown in Fig.
5. Typically, a stereo signal includes a lot of such mono-
phonic signal components such as a speaker person or, when

music signals are considered, a singer or a solo instrument
positioned in front of an orchestra and, therefore, posi-
tioned in front of the audience.
The inventive synthesizer apparatus additionally includes a
time and frequency selective and, furthermore signal depend-
ent signal generator 16 for generating the three output chan-
nels 12a, 12b, 12c using the two input channels 10a, 10b and
information on detected signal components occurring in both
input channels as provided via line 13. Particularly, the in-
ventive signal generator is operative to feed detected signal
components at least partly into the second channel. Further-
more, the generator is operative to only feed a portion of
the detected signal components in the second channel, when
there exists a situation, in which a complete feeding of the
detected signal components would result in exceeding the
maximum threshold.
Thus, the second output channel has a time portion, which
only includes a part of the detected signal components to
avoid clipping, while in a different portion of the second
output channel, the complete detected signal components have
been fed into the second output channel. The remainder of the
detected signal components are included in the first and
third output channels and, therefore, form the "phantom cen-
ter" when these channels are rendered via the speaker setup
for example shown in Fig. 5.
Depending on the implementation of the inventive concept, the
"portion" of the detected signal components located in the
second channel, and the remainder of the detected signal com-
ponents located in the first and third channels can be an en-
ergy portion or frequency portion or any other portion, so
that the second channel only includes a portion of the de-
tected signal components and will not have any value above
the maximum threshold and will, therefore, not induce any
clipping distortions.

Fig. 2a illustrates a preferred embodiment of the inventive
signal analyzer 16 of Fig. 1. Particularly, in the Fig. 2a
embodiment, the signal analyzer includes a 2-3-upmixer 16
performing an upmixing process controlled by the input chan-
nels analyzer 15 of Fig. 1. The output of the 2-3-upmixer L,
R, C are upmixed channels. However, channel C might be sub-
ject to clipping, since channel C is generated using an add-
ing process, in which signal components from the left channel
and from the right channel are added together.
The center channel C is input into a clipping detector 16d,
which feeds a post processor 16c, which also receives infor-
mation on detected signal components. Particularly, the clip-
ping detector 16b is operative to examine the time wave form
of the center channel 12c.
Depending on the implementation, the clipping detector can be
constructed in different ways. When it is assumed that the
Fig. 2a signal generator can process numbers having a magni-
tude being higher than a predetermined maximum threshold,
then the clipping detector 16b simply examines the time wave-
form to see, whether there are higher numbers than the maxi-
mum threshold of the subsequent processing stage. When such a
situation is detected, the post processor 16c is activated
via activation line 16d to start post processing such that
the energy of the center channel is reduced and the energy of
the left and right channels is increased so that the three
output channels 12a, 12b, 12c are finally output by the post
processor 16c. Thus, in accordance with the Fig. 2a embodi-
ment, the LR to LCR conversion process is done as usual. The
internal first-stage center channel signal 20b is analyzed to
check, whether clipping would occur if it has to be output as
an external signal such as in an AES/EBU or as SPDIF format.
When this happens, a part of the signal 20b is removed in the
post processor 16c resulting in a modified center channel
signal 12b and distributed instead to the intermediate left

and right channels 20a, 20c as a "phantom center" contribu-
tion. After the postprocessing, the center channel signal 12b
is again below 0 dBFS.
A preferred embodiment of the post processor 16c is shown in
Fig. 2b. The center channel 20b after the upmixer 16a is in-
put into a part extractor 25. The part extractor receives in-
formation 13 on detected signal components and a control sig-
nal via line 16d from the clipping detector, which may also
include an indication of an amount of extraction. Alterna-
tively, the amount of extraction per iteration step may be
fixed independent of any occurring clipping, and an iterative
trial/error process can be applied to extract increasing
amounts of the detected signal components in a step-by-step
fashion until the clipping detector 16b does not detect any
clipping anymore. Then, the modified center channel 12b is
output by the part extractor, and the remainder of the de-
tected signal components corresponding to the extracted part
have to be re-distributed to the left and right channels 20c,
20a output by the upmixer after multiplying by 0.5. To this
end, the post processor includes two multipliers 26 in each
branch or a single multiplier before branching, and a left
adder 27a and a right adder 27b.
When the detection of the signal components occurring in both
input channels has been perfect, then the left and right
channels 20a, 20c do not include any "phantom center". How-
ever, by adding the extracted components (after multiplica-
tion by 0.5) to these channels, a phantom center is added to
the left and right channels.
Subsequently, a further embodiment of the present invention
and, particularly, of the signal generator 16 of Fig. 1 is
discussed in connection with Fig. 3. The input channels are
input into a controllable 2-3-upmixer receiving information
on detected signal components for generating three output
channels in a first iteration step controlled by an iteration

controller 30. The first step will be equal to the upmixer
operation in Fig. 2a, i.e., the center channel 20b can have
clipping problems. Such a clipping situation will be detected
by a clipping detector 16b. In contrast to the Fig. 2a em-
bodiment, the clipping detector 16b controls the upmixer 16a
in a feed-back way via the upmixer control line 31 to change
the upmixing rule in a certain way so that the generated cen-
ter channel 20b receives, after one or more iteration steps
as controlled by the iteration controller 30, only an allowed
portion of the detected signal components so that no clipping
occurs anymore.
Thus, the Fig. 3 embodiment illustrates an iterative process.
In a first pass of the iterative process, the up-mixer opera-
tion is done as usual. At the output, a detector 16b checks,
whether clipping occurs. When clipping is detected, this time
frame is processed again, now using the re-mapping process
and using re-routing of a part of the center signal energy to
the left and right channels as a phantom center contribution.
The Fig. 4 embodiment completely operates in the parameter
domain. To this end, an up-mixer parameter calculator 40 is
provided, which is connected to a parameter changer 41. Addi-
tionally, a clipping detector 42 is provided, which is opera-
tive to examine the original left and right channels or the
calculated up-mixer parameters to find out, whether clipping
will occur or not after a straight forward up-mix process.
When the clipping detector 42 detects a clipping danger, it
controls a parameter change 41 via a control line 44 to pro-
vide changed up-mix parameters, which are then provided to a
straight-forward up-mixer 16a, which then generates the
first, second, and third output channels so that no clipping
occurs in the second channel and, for a time frame, in which
the clipping detector 42 has originally detected a clipping
problem, the left and right channels 12c, 12a, have a phantom
center contribution.

In contrast to the Fig. 2 and Fig. 3 embodiments, the inven-
tive process is carried out based on processing parameters
that are used for deriving the output signals 20a, 20b, 20c,
or 12a, 12b, 12c from the input stereo signals. Thus, in or-
der to provide implementations with still lower computational
complexity, also the clipping detection and the manipulation
of signal levels or part of it are based on the processing
parameters. This is in contrast to the Fig. 2 and 3 embodi-
ments, in which the inventive process is carried out on ac-
tual audio channel signals that were already created for the
center channel after a possible clipping could be detected.
The inventive clipping detection/control can be performed by
a post-processing. Thus, the intended conversion parameters
are analyzed and modified according to the inventive concept
to provide clipping after the synthesis of the actual output
audio signals. An alternative way to control the parameter
change 41 is via an iterative way. Intended conversion pa-
rameters are analyzed. When, after the synthesis of the real
audio signal, clipping may occur, the conversion parameters
are modified. Then, the process is again started and finally,
the output channel signals are synthesized without any clip-
ping and with real center and phantom center contributions in
the corresponding channels.
Subsequently, a preferred implementation of the input chan-
nels analyzer will be discussed. To this end, reference is
made to Fig. 8, which illustrates such a preferred input
channels analyzer 15. First of all, subsequent or overlapping
frames following each other are generated using a windowing
block 80 so that, at the output of block 80, there is, on
line 81a, a block of values of the left channel and, on
line 81b, a block of values of the right channel. Then, a
frequency analysis is performed for each block individually.
To this end, a frequency analyzer 82 is provided for each
channel.

The frequency analyzer can be any device for generating a
frequency domain representation of a time domain signal. Such
a frequency analyzer can include a short-time Fourier trans-
form, an FFT algorithm, or an MDCT transform or any other
transform device. Alternatively, the frequency analyzer
block 82 may also include a subband filter bank for generat-
ing for example 32 subband channels or a higher or lower num-
ber of subband channels from a block of input signal values.
Depending on the implementation of the subband filter bank,
the functionality of the framing device 80 and the frequency
analysis block 82 can be implemented in a single digitally
implemented subband filter bank.
Then, a band-wise cross correlation is performed as indicated
by device 84. Thus, the cross-correlator determines a cross
correlation measure between corresponding bands, i.e., bands
having the same frequency index. The cross correlation meas-
ure determined by block 84 can have a value between 0 and 1,
wherein 0 indicates no correlation, and wherein 1 indicates
full correlation. When the device 84 outputs a low cross cor-
relation measure, this means that the left and right signal
components in the respective band are different from each
other so that this band does not include signal components
occurring in both bands, which should be inserted into a cen-
ter channel. When, however, the cross correlation measure is
high, indicating that the signals in both bands are very
similar to each other, then this band has a signal component
occurring in the left and right channels so that this band
should be inserted into the center channel.
A further criterion for deciding whether signals in bands are
similar to each other is the signal energy. Therefore, the
preferred embodiment of the inventive input channels analyzer
includes a band-wise energy calculator 85, which calculates
the energy in each band and which outputs an energy similar-
ity measure indicating, whether the energies in the corre-

sponding bands are similar to each other or different from
each other.
The energy similarity measure output by device 85 and the
cross correlation measure output by device 84 are both input
into a final decision stage 86, which comes to a conclusion
that, in a certain frame, a certain band i occurs in both
channels or not. When the decision stage 86 determines that
the signal occurs in both channels, then this signal portion
is fed into the center channel to generate a "real center".
Fig. 8 shows an embodiment for implementing the input chan-
nels analyzer. Additional embodiments are known in the art
and, for example, illustrated in "Spatial enhancement of au-
dio recordings", Jot and Avendano, 23rd International AES
Conference, Copenhagen, Denmark, May 23-25, 2003. Particu-
larly, other methods of analyzing two channels to find signal
components in these channels include statistical or analyti-
cal analyzing methods such as the principle component analy-
sis or the independent subspace analysis or other methods
known in the art of audio analysis. All these methods have in
common that they detect signal components occurring in both
channels, which should be fed into a center channel to gener-
ate a real center.
Subsequently, reference is made to Fig. 7 to illustrate an
energy situation before and after a two-three upmix process
has been implemented by the two-three upmixer 16a in the Fig-
ures. A left input channel L illustrated at 70 in Fig. 7 has
a certain energy. In this example, the right input channel of
the two stereo input channels has a different (lower) energy
as illustrated at 71. It is assumed that the channel analyzer
has found out that there are signal components occurring in
both channels. These signal components occurring in both
channels have an energy as illustrated at 72 in Fig. 7. When
the whole energy 72 would be fed into the center channel as
shown at 73, the energy of the center channel would be above

an energy limit, wherein the energy limit at least roughly
illustrates that the signal having such a high energy has am-
plitude values above the amplitude maximum threshold. There-
fore, only a portion of the energy 72 is input into the real
center, while the exceeding portion is equally (re-) distrib-
uted to the synthesized left and right channels L' and R' as
illustrated by arrows 76.
In this context, it is to be noted that there are different
ways of redistributing energy from the center channel back to
the left and right channels or for introducing a correct
amount of energy from an original left channel and an origi-
nal right channel into the center channel. One could, for ex-
ample, scale down all detected signal components by a certain
downscaling factor and introduce the downscaled signal into
the center channel. This would have equal consequences for
the signal components in each band, when a frequency-
selective analysis was applied. Alternatively, one could also
perform a band-wise energy control. This means that when
there have been detected e.g. 10 bands having detected signal
components, one could introduce only 5 bands into the center
channel and leave the remaining 5 bands in the left and right
channels in order to reduce the energy in the center channel.
Depending on certain implementation requirements of the in-
ventive methods, the inventive method can be implemented in
hardware or in software. The implementation can be performed
using a digital storage medium, in particular a disk or a CD
having electronically readable control signals stored
thereon, which can cooperate with a programmable computer
system such that the inventive method is performed. Gener-
ally, the present invention is, therefore, a computer program
product with a program code stored on a machine-readable car-
rier, the program code being configured for performing the
inventive method, when the computer program product runs on a
computer. In other words, the invention is also a computer

program having a program code for performing the inventive
method, when the computer program runs on a computer.

Claims
1. Apparatus for synthesizing three output channels (12a,
12b, 12c) using two input channels (10a, 10b), wherein a
second channel of the three output channels is feedable
to a speaker in an intended audio rendering scheme,
which is positioned between two speakers being feedable
with the first output channel and the third output channel,
comprising:
an analyzer (15) for analyzing the two input channels
for detecting signal components occurring in both input
channels to obtain detected signal components; and
a signal generator (16) for generating the three output
channels (12a, 12b, 12c) using the two input channels
(10a, 10b), wherein the signal generator comprises:
a two-three up-mixer (16a) for generating at least
a second intermediate channel (12b) including at
least a portion of the detected signal components;
a clipping detector (16b) for detecting a portion
of the second channel having an amplitude above the
maximum threshold; and
a two-three up-mixer controller (30, 31) for controlling
the two-three up-mixer (16a) so that only a portion of
the detected signal components is fed to the second
channel and a remainder of the signal components remains
positioned in the first and the third output channels,
when a complete feeding of the detected signal compo
nents would result in exceeding a maximum threshold
(61a, 61b) for the second channel.

2. Apparatus in accordance with claim 1, in which the signal
generator (16) is operative to generate the three
output channels such that, for a certain time period, a
total energy of the three output channels and potentially
generated additional output channels is equal to
an electrical or acoustical energy of the two input
channels.
3. Apparatus in accordance with claim 1 or 2, in which the
signal generator (16) is operative to generate the second
output channel such that the portion of the detected
signal components fed into the second channel is as
large as possible so that an energy of the second output
channel, which includes only the portion of the detected
signal components always has a maximum amplitude below
or equal to the maximum threshold (61a, 61b).
4. Apparatus in accordance with any one of the preceding
claims, in which the signal generator (16) is adapted so
that a remainder (73) of the detected signal components,
which is not in the second channel, is included in the
first and the third channels.
5. Apparatus in accordance with any one of the preceding
claims, in which the maximum threshold (61a, 61b) is a
full-scale amplitude determined by the apparatus for
synthesizing or a digital or an analog processing device
connected to the apparatus for synthesizing.
6. Apparatus in accordance with claim 5, in which the maximum
threshold (61a, 61b) is equal to a maximum allowable
positive or negative sampling value of a time domain
waveform of a signal.
7. Apparatus in accordance with any one of the preceding
claims, in which the analyzer (15) is operative to determine
(84) a measure for a cross-correlation between

at least a portion of the first input channel and the
second input channel and to detect (86) a portion having
a cross-correlation measure above a similarity threshold.
8. Apparatus in accordance with any one of the preceding
claims, in which the analyzer (15) is operative to detect
(85) an energy of a portion of the first, channel
and a portion of the second channel and to detect (86)
portions of the channels having energies being equal or
differing by less than an equality threshold.
9. Apparatus in accordance with any one of the preceding
claims, in which the analyzer (15) and the signal generator
(16) are operative to perform a frequency selective
or time selective analysis and synthesis.
10. Apparatus in accordance with any one of the preceding
claims, in which the first and the second channels are a
left channel (L) and a right channel (R) of a stereo
representation of an audio signal, and
in which the three output channels are a front-left
channel (1'), a center channel (C ) , and a front-right
channel (R' ), or a rear-left channel (Ls) , a rear-center
channel (Cs) , and a rear-right channel (Cs) .
11. Method of synthesizing three output channels (12a, 12b,
12c) using two input channels (10a, 10b), wherein a second
channel (12b) of the three output channels is feed-
able to a speaker in an intended audio rendering scheme,
which is positioned between two speakers being feedable
with the first output channel and the third output channel,
comprising:
analyzing (15) the two input channels for detecting signal
components occurring in both input channels; and

generating (16) the three output channels using the two
input channels, wherein the step of generating comprises :
generating (16a) at least a second intermediate
channel (12b) including at least a portion of the
detected signal components;
detecting (16b) a portion of the second channel
having an amplitude above the maximum threshold;
and
controlling (30, 31) the step of generating so that
only a portion of the detected signal components is
fed to the second channel and a remainder of the
signal components remains positioned in the first
and the third output channels, when a complete
feeding of the detected signal components would result
in exceeding a maximum threshold for the second
channel.
. Apparatus for synthesizing three output channels (12a,
12b, 12c) using two input channels (10a, 10b), wherein a
second channel of the three output channels is feedable
to a speaker in an intended audio rendering scheme,
which is positioned between two speakers being feedable
with the first output channel and the third output channel,
comprising:
an analyzer (15) for analyzing the two input channels
for detecting signal components occurring in both input
channels to obtain detected signal components; and
a signal generator (16) for generating the three output
channels (12a, 12b, 12c) using the two input channels
(10a, 10b), wherein the signal generator comprises:

a clipping detector (42) for determining a portion
of the input channels, in which there is a clipping
probability;
a two-three up-mixer (16a) for generating three intermediate
channels, wherein a second intermediate
channel includes at least a portion of the detected
signal components; and
a controller (41) for controlling the two-three up-
mixer (16a) so that a generation parameter for up-
mixing the portion determined by the clipping detector
is controlled such that the second channel
always has an amplitude below or equal to the maximum
threshold.
L3. Method of synthesizing three output channels (12a, 12b,
12c) using two input channels (10a, 10b), wherein a second
channel (12b) of the three output channels is feed-
able to a speaker in an intended audio rendering scheme,
which is positioned between two speakers being feedable
with the first output channel and the third output channel,
comprising:
analyzing (15) the two input channels for detecting siq-
nal components occurring in both input channels; and
generating (16) the three output channels using the two
input channels, wherein the step of generating comprises :
determining (42) a portion of the input channels,
in which there is a clipping probability;
generating (16a) three intermediate channels,
wherein a second intermediate channel includes at

least a portion of the detected signal components;
and
controlling (41) the step of generating (16a) so
that a generation parameter for up-mixing the portion
determined by the clipping detector is controlled
such that the second channel always has an
amplitude below or equal to the maximum threshold.
14. Computer program for performing, when running on a computer,
a method of synthesizing in accordance with claim
11 or 13.

For synthesizing at least three output channels using two
stereo input channels, the stereo input channels are analyzed
(15) to detect signal components occurring in both input
channels. A signal generator (16) is operative to introduce
at least a part of the detected signal components into the
second channel (12b) associated with a second speaker in an
intended speaker scheme, which is positioned between a first
and a third speaker in the speaker scheme. When, however,
feeding of the complete detected signal components would result
in a clipping situation, then only a part of the detected
signal components is fed into the second channel as a
real center channel and the remainder is located in the first
and third channels as a phantom center channel.

Documents:

02085-kolnp-2008-abstract.pdf

02085-kolnp-2008-claims.pdf

02085-kolnp-2008-correspondence others.pdf

02085-kolnp-2008-description complete.pdf

02085-kolnp-2008-drawings.pdf

02085-kolnp-2008-form 1.pdf

02085-kolnp-2008-form 2.pdf

02085-kolnp-2008-form 3.pdf

02085-kolnp-2008-form 5.pdf

02085-kolnp-2008-international exm report.pdf

02085-kolnp-2008-international publication.pdf

02085-kolnp-2008-international search report.pdf

02085-kolnp-2008-pct request form.pdf

2085-KOLNP-2008-(02-09-2013)-ABSTRACT.pdf

2085-KOLNP-2008-(02-09-2013)-ANNEXURE TO FORM 3.pdf

2085-KOLNP-2008-(02-09-2013)-CLAIMS.pdf

2085-KOLNP-2008-(02-09-2013)-CORRESPONDENCE-1.pdf

2085-KOLNP-2008-(02-09-2013)-CORRESPONDENCE.pdf

2085-KOLNP-2008-(02-09-2013)-DRAWINGS.pdf

2085-KOLNP-2008-(02-09-2013)-FORM-3.pdf

2085-KOLNP-2008-(02-09-2013)-FORM-5.pdf

2085-KOLNP-2008-(02-09-2013)-OTHERS.pdf

2085-KOLNP-2008-(02-09-2013)-PA.pdf

2085-KOLNP-2008-(02-09-2013)-PETITION UNDER RULE 137.pdf

2085-KOLNP-2008-(11-12-2012)-CORRESPONDENCE.pdf

2085-KOLNP-2008-(26-12-2014)-CORRESPONDENCE.pdf

2085-KOLNP-2008-CORRESPONDENCE 1.1.pdf

2085-kolnp-2008-form 18.pdf

2085-KOLNP-2008-OTHERS.pdf

abstract-2085-kolnp-2008.jpg

« Previous Patent

Next Patent »

Patent Number

265747

Indian Patent Application Number

2085/KOLNP/2008

PG Journal Number

11/2015

Publication Date

13-Mar-2015

Grant Date

11-Mar-2015

Date of Filing

23-May-2008

Name of Patentee

FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.

Applicant Address

HANSASTRASSE 27C 80686 MUNCHEN

Inventors:

#	Inventor's Name	Inventor's Address
1	OLIVER HELLMUTH	GESCHWISTER-VOEMEL-WEG 60 91052 ERLANGEN
2	HARALD POPP	OBERMICHELBCHER STRASSE 18 90587 TUCHENBACH
3	ANDREAS WALTER	BIRKENGRABEN 14A 96052 BAMBERG
4	JUERGEN HERRE	HALLERSTR. 24 91054 BUCKENHOF

PCT International Classification Number

H04S 3/00,H04S 5/00

PCT International Application Number

PCT/EP2005/013738

PCT International Filing date

2005-12-20

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1			NA