Systems and methods for providing voice equalization are provided. In exemplary embodiments, acoustic signals are received from both a near-end and a far-end environment. A power spectrum estimate for the far-end signal and a noise estimate based on the near-end signal are determined. A voice equalization mask based on the power spectrum estimate of the far-end signal and the noise estimate based on the near-end signal is generated and applied to the far-end signal to obtain a modified signal. The modified signal may then be dynamically processed to obtain an enhanced audio signal. The enhanced audio signal is then output.

Patent
   8180064
Priority
Dec 21 2007
Filed
Dec 21 2007
Issued
May 15 2012
Expiry
Apr 30 2029
Extension
496 days
Assg.orig
Entity
Large
23
248
all paid
1. A method for providing voice equalization in an audio device, comprising:
determining a power spectrum estimate of a far-end signal received by the audio device;
determining a noise estimate based on a near-end signal received by the audio device;
generating a voice equalization mask based on the power spectrum estimate of the far-end signal and the noise estimate based on the near-end signal;
applying the voice equalization mask to the far-end signal to obtain a modified signal;
dynamically processing the modified signal to obtain an enhanced audio signal; and
outputting the enhanced audio signal.
2. The method of claim 1 wherein generating the voice equalization mask comprises generating a modification gain (GS).
3. The method of claim 1 wherein generating the voice equalization mask comprises generating a global gain (GO).
4. The method of claim 1 wherein generating the voice equalization mask comprises generating local gains (GN).
5. The method of claim 1 wherein generating the voice equalization mask comprises combining a modification gain (GS), a global gain (GO), and local gains (GN) to generate a final modification mask.
6. The method of claim 1 wherein dynamically processing the modified signal comprises generating a limit gain (GL) that prevents signal peaks from exceeding a pre-defined value.
7. The method of claim 1 wherein dynamically processing the modified signal comprises generating a smoother gain (GP) that smoothes a waveform to prevent quick energy changes.
8. The method of claim 1 wherein dynamically processing the modified signal comprises combining a limit gain (GL) and a smoother gain (GP) and interpolating a result to a predetermined sampling rate.
9. The method of claim 1 wherein determining the noise estimate comprises receiving a primary acoustic signal via a primary microphone and receiving a secondary acoustic signal via a secondary microphone and determining an inter-microphone level difference between the primary and secondary acoustic signals.
10. The method of claim 1 wherein generating the voice equalization mask comprises combining a modification gain (GS), a global gain (GO), and local gains (GN) to maintain a constant signal-to-noise (SNR) target.

1. Field of Invention

The present invention relates generally to audio processing and more particularly to voice equalization.

2. Description of Related Art

Environmental and other background noises pose a challenge for both transmission and reception of speech over communication channels. The noise may corrupt voice signals such that transmitted speech signals are reduced in audibility and intelligibility. For example, if the speech signal originates from a noisy environment and has not been properly noise suppressed prior to transmission, a receiving listener will have to cope with this noise mixed in with the speech.

Additionally, local noise may interfere with reception of speech from a far end (e.g., remote environment) as perceived by a listener. That is, local noise may interfere with the listener's ability to discern received speech. In many cases, playback of the received speech signal may occur through a small loudspeaker, such as that found on a cellular phone. The size restriction of this loudspeaker may introduce limitations in terms of frequency response and power handling capability of the audio device (e.g., mobile phone). The received speech signal may be masked by the local noise. As a result the intelligibility and audibility of the speech signal is reduced.

Currently, there are many methods for reducing background noise in an adverse audio environment prior to transmission. However, these systems do not take into account local noise at the receiver's end. Therefore, it is desirable to be able to provide voice equalization that improve audibility and intelligibility of received speech signals in a noisy local environment.

Embodiments of the present invention overcome or substantially alleviate prior problems associated with output of audio via a loudspeaker in a noisy near-end environment. In exemplary embodiments, acoustic signals are received from both a near-end and a far-end environment. The acoustic signals for the near-end environment may be received by one or more microphones. The acoustic signal for the far-end environment may be received from a network via a receiver.

A power spectrum estimate for the far-end signal and a noise estimate based on the near-end signal are determined. In exemplary embodiments, after frequency analysis of the far-end signal, an energy module calculates a power spectrum estimate for the far-end signal as well as a primary and secondary acoustic signal from the near-end environment. The power spectrum estimates of the primary and secondary acoustic signals may then be used to calculate an inter-microphone level difference (ILD). The ILD may be utilized for voice activity detection by a noise estimate module to determine the noise estimate.

A voice equalization (i.e., final modification) mask based on the power spectrum estimate of the far-end signal and the noise estimate based on the near-end signal is generated and applied to the far-end signal to obtain a modified signal. The voice equalization mask may be based on the power spectrum estimate of the far-end signal and the noise estimate based on the near-end signal. In exemplary embodiments, the voice equalization mask is a result of combining a modification gain (GS), a global gain (GO), and local gains (GN).

The modified signal may then be dynamically processed to obtain a voice equalization audio signal which may be output via the loudspeaker. The dynamic processing may comprise generating a limit gain (GL) that prevents signal peaks from exceeding a pre-defined value, generating a smoother gain (GP) that smoothes a waveform to prevent quick energy changes, and combining the limit gain (GL) and the smoother gain (GP) and interpolating a result to a predetermined sampling rate.

FIG. 1 is an environment in which embodiments of the present invention may be practiced.

FIG. 2 is a block diagram of an exemplary communication device.

FIG. 3a is a functional block diagram of an exemplary speech enhancement system.

FIG. 3b is a graph of signal-to-noise ratios as a function of system constraints and noise levels.

FIG. 4 is a block diagram of an exemplary voice equalization engine coupled to other components of the speech enhancement system.

FIG. 5 is a block diagram of an exemplary noise suppression engine.

FIG. 6 is a flowchart of an exemplary method for voice equalization according to an exemplary embodiment of the present invention.

FIG. 7 is a flowchart of an exemplary method for calculating the voice equalization mask.

The present invention provides exemplary systems and methods for enhancing a received acoustic signal. In exemplary embodiments, the acoustic signal comprises a speech signal. Embodiments of the present invention may be practiced on any audio device that is configured to receive signals such as, but not limited to, cellular telephones, phone handsets, headsets, and conferencing systems. While some embodiments of the present invention will be described in reference to operation on a cellular telephone, the present invention may be practiced on any audio communication device.

Advantageously, exemplary embodiments are configured to improve audibility and intelligibility for acoustic signals that are sent from a noisy far-end (e.g., remote) environment, improve audibility and intelligibility for acoustic signals received in a noisy near-end (e.g., local) environment and/or any combination of these features. Acoustic signals sent from a noisy far-end environment may be processed by a noise suppression engine which attempts to remove the noise. Acoustic signals received in a noisy near-end environment may be processed through a voice equalization engine which is configured to manipulate the signal such that a louder sound to compensate for the near-end noise and is perceived with less audible distortion.

Referring to FIG. 1, an instance in which embodiments of the present invention may be practiced is shown. A user in a far-end (e.g., remote) environment 100 acts as a speech source 102 to an audio communication device 104. The exemplary communication device 104 may comprise a primary microphone 106. In some embodiments, the microphone 106 comprises an omni-directional microphone. In an alternative embodiment, the communication device 104 may comprise two microphones (e.g., the primary microphone 106 and a secondary microphone). In one example, the primary microphone 106 may be relative to the speech source 102 and a secondary microphone located a distance away from the primary microphone 106. In other alternative embodiments, the communication device 104 may comprise any number of microphones.

While the microphone 106 receives speech (i.e., speech signals) from the speech source 102, the microphone 106 may also pick up noise 108. Although the noise 108 is shown coming from a single location in FIG. 1, the noise 108 may comprise any sounds from one or more locations different than the speech source 102, and may include reverberations and echoes. The noise 108 may be stationary, non-stationary, and/or a combination of both stationary and non-stationary noise.

An acoustic signal including the speech and noise 108 may then be transmitted via a communication network 110 to a receiving communication device 112 at a near-end (e.g., local) environment 114 relative to a listener 116. The communication device 112, itself, may comprise a primary microphone 118 and a secondary microphone 120, as well as a loudspeaker 122 through which the listener 116 may hear the acoustic signal from the speech source 102. The near-end environment 114 may further include local noise 124 that can interfere with the listeners 116 comprehension of the received acoustic signal.

It should be noted that the listener 116 may also be a speech source for a far-end listener (i.e., speech source 102). As such, noise 124 in the near-end environment 114 may also be picked up with the listener's speech at the communication device 112 and transmitted to the far-end environment 100.

Some embodiments of the present invention utilize level differences (e.g., energy differences) between the acoustic signals received by the two microphones 118 and 120. Because the primary microphone 118 is much closer to the speaking listener 116 than the secondary microphone 120, the intensity level is higher for the primary-microphone 118 resulting in a larger energy level during a speech/voice segment, for example. In various embodiments, microphone arrays for beamforming may be used to derive the level differences.

In some embodiments, the level difference may then be used to discriminate speech frames from noise frames and drive a noise estimate in the frequency domain. Further embodiments may use a combination of energy level differences and time delays to discriminate speech frames from noise frames. Based on binaural cue decoding, speech signal extraction or speech enhancement may be performed.

Similar to the noise 108, the noise 124 may comprise any sounds from one or more locations different than the listener 116, and may include reverberations and echoes. The noise 124 may be stationary, non-stationary, and/or a combination of both stationary and non-stationary noise.

Referring now to FIG. 2, the exemplary communication device 112 of the listener 116 is shown in more detail. In exemplary embodiments, the communication device 112 is an audio receiving device that comprises a receiver 200, a processor 202, the primary microphone 118, the secondary microphone 120, an audio processing system 204, and an output device 206. The communication device 112 may comprise more or less components necessary for communication device 112 operations. It should be noted that the communication device 104 at the far-end environment 100 may comprise similar components as the communication device 112 of the near-end environment 114.

The exemplary receiver 200 is an acoustic sensor configured to receive a far-end signal from the network. In some embodiments, the receiver may comprise an antenna device. The received far-end signal may then be forwarded to the audio processing system 204.

The audio processing system 204 is configured to receive the acoustic signals from primary microphone 106 and process the acoustic signals. After reception by the microphone, the acoustic signals may be converted into electric signals (i.e., a primary electric signal). The electric signals may themselves be converted by an analog-to-digital converter (not shown) into digital signals for processing in accordance with some embodiments.

In some embodiments, the communication device 104 comprises the primary microphone 106 and a secondary microphone. In one example, the audio processing system 204 is configured to receive the acoustic signals from the primary microphone 106 and the secondary microphone and process the acoustic signals. The acoustic signal received by the primary microphone 106 is a primary acoustic signal, while the acoustic signal received by the secondary microphone is the secondary acoustic signal. It should be noted that embodiments of the present invention may be practiced utilizing any number of microphones.

The output device 206 is any device which provides an audio output to the listener 116. For example, the output device 206 may comprise an earpiece of a headset or handset, or the loudspeaker 122 (e.g., a speaker) on the communication device 112.

FIG. 3a is a functional diagram of the exemplary audio processing system 204 coupled to other components of the communication device 112. In exemplary embodiments, the audio processing system 204 is embodied within a memory device of the communication device 112.

In accordance with exemplary embodiments, the audio processing engine system 204 receives the acoustic signals. A root mean square (RMS) level of the far-end speech signal (PS) is determined. A fixed input gain, GD, may then be applied to PS before processing by a voice equalization engine 302. GD is associated with a digital volume control set by the listener 116. The voice equalization engine 302 also receives a noise estimate, PN, from a noise suppression engine 304. The noise suppression engine 304 will be discussed in more details in connection with FIG. 5 below.

In exemplary embodiments, performance of the voice equalization engine 302 may be associated with a target signal-to-noise (SNRT). This SNRT, in one embodiment, may be implicitly set by the listener 116 through the digital volume control (GD), level of the noise (PN), and level of the far-end speech signal (PS) measured at an instance when the listener 116 sets the volume as represented by an equation,
SNRT=PS+GD+GP−PN.

The gain, GP, is a gain applied by the voice equalization engine 302 at an instance of a last user input (i.e., gain for previous input). Thus, if the noise level changes, the voice equalization engine 302 may attempt to maintain the SNR equal to SNRT by applying a new voice equalization gain, GF. In one embodiment, GF may be limited by power headroom available. The power headroom comprises a difference in level between a highest level present in a given acoustic signal and a maximum level the communication device 112 can handle without noticeable distortion. This determination may assume that an amplifier gain, GA, of the communication device 112 is a fixed maximum gain for the amplifier. It should be noted that since the headroom may be limited, the actual SNR achieved by the voice equalization engine 302 may not reach SNRT as shown in FIG. 3b.

In embodiments where SNRT is higher than actual SNR, it is not desirable to attenuate the far-end signal below some limit relative to a current value. As a result, a lower bound on a feature gain, GM, may be set. The SNR produced by the voice equalization engine 302 may then be defined by SNRF=PS+GD+GF−PN where GF is the new voice equalization gain. In order to achieve SNRT, GF may be defined by GF=SNFT−PS−GD+PN. However, since the minimum and maximum output levels may be limited, the actual SNR achieved by the voice equalization engine 302 may be defined by SNRF=max(min(PS+GD+GF,PMAX),PS−GM)−PN.

As shown, effectiveness of the voice equalization engine 302 may be limited by several factors. These factors may include maximum power, headroom, and noise level in the near-end environment 114.

Although FIG. 3a depicts two microphones (e.g., the primary microphone 106 and the secondary microphone), various embodiments comprising any number of microphones may be used to receive one or more signal(s) and determine a noise estimate.

Referring now to FIG. 4, a detailed block diagram of an exemplary voice equalization engine 302 coupled to other components of the audio processing system 204 is shown. As discussed, the voice equalization engine 302 is configured to compensate for noise (both far-end and near-end) and process the acoustic signals to obtain better sound quality with less audible distortion. In exemplary embodiments, the voice equalization engine 302 may comprise a stationary noise suppression (SNS) module 402, a volume control and unmasking (VCU) module 404, and a mask generator (MG) 406, and a dynamic processor 408 as will be discussed further below.

In operation, the acoustic signal from the far-end environment 100, XF(t), is received from the network 110 and converted to electric signals and processed through a frequency analysis module 410. In one embodiment, the frequency analysis module 410 takes the acoustic signal and mimics the frequency analysis of the cochlea (i.e., cochlear domain) simulated by a filter bank. In one example, the frequency analysis module 410 separates the acoustic signal into frequency bands. Alternatively, other filters such as short-time Fourier transform (STFT), Fast Fourier Transform, Fast Cochlea transform, sub-band filter banks, modulated complex lapped transforms, cochlear models, a gamma-tone filter bank, wavelets, or any generalized spectral analysis filter/method. Because most sounds (e.g., acoustic signals) are complex and comprise more than one frequency, a sub-band analysis on the acoustic signal determines what individual frequencies are present in the acoustic signal during a frame (e.g., a predetermined period of time). According to one embodiment, the frame is 5 to 10 ms long.

The signals are then forwarded to an energy module 412 which computes energy/power estimates during an interval of time for each frequency of data frame of the acoustic signal. As a result, a power spectrum of the acoustic signal across all frequency bands may be determined by the energy module 412.

The power spectrum estimate for each data frame PS is then forwarded to the SNS module 402, which is configured to estimate a noise component and generate a modification gain, GS. The SNS module 402 may comprise a noise estimator which computes a far-end noise estimate PE (i.e., a noise power spectrum estimate) and an enhancement filter which uses the PS and PE to compute the modification gain GS. In one embodiment a stationary noise suppressor is used, but other single-channel non-stationary noise suppressors can be used as well. In alternative embodiments, the VCU module 404 may scale GS in non-speech regions in order to maintain a constant perceived noise level.

GS and PS may then be forwarded to the VCU module 404. The VCU module 404 also receives the power spectrum estimate of the near-end noise, PN, from the noise suppression engine 304. In exemplary embodiments, the VCU module 404 performs three main functions. The first function is to determine a global gain, GO, for the far-end signal such that SNRT is achieved upon playback. This may be accomplished by estimating the SNR in the environment using PN and PS. The second function is to compute a spectral equalization gain, GN, which may be applied to a scaled far-end signal in order to unmask signal components that may be masked by the near-end noise. The final function of the VCU module 404 is to modify the far-end noise suppression gain, GS, so that the perceived noise level after suppression remains relatively constant in spite of the gain modifications applied by GN and GO.

In exemplary embodiments, the VCU module 404 analyzes the far-end signal and PN to determine two gain functions, the global gain (i.e., per frame), GO, and the local gain, GN (i.e., per tap) for each input frame. GO may be a scalar and may correspond to GF as discussed above. According to one embodiment, the goal of the GO is to achieve and maintain the SNRT as specified during run time by both user settings (e.g., digital gain, GD) and the acoustic conditions at the moment at which the user sets the volume. In one example, GN may comprise a vector that represents a spectral equalization function.

Since the global gain, GO, may be computed in the frequency domain (e.g. using a fast cochlea transform (FCT)) there may be several considerations. First, the input signal power may be multiplied by a square of the SNS gain function, GS, to obtain an estimate of the speech power by an equation, {tilde over (P)}S=GS2PS.

The speech power estimate, {tilde over (P)}S, and near-end noise estimate, PN, may be smoothed in time using leaky integrators with time constants. These time constants may be defined by parameters associated with time constants for long term noise energy smoothing and long term speech energy smoothing. Next, {tilde over (P)}S and PN may be weighted using an equal-loudness “A” curve and integrated in frequency to obtain a global power per frame. A purpose of this weighting is to deemphasize very low and very high frequencies.

Next, power envelopes may be computed using a non-linear (e.g., signal dependent) leaky integrator. For the speech power, the power envelope estimate may be computed as,

e S = { λ 1 p s + ( 1 - λ 1 ) e s - 1 if p s > e s - 1 λ 2 p s + ( 1 - λ 2 ) e s - 1 if p s e s - 1
where ps is a weighted and integrated speech power and es-1 is a value computed in a previous frame. The forgetting factors, λ1 and λ2, are such that λ1 corresponds to a short time constant and λ2 to a longer time constant. λ1 and λ2 may be defined by parameters whereby the envelope is tracked quickly if it rises and held longer when it decays. In embodiments where a very short time constant is used for tracking the rising power, the result will be a peak tracker. The envelope update may only be performed if the power estimate corresponds to speech energy and exceeds a threshold. This threshold is computed as a scaled, A-weighted and integrated noise estimate PE from the SNS module 402. The scaling is defined by parameter which specifies an offset above the noise floor.

For the noise power envelope, en, the same non-linear integrator strategy may be used, with time constants defined by a noise peak envelope estimation. However, the update may be performed continuously according to an embodiment. In an alternative embodiment, the update may only be applied if the near-end speech is absent when the near-end noise estimate is more reliable. In order to account for the frequency response of the loudspeaker 122, the noise may be weighted by an estimate of an inverse of such response in the FCT domain. The response may be computed off-line and stored in a table. Additionally, power normalization of the noise estimate may be required to account for gains applied throughout an audio chain (e.g., microphone sensitivity, pre-amplification, A/D, etc.).

Finally, the global gain may be computed using the power envelopes as:

G O = 10 SNR T 10 e n e s

with the constraint that GO should not exceed the headroom and should not be lower than a value of the parameter, GM. The headroom may be computed based on the parameter, PMAX, and a current power estimate as:

G Omax = γ10 P MAX 10 e s
where γ is a scaling constant that translates the power in the FCT domain into the time domain. Depending on the available headroom and the other parameters, the SNRT may be achieved after application of the global gain GO.

In some embodiments, the user gain, GD, is implicitly taken into account in the input signal power, PS. When GD changes, the VCU module 404 uses explicit knowledge of the new gain and the previous gain to adjust the SNRT. This may be performed at the moment the user changes the gain since the power estimates take some time to reach the new value after the new gain is introduced. The time constants (e.g., timers) involved in this process may be implemented and adjusted once a real-time system is available in order to provide a smooth and seamless transition.

Once the global gain, GO, is applied, it may be likely that some speech components may still be masked by the near-end noise. This may be especially true if the SNRT cannot be reached with the available headroom. The masking can result in poor intelligibility, especially if the noise is non-stationary with components that overlap with the speech signal. In order to address this issue, a local gain, GN, which depends on frequency (e.g., tap) is computed such that masked speech components may be made more salient.

In order to determine the GN, first a masking threshold MN is computed based on the near-end noise power as MN=PN*w, where w is a spreading function. Since convolution is applied in the FCT domain, the resulting smoothing is frequency dependent and asymmetrical. This may represent a very crude model of more sophisticated psychoacoustic models where the Bark scale and asymmetrical spreading functions need to be used. In one embodiment, the spreading function is symmetrical in the FCT domain and normalized by its L1 norm.

The local gain may then be computed as GN=√{square root over (NSR+1)}, where

NSR = 10 SNR N 10 M N G O 2 P s
and SNRN is a target unmasking SNR defined by parameter associated with an unmasking target. In one embodiment, the local gain, GN, may be constrained to be applied to speech components that are above a minimum value (e.g., a minimum NSR or maximum SNR for spectral unmasking) with respect to a target local SNR, thus limiting its maximum value. Additionally, signal components that are close to the noise floor level after SNS processing may not be amplified. A purpose in applying this constraint is to avoid boosting low level speech and noise components that may lead to rough sound and unpleasant artifacts.

Intuitively, the local gain may be acting as an adaptive multi-band compressor which boosts weak signal components that lie within certain levels compared to the near-end and far-end noise components. Signal components which are already unmasked after application of the global gain may not be modified due to the regularization by 1 of the NSR in the equation for GN.

Application of the local gain, GN, may not guarantee that the power headroom will be respected. However, as discussed below, the dynamic processor 408 may enforce the restriction. In further embodiments, a reduction in the global gain based on the total power increase caused by the local gain may be performed.

In order to maintain the level of the stationary noise residual after the SNS module 402, the far-end mask, GS, may be modified according to the local and global gains. This may occur only in noise-only regions. The modification is defined by

G ~ S = { G S min ( S max , G O ) if noise region G S if speech region
where a maximum attenuation level, Smax, corresponds to a difference between a maximum suppression level specified by a parameter associated with maximum stationary noise suppression and the SNS suppression level, SL. Determination of speech versus noise regions may be made by finding the mask components down at the noise suppression level (e.g., noise-fill regions).

In one embodiment, a frequency-domain high-pass filter may be utilized to modify the local gain prior to interpolation. This modification may comprise a spectral modification that implements a Butterworth filter.

In exemplary embodiments, the computation of the global and local gains is performed at the frame rate thus and interpolation may be necessary to obtain a mask. Since the modification gains are not constrained in terms of their rate of change, simple linear interpolation may result in framing artifacts (e.g., perceived as roughness and granularity). In order to limit the rate of change of the mask, each gain function is interpolated separately subject to different constraints.

For the global gain, GO, slew rate constraints may be chosen such that a fast rise is permitted while a gain fall is slowed down to reduce pumping effects. For the local gain, GN, the rate of change may be allowed to be a lot faster in order to track short-time spectral changes of both the noise and the speech. Adjustment of the slew rates may determine the trade-off between a pumping and tracking speed of the voice equalization engine 302.

The mask generator 406 computes a final mask to be applied to the input acoustic signal received from the network 110 (i.e., the far-end signal). In exemplary embodiments, the interpolated local gain (GN), global gain (GO) and the modified and linearly interpolated noise suppression gain (GS) are independently up-sampled and interpolated to a sampling rate. Then, the three modification gains are combined by the mask generator 406 to obtain the final modification mask.

The final modification mask is then applied to the input signal, XF(t,k), to obtain a modified far-end signal. The modified far-end signal is then processed through a frequency synthesis module 414 to produce a time domain signal. The time domain signal may not be well suited for playback over the loudspeaker 122 since the time domain signal's dynamic range and slew rate are not limited and may cause excessive distortion. As such, in one embodiment, the time domain signal is sent to the dynamic processor 408, which is configured to minimize distortion upon playback through the loudspeaker 122.

In one embodiment, the dynamic processor 408 comprises a limiter 416, an onset smoother 418, and a linear interpolator 420. The limiter 416 is configured to limit signal envelope peaks, whereas the onset smoother 418 is configured to smear envelope onsets. A purpose of the dynamic processor 408 is to limit dynamic properties of the time domain signal to reduce instantaneous power below a limit that the loudspeaker 122 can handle without sacrificing loudness in steady state regions in the time domain signal.

In exemplary embodiments, the limiter 416 generates a limit gain that prevents signal peaks from exceeding a pre-defined value as defined by a parameter, GL-MAX. That is, the limiter 416 limits power of the signal prior to entering the loudspeaker 122 so as to prevent audible harmonics without distorting sound. Accordingly, the limiter 416 estimates a signal envelope by computing a maximum absolute value of the waveform (i.e., the time domain signal) in a given frame. The envelope is then smoothed using a non-linear integrator with predetermined parameters, such that the rises are followed very quickly and the falls are tracked more slowly. This “peak” envelope, eT, may be used to compute a limiting gain function as

G L = { g L e T if e T > g L 1 else
where the value of gL is set by the GL-MAX parameter.

In order to reduce fast energy changes which contribute to loudspeaker 122 rattling, the envelope onset smoother 418 smoothes the waveform. That is, the onset smoother 418 limits quick energy changes by limiting the envelop in terms of its amplitude. Initially, the envelope, eT, is smoothed with a relatively fast time constant. Then, the envelope, eT, is tracked with a slow-rise/fast-decay leaky integrator to derive a “sluggish” envelope, ep, and compute a smoother gain as

G P = e P e T
As a result, a slowly rising envelope is imposed on the waveform, while leaving the offset mostly unaffected.

Finally, the two gain functions GL and GP are multiplied and interpolated to the sampling rate (since they are computed per frame) with the constrained linear interpolator 420 using predetermined slew rates. It should be noted that correct adjustment of various time constants and slew rates is critical to avoid overshoots, which may result in “click” artifacts. The final result is then applied to the modified far-end signal.

Referring now to FIG. 5, a detailed block of the exemplary noise suppression engine 304 is shown. In operation, the acoustic signals received from the primary and secondary microphones 118 and 120 of the near-end communication device 112 are converted to electric signals and processed through the frequency analysis module 410. In one embodiment, as discussed above, the frequency analysis module 410 takes the acoustic signals and mimics the frequency analysis of the cochlea (i.e., cochlear domain) simulated by a filter bank. In one example, the frequency analysis module 410 separates the acoustic signals into frequency bands. Alternatively, other filters such as short-time Fourier transform (STFT), sub-band filter banks, modulated complex lapped transforms, cochlear models, wavelets, etc., can be used for the frequency analysis and synthesis. Because most sounds (e.g., acoustic signals) are complex and comprise more than one frequency, a sub-band analysis on the acoustic signal determines what individual frequencies are present in the acoustic signal during a frame (e.g., a predetermined period of time). According to one embodiment, the frame is 5-10 ms long.

The signals are then forwarded to the energy module 412 which computes energy/power estimates during an interval of time for each frequency band (i.e., power estimates) of an acoustic signal. As a result, a primary spectrum (i.e., the power spectral density of the primary acoustic signal) across all frequency bands may be determined by the energy module 412. Further details regarding the calculation of the power estimates may be found in co-pending application Ser. No. 11/343,524, filed Jan. 30, 2006, and herein incorporated by reference.

Given the calculated energy levels, an inter-microphone level difference (ILD) may be determined by an ILD module 502. In two microphone embodiments, the power spectrums are used by the ILD module 502 to determine a time and frequency varying ILD. Because the primary and secondary microphones 118 and 120 may be oriented in a particular way, certain level differences may occur when speech is active and other level differences may occur when noise is active. More details regarding the calculation of ILD may be can be found in co-pending U.S. patent application Ser. No. 11/343,524 and co-pending U.S. patent application Ser. No. 11/699,732 filed Jan. 29, 2007, and herein incorporated by reference.

According to an exemplary embodiment of the present invention, a Wiener filter is used to suppress noise/enhance speech. In order to derive a Wiener filter estimate, however, specific inputs are required. These inputs comprise a power spectral density of noise and a power spectral density of the source signal. As such, a noise estimate module 504 may be provided to determine a noise estimate for the acoustic signals.

According to exemplary embodiments, the noise estimate module 504 attempts to estimate the noise components (e.g., noise power estimates) in the microphone signals. In exemplary embodiments, the noise estimate is based only on the acoustic signal from the primary microphone 118. The exemplary noise estimate module 504 is a component which can be approximated mathematically by
N(t,ω)=λ1(t,ω)E1(t,ω)+(1−λ1(t,ω))min[N(t−1,ω),E1(t,ω)]
according to one embodiment of the present invention. As shown, the noise estimate in this embodiment is based on minimum statistics of a current energy estimate of the primary acoustic signal, E1(t,ω) and a noise estimate of a previous time frame, N(t−1,ω).

λI(t,custom character) in the above equation is derived from the ILD approximated by the ILD module 502, as

λ I ( t , ω ) = { 0 if ILD ( t , ω ) < threshold 1 if ILD ( t , ω ) > threshold
That is, when speech at the primary microphone 118 is smaller than a threshold value (e.g., threshold=0.5) above which speech is expected to be, λ1 is small, and thus the noise estimator follows the noise closely. When ILD starts to rise (e.g., because speech is detected), however, λ1 increases. As a result, the noise estimate module 504 slows down the noise estimation process and the speech energy does not contribute significantly to the final noise estimate. Therefore, exemplary embodiments of the present invention may use a combination of minimum statistics and voice activity detection to determine the noise estimate. This noise estimate may also be output to the voice equalization engine 302.

A filter module 506 then derives a filter estimate based on the noise estimate. In one embodiment, the filter is a Weiner filter. Alternative embodiments may contemplate other filters. Accordingly, the Weiner filter approximation may be approximated, according to one embodiment, as

W = ( P s P s + P n ) α
, where Ps is a power spectral density of speech and Pn is a power spectral density of noise. According to one embodiment, Pn is the noise estimate, N(t,ω), which is calculated by the noise estimate module 504. In an exemplary embodiment, Ps=E1(t,ω)−βN(t,ω), where E1(t,ω) is the energy estimate of the primary microphone 118 from the energy module 412, and N(t,ω) is the noise estimate provided by the noise estimate module 504. Because the noise estimate changes with each frame, the filter estimate will also change with each frame.

β is an over-subtraction term which is a function of the ILD. β compensates bias of minimum statistics of the noise estimate module 504 and forms a perceptual weighting. Because time constants are different, the bias will be different between portions of pure noise and portions of noise and speech. Therefore, in some embodiments, compensation for this bias may be necessary. In exemplary embodiments, β is determined empirically (e.g., 2-3 dB at a large ILD, and is 6-9 dB at a low ILD).

α in the above exemplary Weiner filter equation is a factor which further suppresses the noise estimate. α can be any positive value. In one embodiment, nonlinear expansion may be obtained by setting α to 2. According to exemplary embodiments, α is determined empirically and applied when a body of

W = ( P s P s + P n )
falls below a prescribed value (e.g., 12 dB down from the maximum possible value of W, which is unity).

Because the Weiner filter estimation may change quickly (e.g., from one frame to the next frame) and noise and speech estimates can vary greatly between each frame, application of the Weiner filter estimate, as is, may result in artifacts (e.g., discontinuities, blips, transients, etc.). Therefore, an optional filter smoothing module 508 is provided to smooth the Wiener filter estimate applied to the acoustic signals as a function of time. In one embodiment, the filter smoothing module 508 may be mathematically approximated as
M(t,ω)=λs(t,ω)W(t,ω)+(1−λs(t,ω)M(t−1,ω),
, where λs is a function of the Weiner filter estimate and the primary microphone energy, E1.

As shown, the filter smoothing module 508, at time (t) will smooth the Weiner filter estimate using the values of the smoothed Weiner filter estimate from the previous frame at time (t−1). In order to allow for quick response to the acoustic signal changing quickly, the filter smoothing module 508 performs less smoothing on quick changing signals, and more smoothing on slower changing signals. This is accomplished by varying the value of λs according to a weighed first order derivative of E1 with respect to time. If the first order derivative is large and the energy change is large, then λs is set to a large value. If the derivative is small then λs is set to a smaller value.

After smoothing by the filter smoothing module 508, the primary acoustic signal is multiplied by the smoothed Weiner filter estimate to estimate the speech. In the above Weiner filter embodiment, the speech estimate is approximated by S(t,ω)=X1(t,ω)*M(t,ω), where X1 is the acoustic signal from the primary microphone 106. In exemplary embodiments, the speech estimation occurs in a masking module 510.

Next, the speech estimate is converted back into time domain from the cochlea domain. The conversion comprises taking the speech estimate, S(t,ω), and multiplying this with an inverse frequency of the cochlea channels in a frequency synthesis module 414. Once conversion is completed, the signal may be sent to the far-end communication device 104.

It should be noted that the system architecture of the noise suppression engine 304 is exemplary. Alternative embodiments may comprise more components, less components, or equivalent components and still be within the scope of embodiments of the present invention. For example, the frequency analysis module 410 and energy module 412 may be combined into a single module. In yet another embodiment, the frequency analysis module 410, energy module 412, and/or the frequency synthesis module 414 may comprise separate modules within the voice equalization engine 302 and the noise suppression engine 304.

Referring now to FIG. 6, an exemplary flowchart 600 of an exemplary method for providing voice equalization is shown. In step 602, a far-end signal is received by the communication device 112 and forwarded to the audio processing system 204. In exemplary embodiments, the far-end signal is received from the network 110 by a receiver of the communication device 112. The audio processing system 204 then determines a power spectrum estimate of the far-end signal using the frequency analysis module 410 and energy module 412.

The audio processing system 204 also determines a noise estimate for the near-end environment 114 in step 604. In exemplary embodiments, acoustic signals from the near-end environment 114 are received by the primary microphone 106 and a secondary microphone, if one is available. The near-end acoustic signals are then processed through the noise suppression engine 304 to obtain the noise estimate for the near-end environment 114.

Since the far-end signal can be picked up by the primary microphone, the near-end noise suppressor may be able to remove this signal from the noise estimate. In one example, an acoustic echo canceller removes this signal from the noise estimate. In some embodiments, a two-microphone echo suppressor such as the one described co-pending application Ser. No. 12/004,899, filed Dec. 21, 2007, entitled “System and Method for 2-Channel and 3-Channel Acoustic Echo Cancellation” herein incorporated by reference.

In step 606, the voice equalization mask is generated. Step 606 will be discussed in more details in connection with FIG. 7 below.

Once calculated, the voice equalization mask is applied to the far-end signal to obtain a modified far-end signal in step 608. The modified far-end signal is then processed through a frequency synthesis module 414 to produce a time domain signal.

The time domain signal may not be well suited for playback over the loudspeaker 122 since the time domain signal's dynamic range and slew rate are not limited and may cause excessive distortion. As such, the time domain signal may be dynamically processed in step 610 by the dynamic processor 408, which is configured to minimize distortion upon playback through the loudspeaker 122.

In one embodiment, the dynamic processor 408 comprises a limiter 416, an onset smoother 418, and a linear interpolator 420. The limiter 416 is configured to limit signal envelope peaks, whereas the onset smoother 418 is configured to smear envelope onsets. Finally, the two gain functions GL and GP calculated by the limiter 416 and the onset smoother 418, respectively are multiplied and interpolated to the sampling rate (since they are computed per frame) by the constrained linear interpolator 420 using predetermined slew rates. The final result is then applied to the modified far-end signal to produce an enhanced audio signal.

The enhanced audio signal is then output in step 612. In exemplary embodiments, the enhanced audio signal is output through the loudspeaker 122.

Referring now to FIG. 7, a flowchart of an exemplary method for calculating the voice equalization mask (step 606) is shown. In step 702, the voice equalization engine 302 receives the energy estimate for the far-end signal and a near-end noise estimate. In exemplary embodiments, the power spectrum estimate for the far-end signal is received from the energy module 412, while the near-end noise estimate is received from the noise suppression engine 304.

In step 704, the modification gain, GS, is calculated. In exemplary embodiments, the SNS module 402 receives the power estimate for each data frame PS. The SNS module 402 may comprise a stationary noise estimator which computes a noise estimate PE and an enhancement filter which uses the PS and PE to compute the modification gain, GS.

In step 706, the global gain (i.e., per frame), GO is calculated. In exemplary embodiments, the GO is calculated by the VCU module 404. The VCU module 404 receives GS and PS along with the power estimate of the near-end noise, PN, from the noise suppression engine 304. The VCU module 404 then determines the global gain, GO, for the far-end signal such that SNRT is achieved upon playback. In one embodiment, this may be accomplished by estimating the SNR in the environment using PN and PS.

The local gain, GN (i.e., per frequency) for each input frame is calculated in step 708. GN comprises a spectral equalization gain, which may be applied to a scaled far-end signal in order to unmask signal components that may be masked by the noise. The VCU module 404 may, in some embodiments, modify the far-end noise suppression gain, GS, so that the perceived noise level after suppression remains relatively constant in spite of the gain modifications applied by GN and GO.

In step 710, the modification gain (GS), global gain (GO), and interpolated local gains (GN) are combined to generate the voice equalization gain. In exemplary embodiments, the mask generator 406 computes the final mask (i.e., voice equalization mask) to be applied to the far-end signal received from the network 110. In exemplary embodiments, the interpolated local gains (GN), global gain (GO) and the modified and linearly interpolated noise suppression gain (GS) are independently up-sampled and interpolated to a sampling rate. Then, the three modification gains are combined by the mask generator 406 to obtain the final mask.

The above-described modules can be comprised of instructions that are stored on storage media. The instructions can be retrieved and executed by a processor. Some examples of instructions include software, program code, and firmware. Some examples of storage media comprise memory devices and integrated circuits. The instructions are operational when executed by the processor 202 to direct the processor 202 to operate in accordance with embodiments of the present invention. Those skilled in the art are familiar with instructions, processor(s), and storage media.

The present invention is described above with reference to exemplary embodiments. It will be apparent to those skilled in the art that various modifications may be made and other embodiments can be used without departing from the broader scope of the present invention. For example, embodiments of the present invention may be applied to any system (e.g., non speech enhancement system) as long as a noise power spectrum estimate is available. Therefore, these and other variations upon the exemplary embodiments are intended to be covered by the present invention.

Avendano, Carlos, Murgia, Carlo

Patent Priority Assignee Title
10068578, Jul 16 2013 CRYSTAL CLEAR CODEC, LLC Recovering high frequency band signal of a lost frame in media bitstream according to gain gradient
10090882, Feb 28 2014 Oki Electric Industry Co., Ltd. Apparatus suppressing acoustic echo signals from a near-end input signal by estimated-echo signals and a method therefor
10121491, Dec 22 2014 Nokia Technologies Oy Intelligent volume control interface
10218327, Jan 10 2011 JI AUDIO HOLDINGS LLC; Jawbone Innovations, LLC Dynamic enhancement of audio (DAE) in headset systems
10230346, Jan 10 2011 JI AUDIO HOLDINGS LLC; Jawbone Innovations, LLC Acoustic voice activity detection
10311885, Jun 25 2014 Huawei Technologies Co., Ltd. Method and apparatus for recovering lost frames
10529351, Jun 25 2014 Huawei Technologies Co., Ltd. Method and apparatus for recovering lost frames
10600432, Mar 28 2017 Amazon Technologies, Inc Methods for voice enhancement
10614817, Jul 16 2013 CRYSTAL CLEAR CODEC, LLC Recovering high frequency band signal of a lost frame in media bitstream according to gain gradient
11037581, Jun 24 2016 SAMSUNG ELECTRONICS CO , LTD Signal processing method and device adaptive to noise environment and terminal device employing same
8364298, Jul 29 2009 MAPLEBEAR INC Filtering application sounds
9076456, Dec 21 2007 SAMSUNG ELECTRONICS CO , LTD System and method for providing voice equalization
9160843, Dec 08 2009 Microsoft Technology Licensing, LLC Speech signal processing to improve naturalness
9232309, Jul 13 2011 DTS, INC Microphone array processing system
9536540, Jul 19 2013 SAMSUNG ELECTRONICS CO , LTD Speech signal separation and synthesis based on auditory scene analysis and speech modeling
9601131, Jun 25 2015 HTC Corporation Sound processing device and method
9640194, Oct 04 2012 SAMSUNG ELECTRONICS CO , LTD Noise suppression for speech processing based on machine-learning mask estimation
9699554, Apr 21 2010 SAMSUNG ELECTRONICS CO , LTD Adaptive signal equalization
9794678, May 13 2011 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Psycho-acoustic noise suppression
9799330, Aug 28 2014 SAMSUNG ELECTRONICS CO , LTD Multi-sourced noise suppression
9830899, Apr 13 2009 SAMSUNG ELECTRONICS CO , LTD Adaptive noise cancellation
9866955, Aug 23 2013 GOOGLE LLC Enhancement of intelligibility in noisy environment
9978391, Nov 27 2013 TENCENT TECHNOLOGY SHENZHEN COMPANY LIMITED Method, apparatus and server for processing noisy speech
Patent Priority Assignee Title
3976863, Jul 01 1974 Alfred, Engel Optimal decoder for non-stationary signals
3978287, Dec 11 1974 Real time analysis of voiced sounds
4137510, Jan 22 1976 Victor Company of Japan, Ltd. Frequency band dividing filter
4433604, Sep 22 1981 Texas Instruments Incorporated Frequency domain digital encoding technique for musical signals
4516259, May 11 1981 Kokusai Denshin Denwa Co., Ltd. Speech analysis-synthesis system
4536844, Apr 26 1983 National Semiconductor Corporation Method and apparatus for simulating aural response information
4581758, Nov 04 1983 AT&T Bell Laboratories; BELL TELEPHONE LABORATORIES, INCORPORATED, A CORP OF NY Acoustic direction identification system
4628529, Jul 01 1985 MOTOROLA, INC , A CORP OF DE Noise suppression system
4630304, Jul 01 1985 Motorola, Inc. Automatic background noise estimator for a noise suppression system
4649505, Jul 02 1984 Ericsson Inc Two-input crosstalk-resistant adaptive noise canceller
4658426, Oct 10 1985 ANTIN, HAROLD 520 E ; ANTIN, MARK Adaptive noise suppressor
4674125, Jun 27 1983 RCA Corporation Real-time hierarchal pyramid signal processing apparatus
4718104, Nov 27 1984 RCA Corporation Filter-subtract-decimate hierarchical pyramid signal analyzing and synthesizing technique
4811404, Oct 01 1987 Motorola, Inc. Noise suppression system
4812996, Nov 26 1986 Tektronix, Inc. Signal viewing instrumentation control system
4864620, Dec 21 1987 DSP GROUP, INC , THE, A CA CORP Method for performing time-scale modification of speech information or speech signals
4920508, May 22 1986 SGS-Thomson Microelectronics Limited Multistage digital signal multiplication and addition
5027410, Nov 10 1988 WISCONSIN ALUMNI RESEARCH FOUNDATION, MADISON, WI A NON-STOCK NON-PROFIT WI CORP Adaptive, programmable signal processing and filtering for hearing aids
5054085, May 18 1983 Speech Systems, Inc. Preprocessing system for speech recognition
5058419, Apr 10 1990 NORWEST BANK MINNESOTA NORTH, NATIONAL ASSOCIATION Method and apparatus for determining the location of a sound source
5099738, Jan 03 1989 ABRONSON, CHARLES J MIDI musical translator
5119711, Nov 01 1990 INTERNATIONAL BUSINESS MACHINES CORPORATION, A CORP OF NY MIDI file translation
5142961, Nov 07 1989 Method and apparatus for stimulation of acoustic musical instruments
5150413, Mar 23 1984 Ricoh Company, Ltd. Extraction of phonemic information
5175769, Jul 23 1991 Virentem Ventures, LLC Method for time-scale modification of signals
5187776, Jun 16 1989 International Business Machines Corp. Image editor zoom function
5208864, Mar 10 1989 Nippon Telegraph & Telephone Corporation Method of detecting acoustic signal
5210366, Jun 10 1991 Method and device for detecting and separating voices in a complex musical composition
5230022, Jun 22 1990 Clarion Co., Ltd. Low frequency compensating circuit for audio signals
5319736, Dec 06 1989 National Research Council of Canada System for separating speech from background noise
5323459, Nov 10 1992 NEC Corporation Multi-channel echo canceler
5341432, Oct 06 1989 Matsushita Electric Industrial Co., Ltd. Apparatus and method for performing speech rate modification and improved fidelity
5381473, Oct 29 1992 Andrea Electronics Corporation Noise cancellation apparatus
5381512, Jun 24 1992 Fonix Corporation Method and apparatus for speech feature recognition based on models of auditory signal processing
5400409, Dec 23 1992 Nuance Communications, Inc Noise-reduction method for noise-affected voice channels
5402493, Nov 02 1992 Hearing Emulations, LLC Electronic simulator of non-linear and active cochlear spectrum analysis
5402496, Jul 13 1992 K S HIMPP Auditory prosthesis, noise suppression apparatus and feedback suppression apparatus having focused adaptive filtering
5471195, May 16 1994 C & K Systems, Inc. Direction-sensing acoustic glass break detecting system
5473702, Jun 03 1992 Oki Electric Industry Co., Ltd. Adaptive noise canceller
5473759, Feb 22 1993 Apple Inc Sound analysis and resynthesis using correlograms
5479564, Aug 09 1991 Nuance Communications, Inc Method and apparatus for manipulating pitch and/or duration of a signal
5502663, Dec 14 1992 Apple Inc Digital filter having independent damping and frequency parameters
5544250, Jul 18 1994 Google Technology Holdings LLC Noise suppression system and method therefor
5574824, Apr 11 1994 The United States of America as represented by the Secretary of the Air Analysis/synthesis-based microphone array speech enhancer with variable signal distortion
5583784, May 14 1993 FRAUNHOFER-GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG E V Frequency analysis method
5587998, Mar 03 1995 AT&T Corp Method and apparatus for reducing residual far-end echo in voice communication networks
5590241, Apr 30 1993 SHENZHEN XINGUODU TECHNOLOGY CO , LTD Speech processing system and method for enhancing a speech signal in a noisy environment
5602962, Sep 07 1993 U S PHILIPS CORPORATION Mobile radio set comprising a speech processing arrangement
5675778, Oct 04 1993 Fostex Corporation of America Method and apparatus for audio editing incorporating visual comparison
5682463, Feb 06 1995 GOOGLE LLC Perceptual audio compression based on loudness uncertainty
5694474, Sep 18 1995 Vulcan Patents LLC Adaptive filter for signal processing and method therefor
5706395, Apr 19 1995 Texas Instruments Incorporated Adaptive weiner filtering using a dynamic suppression factor
5717829, Jul 28 1994 Sony Corporation Pitch control of memory addressing for changing speed of audio playback
5729612, Aug 05 1994 CREATIVE TECHNOLOGY LTD Method and apparatus for measuring head-related transfer functions
5732189, Dec 22 1995 THE CHASE MANHATTAN BANK, AS COLLATERAL AGENT Audio signal coding with a signal adaptive filterbank
5749064, Mar 01 1996 Texas Instruments Incorporated Method and system for time scale modification utilizing feature vectors about zero crossing points
5757937, Jan 31 1996 Nippon Telegraph and Telephone Corporation Acoustic noise suppressor
5792971, Sep 29 1995 Opcode Systems, Inc. Method and system for editing digital audio information with music-like parameters
5796819, Jul 24 1996 Ericsson Inc. Echo canceller for non-linear circuits
5806025, Aug 07 1996 Qwest Communications International Inc Method and system for adaptive filtering of speech signals using signal-to-noise ratio to choose subband filter bank
5809463, Sep 15 1995 U S BANK NATIONAL ASSOCIATION Method of detecting double talk in an echo canceller
5825320, Mar 19 1996 Sony Corporation Gain control method for audio encoding device
5839101, Dec 12 1995 Nokia Technologies Oy Noise suppressor and method for suppressing background noise in noisy speech, and a mobile station
5920840, Feb 28 1995 Motorola, Inc. Communication system and method using a speaker dependent time-scaling technique
5933495, Feb 07 1997 Texas Instruments Incorporated Subband acoustic noise suppression
5943429, Jan 30 1995 Telefonaktiebolaget LM Ericsson Spectral subtraction noise suppression method
5956674, Dec 01 1995 DTS, INC Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channels
5974380, Dec 01 1995 DTS, INC Multi-channel audio decoder
5978824, Jan 29 1997 NEC Corporation Noise canceler
5983139, May 01 1997 MED-EL ELEKTROMEDIZINISCHE GERATE GES M B H Cochlear implant system
5990405, Jul 08 1998 WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT System and method for generating and controlling a simulated musical concert experience
6002776, Sep 18 1995 Interval Research Corporation Directional acoustic signal processor and method therefor
6061456, Oct 29 1992 Andrea Electronics Corporation Noise cancellation apparatus
6072881, Jul 08 1996 Chiefs Voice Incorporated Microphone noise rejection system
6097820, Dec 23 1996 THE CHASE MANHATTAN BANK, AS COLLATERAL AGENT System and method for suppressing noise in digitally represented voice signals
6108626, Oct 27 1995 Nuance Communications, Inc Object oriented audio coding
6122610, Sep 23 1998 GCOMM CORPORATION Noise suppression for low bitrate speech coder
6134524, Oct 24 1997 AVAYA Inc Method and apparatus to detect and delimit foreground speech
6137349, Jul 02 1997 Micronas Intermetall GmbH Filter combination for sampling rate conversion
6140809, Aug 09 1996 Advantest Corporation Spectrum analyzer
6173255, Aug 18 1998 Lockheed Martin Corporation Synchronized overlap add voice processing using windows and one bit correlators
6180273, Aug 30 1995 Honda Giken Kogyo Kabushiki Kaisha Fuel cell with cooling medium circulation arrangement and method
6216103, Oct 20 1997 Sony Corporation; Sony Electronics Inc. Method for implementing a speech recognition system to determine speech endpoints during conditions with background noise
6222927, Jun 19 1996 ILLINOIS, UNIVERSITY OF, THE Binaural signal processing system and method
6223090, Aug 24 1998 The United States of America as represented by the Secretary of the Air Manikin positioning for acoustic measuring
6226616, Jun 21 1999 DTS, INC Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility
6263307, Apr 19 1995 Texas Instruments Incorporated Adaptive weiner filtering using line spectral frequencies
6266633, Dec 22 1998 Harris Corporation Noise suppression and channel equalization preprocessor for speech and speaker recognizers: method and apparatus
6317501, Jun 26 1997 Fujitsu Limited Microphone array apparatus
6339758, Jul 31 1998 Kabushiki Kaisha Toshiba Noise suppress processing apparatus and method
6355869, Aug 19 1999 Method and system for creating musical scores from musical recordings
6363345, Feb 18 1999 Andrea Electronics Corporation System, method and apparatus for cancelling noise
6381570, Feb 12 1999 Telogy Networks, Inc. Adaptive two-threshold method for discriminating noise from speech in a communication signal
6430295, Jul 11 1997 Telefonaktiebolaget LM Ericsson (publ) Methods and apparatus for measuring signal level and delay at multiple sensors
6434417, Mar 28 2000 Cardiac Pacemakers, Inc Method and system for detecting cardiac depolarization
6449586, Aug 01 1997 NEC Corporation Control method of adaptive array and adaptive array apparatus
6469732, Nov 06 1998 Cisco Technology, Inc Acoustic source location using a microphone array
6487257, Apr 12 1999 Telefonaktiebolaget LM Ericsson Signal noise reduction by time-domain spectral subtraction using fixed filters
6496795, May 05 1999 Microsoft Technology Licensing, LLC Modulated complex lapped transform for integrated signal enhancement and coding
6513004, Nov 24 1999 Panasonic Intellectual Property Corporation of America Optimized local feature extraction for automatic speech recognition
6516066, Apr 11 2000 NEC Corporation Apparatus for detecting direction of sound source and turning microphone toward sound source
6529606, May 16 1997 Motorola, Inc. Method and system for reducing undesired signals in a communication environment
6549630, Feb 04 2000 Plantronics, Inc Signal expander with discrimination between close and distant acoustic source
6584203, Jul 18 2001 Bell Northern Research, LLC Second-order adaptive differential microphone array
6622030, Jun 29 2000 TELEFONAKTIEBOLAGET L M ERICSSON Echo suppression using adaptive gain based on residual echo energy
6717991, May 27 1998 CLUSTER, LLC; Optis Wireless Technology, LLC System and method for dual microphone signal noise reduction using spectral subtraction
6718309, Jul 26 2000 SSI Corporation Continuously variable time scale modification of digital audio signals
6738482, Sep 26 2000 JEAN-LOUIS HUARL, ON BEHALF OF A CORPORATION TO BE FORMED Noise suppression system with dual microphone echo cancellation
6760450, Jun 26 1997 Fujitsu Limited Microphone array apparatus
6785381, Nov 27 2001 ENTERPRISE SYSTEMS TECHNOLOGIES S A R L Telephone having improved hands free operation audio quality and method of operation thereof
6792118, Nov 14 2001 SAMSUNG ELECTRONICS CO , LTD Computation of multi-sensor time delays
6795558, Jun 26 1997 Fujitsu Limited Microphone array apparatus
6798886, Oct 29 1998 Digital Harmonic LLC Method of signal shredding
6810273, Nov 15 1999 Nokia Technologies Oy Noise suppression
6882736, Sep 13 2000 Sivantos GmbH Method for operating a hearing aid or hearing aid system, and a hearing aid and hearing aid system
6915264, Feb 22 2001 Lucent Technologies Inc. Cochlear filter bank structure for determining masked thresholds for use in perceptual audio coding
6917688, Sep 11 2002 Nanyang Technological University Adaptive noise cancelling microphone system
6944510, May 21 1999 KONINKLIJKE PHILIPS ELECTRONICS, N V Audio signal time scale modification
6978159, Jun 19 1996 Board of Trustees of the University of Illinois Binaural signal processing using multiple acoustic sensors and digital filtering
6982377, Dec 18 2003 Texas Instruments Incorporated Time-scale modification of music signals based on polyphase filterbanks and constrained time-domain processing
6999582, Mar 26 1999 ZARLINK SEMICONDUCTOR INC Echo cancelling/suppression for handsets
7016507, Apr 16 1997 Semiconductor Components Industries, LLC Method and apparatus for noise reduction particularly in hearing aids
7020605, Sep 15 2000 Macom Technology Solutions Holdings, Inc Speech coding system with time-domain noise attenuation
7031478, May 26 2000 KONINKLIJKE PHILIPS ELECTRONICS, N V Method for noise suppression in an adaptive beamformer
7054452, Aug 24 2000 Sony Corporation Signal processing apparatus and signal processing method
7065485, Jan 09 2002 Nuance Communications, Inc Enhancing speech intelligibility using variable-rate time-scale modification
7076315, Mar 24 2000 Knowles Electronics, LLC Efficient computation of log-frequency-scale digital filter cascade
7092529, Nov 01 2002 Nanyang Technological University Adaptive control system for noise cancellation
7092882, Dec 06 2000 NCR Voyix Corporation Noise suppression in beam-steered microphone array
7099821, Jul 22 2004 Qualcomm Incorporated Separation of target acoustic signals in a multi-transducer arrangement
7142677, Jul 17 2001 CSR TECHNOLOGY INC Directional sound acquisition
7146316, Oct 17 2002 CSR TECHNOLOGY INC Noise reduction in subbanded speech signals
7155019, Mar 14 2000 Ototronix, LLC Adaptive microphone matching in multi-microphone directional system
7164620, Oct 06 2003 NEC Corporation Array device and mobile terminal
7171008, Feb 05 2002 MH Acoustics, LLC Reducing noise in audio systems
7171246, Nov 15 1999 Nokia Mobile Phones Ltd. Noise suppression
7174022, Nov 15 2002 Fortemedia, Inc Small array microphone for beam-forming and noise suppression
7206418, Feb 12 2001 Fortemedia, Inc Noise suppression for a wireless communication device
7209567, Jul 09 1998 Purdue Research Foundation Communication system with adaptive noise suppression
7225001, Apr 24 2000 Telefonaktiebolaget L M Ericsson System and method for distributed noise suppression
7242762, Jun 24 2002 SHENZHEN XINGUODU TECHNOLOGY CO , LTD Monitoring and control of an adaptive filter in a communication system
7246058, May 30 2001 JI AUDIO HOLDINGS LLC; Jawbone Innovations, LLC Detecting voiced and unvoiced speech using both acoustic and nonacoustic sensors
7254242, Jun 17 2002 Alpine Electronics, Inc Acoustic signal processing apparatus and method, and audio device
7359520, Aug 08 2001 Semiconductor Components Industries, LLC Directional audio signal processing using an oversampled filterbank
7412379, Apr 05 2001 Koninklijke Philips Electronics N V Time-scale modification of signals
20010016020,
20010031053,
20020002455,
20020009203,
20020041693,
20020080980,
20020106092,
20020116187,
20020133334,
20020147595,
20020184013,
20030014248,
20030026437,
20030033140,
20030039369,
20030040908,
20030061032,
20030063759,
20030072382,
20030072460,
20030095667,
20030099345,
20030101048,
20030103632,
20030128851,
20030138116,
20030147538,
20030169891,
20030228023,
20040013276,
20040047464,
20040057574,
20040078199,
20040131178,
20040133421,
20040165736,
20040196989,
20040263636,
20050025263,
20050027520,
20050049864,
20050060142,
20050152559,
20050185813,
20050213778,
20050216259,
20050228518,
20050276423,
20050288923,
20060072768,
20060074646,
20060098809,
20060120537,
20060133621,
20060149535,
20060184363,
20060198542,
20060222184,
20070021958,
20070027685,
20070033020,
20070067166,
20070078649,
20070094031,
20070100612,
20070116300,
20070150268,
20070154031,
20070165879,
20070195968,
20070230712,
20070276656,
20080019548,
20080033723,
20080140391,
20080201138,
20080228478,
20080260175,
20090012783,
20090012786,
20090129610,
20090220107,
20090238373,
20090253418,
20090271187,
20090323982,
20100094643,
20100278352,
20110178800,
JP10313497,
JP11249693,
JP2005110127,
JP2005195955,
JP4184400,
JP5053587,
JP62110349,
JP6269083,
WO174118,
WO3043374,
WO3069499,
WO2007081916,
WO2007140003,
WO2010005493,
//////
Executed onAssignorAssigneeConveyanceFrameReelDoc
Dec 21 2007Audience, Inc.(assignment on the face of the patent)
Feb 06 2008AVENDANO, CARLOSAUDIENCE, INC ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0205810094 pdf
Feb 06 2008MURGIA, CARLOAUDIENCE, INC ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0205810094 pdf
Dec 17 2015AUDIENCE, INC AUDIENCE LLCCHANGE OF NAME SEE DOCUMENT FOR DETAILS 0379270424 pdf
Dec 21 2015AUDIENCE LLCKnowles Electronics, LLCMERGER SEE DOCUMENT FOR DETAILS 0379270435 pdf
Dec 19 2023Knowles Electronics, LLCSAMSUNG ELECTRONICS CO , LTD ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0662150911 pdf
Date Maintenance Fee Events
Oct 07 2015M1551: Payment of Maintenance Fee, 4th Year, Large Entity.
Oct 07 2015STOL: Pat Hldr no Longer Claims Small Ent Stat
Nov 15 2019M1552: Payment of Maintenance Fee, 8th Year, Large Entity.
Jan 01 2024REM: Maintenance Fee Reminder Mailed.
Jan 30 2024M1553: Payment of Maintenance Fee, 12th Year, Large Entity.
Jan 30 2024M1556: 11.5 yr surcharge- late pmt w/in 6 mo, Large Entity.


Date Maintenance Schedule
May 15 20154 years fee payment window open
Nov 15 20156 months grace period start (w surcharge)
May 15 2016patent expiry (for year 4)
May 15 20182 years to revive unintentionally abandoned end. (for year 4)
May 15 20198 years fee payment window open
Nov 15 20196 months grace period start (w surcharge)
May 15 2020patent expiry (for year 8)
May 15 20222 years to revive unintentionally abandoned end. (for year 8)
May 15 202312 years fee payment window open
Nov 15 20236 months grace period start (w surcharge)
May 15 2024patent expiry (for year 12)
May 15 20262 years to revive unintentionally abandoned end. (for year 12)