This invention provides a method of detecting time alignment of an analog audio signal and a digital audio signal in a hybrid radio system. The method comprises the steps of filtering the analog audio signal to produce a filtered analog audio signal, filtering the digital audio signal to produce a filtered digital audio signal, and using the filtered analog audio signal and the filtered digital audio signal to calculate a plurality of correlation coefficients, wherein the correlation coefficients are representative of time alignment between the analog audio signal and the digital audio signal. An apparatus for performing the method is also provided.
|
1. A method comprising the steps of:
using a transmitter to transmit an analog audio signal and a digital audio signal in a hybrid waveform;
receiving the hybrid waveform including a carrier signal modulated by the analog audio signal and a plurality of subcarriers modulated by the digital audio signal;
filtering the analog audio signal to produce a filtered analog audio signal having a reduced bandwidth;
filtering the digital audio signal to produce a filtered digital audio signal having a reduced bandwidth;
using the filtered analog audio signal and the filtered digital audio signal to calculate a plurality of correlation coefficients, wherein the correlation coefficients are representative of time alignment between the analog audio signal and the digital audio signal; and
adjusting the timing of the analog audio signal or the digital audio signal at the transmitter in response to the correlation coefficients.
10. An apparatus comprising:
a transmitter for transmitting a hybrid waveform including a carrier signal modulated by an analog audio signal and a plurality of subcarriers modulated by a digital audio signal;
an input for receiving the hybrid waveform;
a first filter for filtering the analog audio signal to produce a filtered analog audio signal having a reduced bandwidth;
a second filter for filtering the digital audio signal to produce a filtered digital audio signal having a reduced bandwidth;
a processor for using the filtered analog audio signal and the filtered digital audio signal to calculate a plurality of correlation coefficients, wherein the correlation coefficients are representative of time alignment between the analog audio signal and the digital audio signal; and
an audio processor for adjusting the timing of the analog audio signal or the digital audio signal at the transmitter in response to the correlation coefficients.
9. A method of detecting level alignment of an analog audio signal and a digital audio signal in a hybrid radio system comprising the steps of:
receiving a hybrid waveform transmitted by a transmitter and including a carrier signal modulated by an analog audio signal and a plurality of subcarriers modulated by a digital audio signal;
filtering the analog audio signal to produce a filtered analog audio signal having a reduced bandwidth;
filtering the digital audio signal to produce a filtered digital audio signal having a reduced bandwidth;
computing the signal power of the analog audio signal and the signal power of the digital audio signal for an audio segment;
using a ratio of the signal power of the analog audio signal and the signal power of the digital audio signal to produce a signal representative of a level alignment of the analog audio signal and the digital audio signal; and
adjusting the level of the analog audio signal or the digital audio signal at the transmitter in response the signal representative of a level alignment.
14. An apparatus comprising:
a transmitter for transmitting a hybrid waveform including a carrier signal modulated by an analog audio signal and a plurality of subcarriers modulated by a digital audio signal;
an input for receiving the hybrid waveform including a carrier signal modulated by an analog audio signal and a plurality of subcarriers modulated by a digital audio signal;
a first filter for filtering the analog audio signal to produce a filtered analog audio signal having a reduced bandwidth;
a second filter for filtering the digital audio signal to produce a filtered digital audio signal having a reduced bandwidth;
a processor for computing the signal power of the filtered analog audio signal and the signal power of the filtered digital audio signal for an audio segment, and for using a ratio of the signal power of the filtered analog audio signal and the signal power of the filtered digital audio signal to produce a signal representative of a level alignment of the analog audio signal and the digital audio signal; and
an audio processor for adjusting the level of the analog audio signal or the digital audio signal at the transmitter in response to the signal representative of a level alignment.
2. The method of
3. The method of
the correlation coefficients are determined using a normalized cross-correlation function.
4. The method of
dc components of the analog and digital audio signals are removed prior to determination of the correlation coefficients.
5. The method of
the correlation coefficients approach 1 when the analog and digital audio signals are time aligned and the correlation coefficients become smaller as the time alignment error increases.
6. The method of
performing a peak search over the correlation coefficients followed by a lower limiter on the correlation coefficients.
7. The method of
the filtering steps use filters with passbands between about 600 Hz and about 1600 Hz.
8. The method of
filtering the correlation coefficients using a moving average to produce an output signal that is representative of the number of samples that are misaligned.
11. The apparatus of
a peak detector for detecting peaks in the correlation coefficients.
12. The apparatus of
the first and second filters have passbands between about 600 Hz and about 1600 Hz.
13. The apparatus of
a third filter for filtering the correlation coefficients using a moving average to produce an output signal that is representative of the number of samples that are misaligned.
15. The apparatus of
the first and second filters have passbands between about 600 Hz and about 1600 Hz.
|
This invention relates to signal processing, and more particularly to methods and apparatus for detecting and controlling alignment of digital and analog audio signals in an in-band on-channel broadcasting system.
The iBiquity Digital Corporation HD Radio™ system is designed to permit a smooth evolution from current analog amplitude modulation (AM) and frequency modulation (FM) radio to a fully digital in-band on-channel (IBOC) system. This system delivers digital audio and data services to mobile, portable, and fixed receivers from terrestrial transmitters in the existing medium frequency (MF) and very high frequency (VHF) radio bands. Broadcasters may continue to transmit analog AM and FM signal simultaneously with the new, higher-quality and more robust digital signals, allowing themselves and their listeners to convert from analog to digital radio while maintaining their current frequency allocations.
The system provides a flexible means of transitioning to a digital broadcast system by providing three waveform types: Hybrid, Extended Hybrid, and All Digital. The Hybrid and Extended Hybrid types retain the analog FM signal, while the All Digital type does not. All three waveform types conform to the currently allocated spectral emissions mask. Details on the Hybrid, Extended Hybrid, and All Digital waveforms are shown in United States Patent Application Publication No. 2004/0076188, which is hereby incorporated by reference.
The digital signal is modulated using Orthogonal Frequency Division Multiplexing (OFDM). OFDM is a parallel modulation scheme in which the data stream modulates a large number of orthogonal subcarriers, which are transmitted simultaneously. OFDM is inherently flexible, readily allowing the mapping of logical channels to different groups of subcarriers.
During the transition from analog to digital broadcasting, it is envisioned that the predominant transmit modes for the HD Radio™ system will be the Hybrid modes. The Hybrid signal includes the conventional analog signal (for compatibility with existing radios) as well as digital signal subcarriers carrying the same analog audio content, but in higher-quality digital format. The digital signal is delayed with respect to its analog counterpart such that this time diversity can be used to mitigate the effects of short signal outages. In these modes, hybrid-compatible digital radios will incorporate a feature called “blend” which attempts to smoothly transition from outputting digital audio to analog audio during initial tuning, or whenever the digital waveform quality falls below an acceptable level. The blend function is described in U.S. Pat. Nos. 6,590,944 and 6,735,257, which are hereby incorporated by reference.
Blending will typically occur at the edge of digital coverage and at other locations within the coverage contour where the digital waveform is corrupted. When a short outage does occur, such as traveling under a bridge, the loss of digital audio is replaced by an analog signal. When blending occurs, it is important that the content on the analog audio and digital audio channels are aligned in both time and level to ensure that the transition is barely noticed by the listener. Optimally, the listener will notice little other than possible inherent quality differences in analog and digital audio at these blend points. However, if the broadcast station does not have the analog and digital audio signals aligned, then the result could be a harsh sounding transition between digital and analog audio. The misalignment may occur because of audio processing differences between the analog audio and digital audio paths at the broadcast facility. Furthermore the analog and digital signals are typically generated with two separate signal generation paths before combining for output. The use of different analog processing techniques and different signal generation methods makes the alignment of these two signals nontrivial. The blending must be smooth and continuous, which can happen only if the analog and digital audio is both time and level aligned.
The alignment or calibration of an HD Radio™ broadcast station's digital and analog signals is presently done manually with test equipment located at the transmitter site. This calibration requires the use of a test signal and special measurement equipment used to measure the time and level differences of the analog and digital signals. It also accounts for the intentional diversity delay imposed on the analog signal path. Furthermore the relative delays may change occasionally if the audio processing is changed, which may occur if or when the broadcast changes from music to news, for example. It is presently impractical, or cumbersome, to manually realign the signals when these modifications occur. Therefore it would be a significant benefit and convenience if the ability to automatically detect and correct alignment errors were available.
This invention provides a method of detecting time alignment of an analog audio signal and a digital audio signal in a hybrid radio system. The method comprises the steps of filtering the analog audio signal to produce a filtered analog audio signal, filtering the digital audio signal to produce a filtered digital audio signal, and using the filtered analog audio signal and the filtered digital audio signal to calculate a plurality of correlation coefficients, wherein the correlation coefficients are representative of time alignment between the analog audio signal and the digital audio signal.
The invention also encompasses an apparatus for detecting time alignment of an analog audio signal and a digital audio signal in a radio system. The apparatus comprises a first filter for filtering the analog audio signal to produce a filtered analog audio signal, a second filter for filtering the digital audio signal to produce a filtered digital audio signal, and a processor for using the filtered analog audio signal and the filtered digital audio signal to calculate a plurality of correlation coefficients, wherein the correlation coefficients are representative of alignment between the analog audio signal and the digital audio signal.
In another aspect, the invention provides a method of detecting level alignment of an analog audio signal and a digital audio signal in a hybrid radio system. The method comprises the steps of filtering the analog audio signal to produce a filtered analog audio signal, filtering the digital audio signal to produce a filtered digital audio signal, computing the signal power of the analog audio signal and the signal power of the digital audio signal for an audio segment, and using a ratio of the signal power of the analog audio signal and the signal power of the digital audio signal to produce a signal representative of the level alignment of the analog audio signal and the digital audio signal.
The invention further encompasses an apparatus for detecting level alignment of an analog audio signal and a digital audio signal in a hybrid radio system. The apparatus comprises a first filter for filtering the analog audio signal to produce a filtered analog audio signal, a second filter for filtering the digital audio signal to produce a filtered digital audio signal, and a processor for computing the signal power of the analog audio signal and the signal power of the digital audio signal for an audio segment, and for using a ratio of the signal power of the analog audio signal and the signal power of the digital audio signal to produce a signal representative of the level alignment of the analog audio signal and the digital audio signal.
Time and level alignment between the analog audio and digital audio of a HD Radio™ waveform is critical to assure a smooth blend from digital to analog in the HD Radio™ system. This invention provides a method and apparatus for verifying proper station analog/digital alignment (in both time and level). In addition, the invention can be used in a feedback design to automatically correct the misalignment of the analog audio and digital audio at the broadcast facility.
The receiver separates the analog and digital audio signals. The analog audio signal is sampled at the same rate as the digital audio signal. A monitor 28 receives the analog and digital audio signals from the receiver, determines the time and level alignment between the analog and digital audio signals, and produces an adjustment signal on line 30, that can be fed back to the broadcasting station and used to adjust the relative timing and level of the analog audio and digital audio signals. In the example illustrated in
This invention provides a method for detecting the relative alignment of the analog audio and digital audio in both time and level. This method does not require a test waveform to be transmitted. This method can be incorporated into a system that monitors a broadcast station's hybrid waveform. In addition, with specific knowledge of the blend algorithm used in the receivers, the measured alignment information can be used to develop a feedback path to the broadcasting station so that, as audio processing changes between analog and digital paths in a station, a signal representative of the relative alignment can be fed back to the station to keep the analog and digital audio content aligned, thus persevering the receiver's ability to smoothly blend between the analog and digital audio.
Although a dedicated measurement device could be implemented to measure time and level alignment, it is more convenient to utilize an existing HD Radio™ receiver, which possesses most of the functionality required for the alignment measurements. One operating mode of the HD Radio™ receiver, which is important to the development of a system for monitoring signal alignment, is termed the split operating mode. A radio that is operating in the split mode outputs left, right or mono analog audio on one channel while it outputs left, right or mono digital audio on the other channel. The monophonic split mode is preferred over stereo for the measurements of interest in this invention, since the stereo images in the analog and digital audio signals may differ. Stereo image and stereo separation fidelity may be compromised in some digital audio encoders operating at high compression ratios. In the split mode, a standard audio card in a personal computer can be used as a measurement device to process information from the HD Radio™ receiver output to determine the relative alignment of the analog and digital audio.
The invention uses analog and digital audio signals that contain the same audio information. For example, each signal represents either left, right or mono audio information, although the mono mode is most useful for this measurement/calibration. It is assumed here that the analog and digital audio streams are sampled simultaneously and input into the measurement device. The metric for estimating time alignment for the analog and digital audio signals is the correlation coefficient function implemented as a normalized cross-correlation function, assuming the dc components of the analog and digital audio signals are removed. The correlation coefficient function has the property that it approaches 1 when the two signals are time aligned and identical, except for possibly an arbitrary scalar factor difference. The coefficient becomes statistically smaller as the time alignment error increases.
Since the HD Radio™ system imposed an intentional diversity delay (e.g., 4.5 seconds) on the analog signal path at the transmitter, the receiver must match this delay on the path of the digital audio. Then the analog/digital audio delays are matched at the receiver output for subsequent alignment processing. If the alignment measurement indicates a time error (due to the transmitter misalignment, assuming the pre-calibrated receiver is correct), then this error can be passed back to the transmitter component to readjust the diversity delay.
The algorithm presumes that identically-sampled (e.g. using a 44,100 Hz sample rate) analog and digital audio signals are processed through identical digital infinite impulse response (IIR) filters. For example the IIR filters for analog and digital audio streams can be identical 10 pole elliptical filters with passbands between about 600 Hz and about 1600 Hz. The filters serve to reduce the bandwidth of the audio signals. This reduces the measurement alignment ambiguities that may occur in parts of the audio spectrum where audio processing differences are more likely to occur. For example, the analog signal will likely have a lower bandwidth than the digital signal, and filtering on the high and low frequency extremes may result in group delay differences. A filter bandwidth of roughly between 600 to 1600 Hz has been determined to be most useful for the alignment bandwidth.
The correlation coefficient ρx,y between analog and digital signals represented by x and y, respectively, can be defined using statistical expectations as
where μ is the mean, and σ is the standard deviation of process x or y. The above equation is an analog generalization; however, in practice both the analog audio (e.g., x) and digital audio (e.g., y) must be identically sampled (e.g., at 44100 Hz for monophonic signals only) for the computations that follow. The mean and standard deviation of analog audio (x) and digital audio (y) over the time segment are used in this computation. The mean is the average (i.e. dc component) and standard deviation is the square root of the variance of the samples over the time segment.
The bandpass filter rejects any dc component, as well as high frequencies out of the band of interest in this computation. The mean (average) is zero since the dc is rejected here. Since the means of the analog and digital audio signals are zero after bandpass filtering and prior to the computation of the correlation coefficient, the expression can be simplified. For the discrete N-sample, zero-mean sequences x and y, the expression for the correlation coefficient ρ with lag k becomes
where k is the number of samples of lag between the two sequences. The lag is the relative time offset between the x and y signals. This lag allows adjustment of the relative timing so we can determine where the correlation peak occurs at a specific lag. This peak lag is then the timing offset we are trying to find/measure.
The range of k is determined by the maximum possible value of time alignment error. This maximum value of lag represents the size of the search window. Clearly we have some time/memory limits in the computations and can assume that the lag range is limited by the implementation to some practical value. The number of samples N should be sufficiently large to avoid possible group delay anomalies over short segments. Furthermore, it is preferable to use a larger value of N than to average more values of the correlation coefficient function. One way to use a large N is to compute the numerator and denominators separately over smaller time segments, then average the times epochs together before a computation of the correlation coefficient function. The epochs are time segments where the measurement occurs. Multiple epochs can then be averaged to improve the measurement accuracy/reliability over any one single epoch. Specifically, let
where zj(k) is defined to be the cross-correlation of x and y over the jth epoch of time. The epochs of time where the measurements are taken can be disconnected from other epochs of time. Let
Then ρ(k) can be represented as
for any j (epoch of time).
If we want to average over epochs of time using a lossy integration technique, then we can define
where α is a value >0 (for infinite averaging) and <1 (for no averaging), where α is a parameter that allows adjustment of the effective time span for continuous averaging. This is a single pole lossy integrator. The lossy integrator allows the alignment to “forget” the measurements sufficiently long in the past where the audio processing parameters may be different. This filtering can be made more sophisticated by including information regarding the time between samples such that the measurements can be performed on an irregular schedule while maintaining appropriate filter coefficients.
Now we can calculate
The correlation coefficient function computation follows the IIR filtering and typically is processed over as little as 50 milliseconds to as much as 3 seconds of data. Typically 100 to 300 milliseconds of data are sufficient to compute the correlation coefficient function. Couple this with an α of 0.1, and we obtain reasonable estimates. The correlation coefficient is computed for each lag value over its range. The number of lags computed will depend on the actual alignment per station. For example, we can choose 1000 (or whatever the maximum search range) discrete lag values over the search range, computing the correlation for each value to search for the lag with maximum correlation.
The post processing on the alignment vector performs a peak search over all correlation coefficients followed by a lower limiter on the correlation coefficient. The alignment vector is the vector (set) of lag values over the search range. If the peak correlation for any one epoch does not exceed a good threshold, then we eliminate this for the subsequent averaging over the multiple epochs. This “limiting” prevents anomalous values from being averaged. Typically 0.92 to 0.95 can be used as a lower limit to assure that the average to follow is building up on more reliable correlations. If there is a bad section of audio that does not correlate well between the analog and digital signals, then the correlation coefficient will typically be below 0.5 and this value will not be used in determining the average. Another single pole integrator can be used to accumulate the samples that pass the limiter criteria. This estimator will usually produce a very good estimate or no estimate. A no estimate condition is likely caused by the analog digital lag (±) being out of range (misaligned by too many samples). In this case the range of the correlations should be increased (number of lags increased) and the correlation run again. The limiter and the post detection averaging are required because there could be different processing applied to the analog audio and the digital audio at the broadcast facility. These different processes will lead to different group delays for different audio bands. Thus, there will be times where the correlation will be rather bad. If these segments are examined, they typically have either channel effects on the analog audio or large processing group delay differences between the digital and analog audio streams. Thus, using a limiter and single pole filter greatly stabilizes the estimate of misalignment.
The audio gain level alignment algorithm simply uses the same IIR filtering of the split mode inputs and compares the computed sums of the squared values of the filtered analog to the filtered digital audio signals.
Computing the signal powers over several seconds and computing the ratio, optionally in dB, leads to a stable estimate of the level misalignment. A ratio of 1, or 0 dB, would imply that the analog and digital signals are level aligned, while any magnitude, positive or negative would imply a level misalignment. The ratio in dB is
The computation of the sums of squares must be done using lag value k where the analog and digital audio signals are time aligned. Specifically the signal powers must be estimated over the same audio signal segments. For efficiency, it is beneficial to accumulate the squared samples over the ranges of N samples already computed in the correlation coefficient processing that are time aligned and have a high correlation coefficient value.
The various functions described above can be implemented using known filtering and processing hardware.
While the invention has been described in terms of several embodiments, it will be apparent to those skilled in the art that various changes can be made to the described embodiments without departing from the scope of the invention as set forth in the following claims.
Kroeger, Brian William, Chalmers, Harvey, Iannuzzelli, Russell
Patent | Priority | Assignee | Title |
10177729, | Feb 19 2018 | iBiquity Digital Corporation | Auto level in digital radio systems |
10255926, | Dec 11 2015 | iBiquity Digital Corporation | Method and apparatus for automatic audio alignment in a hybrid radio system |
10484115, | Feb 09 2018 | iBiquity Digital Corporation | Analog and digital audio alignment in the HD radio exciter engine (exgine) |
8165548, | Mar 13 2009 | DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT | Receiving apparatus |
9407383, | Dec 23 2014 | BANK OF AMERICA, N A | Apparatus and method for distributing content from an HD radio system |
9755598, | Dec 18 2015 | iBiquity Digital Corporation | Method and apparatus for level control in blending an audio signal in an in-band on-channel radio system |
9819480, | Aug 04 2015 | iBiquity Digital Corporation | System and method for synchronous processing of analog and digital pathways in a digital radio receiver |
9947332, | Dec 11 2015 | iBiquity Digital Corporation | Method and apparatus for automatic audio alignment in a hybrid radio system |
RE49210, | Dec 18 2015 | iBiquity Digital Corporation | Method and apparatus for level control in blending an audio signal in an in-band on-channel radio system |
Patent | Priority | Assignee | Title |
3689841, | |||
4192003, | Apr 13 1978 | International Business Machines Corporation | Signal recovery method and apparatus |
4371953, | Dec 28 1970 | Analog read only memory | |
4382299, | Nov 07 1980 | RCA Corporation | Disc record system employing signal redundancy |
4759069, | Mar 25 1987 | SY/Lert System; SY LERT SYSTEM LIMITED PARTNERSHIP | Emergency signal warning system |
4817014, | Oct 03 1986 | RAMSEY TECHNOLOGY, INC | Digital correlator |
5095507, | Jul 24 1990 | SPECTRUM SIGNAL PROCESSING, INC ; J&C RESOURCES, INC | Method and apparatus for generating incoherent multiples of a monaural input signal for sound image placement |
5483373, | Jun 11 1993 | ALCATEL N V | Optical transmitter for the common transmission of analog signals and digital qam signals |
5675612, | Jul 13 1995 | Telefonaktiebolaget LM Ericsson | Method and apparatus for timing recovery |
5859870, | Oct 23 1995 | NEC Corporation | Time diversity transmission-reception system |
6047016, | Jun 23 1997 | LANDIS+GYR INNOVATIONS, INC | Processing a spread spectrum signal in a frequency adjustable system |
6148008, | Mar 04 1997 | Sony Corporation | Apparatus for receiving broadcasting signals |
6590944, | Feb 24 1999 | iBiquity Digital Corporation | Audio blend method and apparatus for AM and FM in band on channel digital audio broadcasting |
6683919, | Jun 16 1999 | National Semiconductor Corporation | Method and apparatus for noise bandwidth reduction in wireless communication signal reception |
6735257, | Feb 24 1999 | iBiquity Digital Corporation | Audio blend method and apparatus for AM and FM in-band on-channel digital audio broadcasting |
6937723, | Oct 25 2002 | AVAYA LLC | Echo detection and monitoring |
7088740, | Dec 21 2000 | OL SECURITY LIMITED LIABILITY COMPANY | Digital FM radio system |
20010012316, | |||
20030189989, | |||
20040043730, | |||
20040076188, | |||
EP1370016, | |||
JP2003289278, | |||
JP8265217, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Apr 07 2005 | CHALMERS, HARVEY | iBiquity Digital Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 016464 | /0819 | |
Apr 07 2005 | IANNUZZELLI, RUSSELL | iBiquity Digital Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 016464 | /0819 | |
Apr 07 2005 | KROEGER, BRIAN WILLIAM | iBiquity Digital Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 016464 | /0819 | |
Apr 08 2005 | iBiquity Digital Corporation | (assignment on the face of the patent) | / | |||
Dec 01 2006 | iBiquity Digital Corporation | MERRILL LYNCH CREDIT PRODUCTS, LLC, AS ADMINISTRATIVE AND COLLATERAL AGENT | PATENT SECURITY AGREEMENT | 018606 | /0578 | |
Oct 01 2015 | MERRILL LYNCH CREDIT PRODUCTS, LLC | iBiquity Digital Corporation | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 036877 | /0146 | |
Oct 01 2015 | iBiquity Digital Corporation | WELLS FARGO BANK, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 037069 | /0153 | |
Dec 01 2016 | ZIPTRONIX, INC | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | DigitalOptics Corporation | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | DigitalOptics Corporation MEMS | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | DTS, LLC | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | iBiquity Digital Corporation | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | Wells Fargo Bank, National Association | iBiquity Digital Corporation | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 040821 | /0108 | |
Dec 01 2016 | TESSERA ADVANCED TECHNOLOGIES, INC | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | Tessera, Inc | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | Invensas Corporation | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Dec 01 2016 | PHORUS, INC | ROYAL BANK OF CANADA, AS COLLATERAL AGENT | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 040797 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | iBiquity Digital Corporation | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | Invensas Corporation | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | PHORUS, INC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | FOTONATION CORPORATION F K A DIGITALOPTICS CORPORATION AND F K A DIGITALOPTICS CORPORATION MEMS | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | INVENSAS BONDING TECHNOLOGIES, INC F K A ZIPTRONIX, INC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | Tessera, Inc | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | DTS, INC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | ROYAL BANK OF CANADA | TESSERA ADVANCED TECHNOLOGIES, INC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 052920 | /0001 | |
Jun 01 2020 | Rovi Solutions Corporation | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | Rovi Technologies Corporation | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | Rovi Guides, Inc | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | TIVO SOLUTIONS INC | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | Invensas Corporation | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | INVENSAS BONDING TECHNOLOGIES, INC | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | Tessera, Inc | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | TESSERA ADVANCED TECHNOLOGIES, INC | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | DTS, INC | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | PHORUS, INC | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | iBiquity Digital Corporation | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Jun 01 2020 | Veveo, Inc | BANK OF AMERICA, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 053468 | /0001 | |
Oct 25 2022 | BANK OF AMERICA, N A , AS COLLATERAL AGENT | VEVEO LLC F K A VEVEO, INC | PARTIAL RELEASE OF SECURITY INTEREST IN PATENTS | 061786 | /0675 | |
Oct 25 2022 | BANK OF AMERICA, N A , AS COLLATERAL AGENT | DTS, INC | PARTIAL RELEASE OF SECURITY INTEREST IN PATENTS | 061786 | /0675 | |
Oct 25 2022 | BANK OF AMERICA, N A , AS COLLATERAL AGENT | iBiquity Digital Corporation | PARTIAL RELEASE OF SECURITY INTEREST IN PATENTS | 061786 | /0675 | |
Oct 25 2022 | BANK OF AMERICA, N A , AS COLLATERAL AGENT | PHORUS, INC | PARTIAL RELEASE OF SECURITY INTEREST IN PATENTS | 061786 | /0675 |
Date | Maintenance Fee Events |
Sep 29 2014 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Mar 27 2019 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Mar 14 2023 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Sep 27 2014 | 4 years fee payment window open |
Mar 27 2015 | 6 months grace period start (w surcharge) |
Sep 27 2015 | patent expiry (for year 4) |
Sep 27 2017 | 2 years to revive unintentionally abandoned end. (for year 4) |
Sep 27 2018 | 8 years fee payment window open |
Mar 27 2019 | 6 months grace period start (w surcharge) |
Sep 27 2019 | patent expiry (for year 8) |
Sep 27 2021 | 2 years to revive unintentionally abandoned end. (for year 8) |
Sep 27 2022 | 12 years fee payment window open |
Mar 27 2023 | 6 months grace period start (w surcharge) |
Sep 27 2023 | patent expiry (for year 12) |
Sep 27 2025 | 2 years to revive unintentionally abandoned end. (for year 12) |