A system and method for processing a video signal to make the corresponding image suitable for display on a progressive display device, i.e., a computer monitor, is disclosed. The processing system includes a video display module which is capable of processing an image frame based on the content of the frame information itself, or the values of the pixels that comprise the frame.
|
17. A video signal processing method, comprising the steps of:
obtaining current video information from an input video signal;
separating said input video signal into a plurality of video frames;
generating a look-up table from substantially all of said plurality of video frames prior to processing any one of said plurality of video frames for display, said look-up table having entered therein one of a plurality of processing type entries, each of said processing type entries respectively storing an indication of a processing algorithm for processing field data of a corresponding one of said plurality of video frames;
retrieving, successively, one of said plurality of processing type entries corresponding to each of said plurality of video frames prior to the display thereof; and
processing said field data of said one of said video frames in accordance with said processing algorithm indicated by said corresponding processing type entry.
9. A video signal processing method, comprising the steps of:
obtaining current video information from an input video signal;
separating said input video signal into a plurality of video frames;
generating a look-up table from substantially all of said plurality of video frames prior to processing any one of said plurality of video frames, said look-up table having a plurality of processing type entries respectively storing an indication of a processing algorithm for processing field data of a corresponding one of said plurality of video frames;
retrieving one of said plurality of processing type entries corresponding to one of said plurality of video frames prior to the display thereof;
processing said one of said video frames in accordance with said processing algorithm indicated by said corresponding processing type entry; and
selectively overriding said processing type entry in said look-up table for any of said plurality of video frames with a user selection of processing type.
16. A digital video display device, comprising:
a navigation unit operative to isolate an input video signal;
a decoder operative to separate said input video signal into a plurality of frames, each frame containing a series of fields;
a detection unit having means for generating a look-up table from substantially all of said plurality of frames prior to processing any one of said plurality of frames for display, said look-up table having entered therein one of a plurality of processing types associated with a corresponding one of said plurality of frames, said detection unit further having means for providing an indication of said processing type entry corresponding to said each frame from said look-up table; and
a processing unit operative to execute a processing algorithm on said series of fields of said corresponding frame in accordance with said indication of said processing type entry and providing thereby for display on a progressive display device a filtered video frame corresponding one of said plurality of frames.
5. A digital video display system, comprising:
a navigation module operative to isolate an input video signal from a digital data stream;
a decoder operative to separate said input video into a plurality of video frames;
a detection module having means for generating a look-up table from substantially all of said plurality of video frames prior to processing any one of said plurality of frames for display, said look-up table having entered therein one of a plurality of processing types associated with a corresponding one of said plurality of video frames, said detection module further having means for providing an indication of said processing type entry corresponding to said each video frame from said look-up table, said detection module further including means for user selection of processing type for said each video frame, said user selection overriding said processing type entry thereof; and
a processing module responsive to said indication of said processing type entry for providing a filtered video frame for display on a progressive display device, said filtered video frame processed in accordance with one of said processing type entry and said user selection of processing type.
1. A digital video display device, comprising:
a navigation unit operative to isolate an input video signal;
a decoder operative to separate said input video signal into a plurality of frames, each frame containing a series of fields;
a detection unit having means for generating a look-up table from substantially all of said plurality of frames prior to processing any one of said plurality of frames for display, said look-up table having entered therein one of a plurality of processing types associated with a corresponding one of said plurality of frames, said plurality of processing types including a null processing type corresponding to no predetermined processing type associated with said corresponding frame, said detection unit further having means for providing an indication of said processing type entry corresponding to said each frame from said look-up table; and
a processing unit operative to execute a first processing algorithm corresponding to said indication of said processing type entry, said processing unit providing at an output thereof a filtered video frame corresponding to one of said plurality of frames for display on a progressive display device, wherein said first processing algorithm produces said filtered video frame from field data of said corresponding frame and a predetermined number of preceding frames.
10. A method of processing a video signal to remove artifacts, comprising the steps of:
(a) separating a video image frame into its component fields, each of said component fields including a plurality of pixel lines;
(b) determining which of said component fields is a first component field, where said first component field is associated with a display time preceding that of a second component field;
(c) selecting one of either said first component field or said second component field of said video image frame for processing to a filtered video frame;
(d) setting pixel values of a first pixel line of said filtered video frame respectively to pixel values of a first one of said plurality of pixel lines of said component field selected in step (c);
(e) setting pixel values of a second pixel line of said filtered video frame respectively to pixel values of said first one of said plurality of pixel lines of said component field selected in step (c) if said selected component field is said second component field;
(f) generating a pixel line having pixel values equal to an average of corresponding pixels in each adjacent pair of pixel lines of said selected component field; and
(g) inserting said generated pixel line between said corresponding adjacent pair of pixel lines of said filtered video frame except said first pixel line and except said second pixel line if said selected component field is said second component field.
2. The device of
3. The device of
4. The device of
6. The system of
7. The system of
11. The device of
12. The device of
13. The device of
14. The video signal processing method of
15. The video signal processing method of
multiplying pixel values of each field line in each of said plurality of video frames by a corresponding scalar value; and
summing adjacent scaled field lines,
when said processing type entry indicates a second processing algorithm.
|
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever.
The present invention generally relates to digital media players and, more particularly, to a system and method for detecting interlaced video signals and processing such signals to make them viewable on progressive display devices.
Conventional video signals provided from a standard television camera (NTSC signal) or a standard video camera are comprised of a series of frames. Each frame is comprised of two fields (top and bottom) having opposite polarity. Each field contains a series of pixels that contain data representative of the image present in each frame. The frames are interlaced in such a fashion that the top and bottom fields are configured either in sequential order as illustrated in
To maintain the image quality of a moving figure, an NTSC signal, for example, can only be displayed on an interlaced monitor such as a television screen or equivalent RGB screen. When images comprised of NTSC signals are displayed on progressive display devices, i.e. computer monitors, without processing, the frames that provide an NTSC image of a moving object such as, for example, the glass illustrated in
The present invention is directed to a video display system and associated processing method that is capable of removing artifacts from a video signal, resulting in a filtered video signal that is capable of being displayed on progressive display devices. The artifacts are removed from an input video signal in a frame by frame fashion. Frame processing is performed in one of two fashions: (1) based on the information contained in each individual frame or (2) by an interpretive process based on the processing performed on a preceding number of adjacent frames. In an exemplary embodiment, artifacts are removed from an input video signal by: (a) obtaining the current frame information from an input video signal; (b) detecting the current frame delimiter from the input video signal; (c) determining whether the current frame is within a predetermined time interval; (d) determining the type of processing to be performed within the time interval from a corresponding table; and (e) generating a video frame signal in response to the predetermined parameters contained in the table.
In an alternate embodiment, artifacts are removed from an input video signal by: (a) separating a video image frame into its component fields; (b) determining which of the component fields is a first component field; (c) discarding the second component field of the video image frame; and (d) generating a combined video image frame signal based solely on the first component field.
The aforementioned signal processing steps are performed by a larger digital video system comprising a navigation unit operative to isolate an input video signal present on a digital media element provided therein; and a video unit operative to process the isolated video signal such that the video signal can be displayed on a progressive display device, the video unit comprising a decoder for separating the video signal into a plurality of frames, each frame containing a series of fields; and a video display module operative to process the fields based on information contained within each of the plurality of frames. In an exemplary embodiment, the video display module further includes a detection unit operative to determine the type of processing to be performed on the input video signal based on the fields.
An advantage of the present invention is that it provides the ability to view a conventional digital image on progressive display devices.
Another advantage of the present invention is that it provides the ability to process a conventional digital image during run time.
Still another advantage of the present invention is that it is simple to implement.
Yet still another advantage of the present invention is that it improves video signal processing efficiency.
The aforementioned and related advantages and features of the present invention will become apparent upon review of the following detailed description of the invention, taken in conjunction with the accompanying drawings, where like numerals represent like elements, in which:
The video display system of the present invention will now be described with reference to
The navigation unit 12 accepts a digital media element such as, for example, a digital versatile disk 11 and is capable of providing the audio information stored on the versatile disk 11 on a first data line (AUDIO) the video information stored on the versatile disk 11 on a second data line (VIDEO), and any additional information stored on the versatile disk 11 on a third data line (ADDT'L). In operation, the navigation unit 12 is also capable of providing the current playback position of the versatile disk 11 being played by the DVD player 10 on line 22.
The video information present on the VIDEO line is transferred to the video unit 14 through the decoder 15. The decoder 15 is capable of separating the input video signal into a plurality of frames that make-up the input video signal. The plurality of frames have the same structure as the frames illustrated in
The video display module 18 is comprised of a detection unit 16 (
The operation of the DVD player 10 of the present invention will now be described with reference to
TABLE 1 | |||||
DVD Volume | Procession | ||||
Title | Title No. | Chapter | Start Time | End Time | Decision |
INTERVIDEO | 1 | 1 | TA | TB | BOB |
INTERVIDEO | 1 | 2 | TC | TD | WEAVE |
After the Volume table has been generated, in step 52 the DVD player 10 next detects the presence of an entry in the Volume table which corresponds to the information (i.e. movie title) stored on the versatile disk 11. If the Volume table contains an entry which matches the title of the versatile disk 11, the processing (filtering) information maintained in the memory of the processing unit 17 (
After initialization has been completed, the DVD player 10 knows what type of filtering needs to be performed on the input video signal during run time and when such filtering is to be performed. The DVD player 10 then enters the run time (or playback) phase of operation.
If no user selection was made or auto detection was selected in step 60, a determination as to whether content based detection should be performed on the current frame is made in step 62. If content based detection is to be performed on the current frame, control is then transferred to the content based detection module in step 70. If content based detection is not to be performed on the current frame, control is then transferred to the bit-stream based detection module in step 80. The content based detection algorithm and bit-stream based detection algorithm will be described in greater detail below.
As illustrated in
Next, control is then transferred to the processing unit 17 (
On the other hand, if no Volume table match is detected in step 72, control is then passed to the bit-stream detection algorithm in step 73 where bit-stream based detection is performed on the current frame. This content based detection scheme is very precise in that the particular type of video signal processing to be performed on a given frame is provided directly from the frame data contained within the Volume table. No additional calculations are required. In experiments performed by the inventors, processing of video frames using content based detection yielded images that have much better visual quality than conventional video frame signals.
Bit-stream based detection will now be described with reference to
IN=NOT ((picture—structure=FRAME—PICTURE) AND progressive—frame)
TABLE 2 | ||
IN | progressive—frame | picture—structure |
1 | 0 | — |
0 | 1 | FRAME—PICTURE |
After IN has been calculated in step 81, it is stored as the last entry in a first in first out (FIFO) memory in step 82, along with the calculated IN values of the three immediately preceding frames. Next, in step 84, a decision is made using the four stored IN values regarding what type of processing is to be performed on the current frame based on the following boolean condition:
(IN=TRUE) AND (IN-1=TRUE) AND (IN-2=TRUE) AND (IN-3=TRUE)
Thus, if IN, IN-1, IN-2 and IN-3 are all 1 (TRUE), then control is transferred to processing unit 17 where BOB processing will be performed on the current frame in step 85.
On the other hand, if any of the IN values are a zero (false), then control will be transferred to the processing unit 17 where WEAVE processing will be performed on the current frame in; step 86. In the bit-stream detection method described above, IN is calculated for every frame, and the decision as to the type of processing to be performed on the current frame is always determined by comparing the current IN with the interlace indicator values of the three immediately preceding frames (IN-1, IN-2, and IN-3). The signal processing that is performed by the processing unit 17 of the present invention will now be described with reference to
bBottomFirst=((picture—structure=FRAME—PICTURE) AND (NOT top—field—first)) OR (picture—structure=BOTTOM—FIELD)
Thus, if variable picture—structure is set to BOTTOM—FIELD, then bBottomFirst is true. Also, if picture—structure is set to FRAME—PICTURE, indicating the current frame is not separated into fields, and top—field—first is not set, then bBottomFirst is also true.
If bBottomFirst is true, then the bottom field is used to construct the resulting video frame. Otherwise, the top field is used to construct the resulting video frame.
As illustrated by the boolean pseudocode above, BOB processing first determines which of the two frame fields is first. That first field is then used to generate the resulting video frame as illustrated in
In an alternate embodiment of the present invention, the resulting video frame generated as a result of BOB processing can be represented as an n-tap low pass filter as illustrated in
XN=(c3Xn−3+c1Xn−1+c1Xn+1+c3Xn+3)/(c1+c3+c3+c1)
where c3 and c1 are constants which are used to provide a weighted average for the pixels represented by Xn−3, Xn−, Xn+1 and Xn+3. In experiments performed by the inventors, it was determined that c1=3 and c3=1 provides the optimum values for the resulting video frame.
Upon completion of either the BOB or WEAVE processing, the artifacts from each frame of the video signal will have been removed, thereby resulting in an image being displayed on a progressive display device, such as a computer monitor, having no jagged edges as illustrated in
The foregoing detailed description of the invention has been provided for the purposes of illustration and description. Although an exemplary embodiment of the present invention has been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiment disclosed, and that various changes and modifications to the invention are possible in light of the above teaching. Accordingly, the scope of the present invention is to be defined by the claims appended hereto.
Patent | Priority | Assignee | Title |
7667771, | Dec 02 2004 | Harman Becker Automotive Systems GmbH | System for interlaced video data |
8233086, | Jun 08 2007 | NINTENDO CO , LTD | Process for digitizing video over analog component video cables |
8810725, | Jun 08 2007 | Nintendo Co., Ltd. | Process for digitizing video over analog component video cables |
Patent | Priority | Assignee | Title |
5134479, | Feb 16 1990 | Sharp Kabushiki Kaisha | NTSC high resolution television converting apparatus for converting television signals of an NTSC system into high resolution television signals |
5257103, | Feb 05 1992 | NVIEW CORPORATION A CORPORATION OF VA | Method and apparatus for deinterlacing video inputs |
5574504, | Jun 26 1992 | Sony Corporation | Methods and systems for encoding and decoding picture signals and related picture-signal recording media |
5621470, | Dec 08 1992 | Interpixel and interframe interpolation of television pictures with conversion from interlaced to progressive scanning | |
5633687, | Oct 23 1995 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | Method and system for providing an interlaced image on an display |
5796437, | Dec 09 1994 | Matsushita Electric Industrial Co., Ltd. | Progressive scanning conversion apparatus |
5864369, | Jun 16 1997 | ATI International SRL | Method and apparatus for providing interlaced video on a progressive display |
6034733, | Jul 29 1998 | S3 GRAPHICS CO , LTD | Timing and control for deinterlacing and enhancement of non-deterministically arriving interlaced video data |
6037990, | May 23 1997 | Kabushiki Kaisha Toshiba | Video signal processor for generating a progressive scanning format signal |
6141056, | Aug 08 1997 | RAKUTEN, INC | System for conversion of interlaced video to progressive video using horizontal displacement |
6166772, | Apr 01 1997 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Method and apparatus for display of interlaced images on non-interlaced display |
6359654, | Feb 14 1996 | QUARTERHILL INC ; WI-LAN INC | Methods and systems for displaying interlaced video on non-interlaced monitors |
6380978, | Oct 06 1997 | DVDO, INC | Digital video system and methods for providing same |
6380985, | Sep 14 1998 | Microsoft Technology Licensing, LLC | Resizing and anti-flicker filtering in reduced-size video images |
6456329, | Apr 19 1999 | MEDIATEK, INC | De-interlacing of video signals |
6459455, | Aug 31 1999 | CHIPS AND TECHNOLOGIES, LLC | Motion adaptive deinterlacing |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Oct 20 2000 | Intervideo, Inc. | (assignment on the face of the patent) | ||||
Jan 21 2002 | DING, WEI | INTER VIDEO, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 012646 | 0920 | |
Jan 21 2002 | HUNG, ANDY C | INTER VIDEO, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 012646 | 0920 | |
May 03 2002 | INTERVIDEO, INC | INTERVIDEO, INC | MERGER SEE DOCUMENT FOR DETAILS | 018606 | 0435 | |
Dec 12 2006 | INTERVIDEO, INC | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | COREL US HOLDINGS, LLC | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | Corel Corporation | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | WINZIP INTERNATIONAL LLC | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | WINZIP COMPUTING LLC | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | INTERVIDEO DIGITAL TECHNOLOGY CORP | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | CAYMAN LTD HOLDCO | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | WINZIP HOLDINGS SPAIN, S L U | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | WINZIP COMPUTING, S L U | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Dec 12 2006 | WINZIP COMPUTING LP | JPMORGAN CHASE BANK, N A | REAFFIRMATION AND JOINDER AGREEMENT | 018688 | 0199 | |
Sep 01 2007 | INTERVIDEO, INC | COREL INC | MERGER SEE DOCUMENT FOR DETAILS | 022380 | 0387 | |
Nov 22 2010 | COREL INCORPORATED | Corel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025404 | 0588 | |
May 07 2013 | Corel Corporation | VECTOR CC HOLDINGS IV, SRL | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030427 | 0331 | |
May 07 2013 | VECTOR CC HOLDINGS IV, SRL | 8324450 CANADA INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030427 | 0403 | |
May 07 2013 | Corel Corporation | VECTOR CC HOLDINGS, SRL | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030427 | 0331 | |
May 07 2013 | Corel Corporation | VECTOR CC HOLDINGS III, SRL | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030427 | 0331 | |
May 07 2013 | VECTOR CC HOLDINGS, SRL | 8324450 CANADA INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030427 | 0403 | |
May 07 2013 | VECTOR CC HOLDINGS III, SRL | 8324450 CANADA INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030427 | 0403 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | WINZIP COMPUTING LLC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | WINZIP COMPUTING LP | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | WINZIP INTERNATIONAL LLC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | Corel Corporation | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | INTERVIDEO, INC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | CAYMAN LTD HOLDCO | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | WINZIP COMPUTING, S L U | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | WINZIP HOLDINGS SPAIN, S L U | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | INTERVIDEO DIGITAL TECHNOLOGY CORP | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jun 07 2013 | JPMORGAN CHASE BANK, N A | COREL US HOLDINGS, LLC | RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS | 030591 | 0383 | |
Jul 25 2013 | Corel Corporation | 8324450 CANADA INC | CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE, AND REPLACE THE ASSIGNMENT PREVIOUSLY RECORDED ON REEL 030427 FRAME 0331 ASSIGNOR S HEREBY CONFIRMS THE ASSIGNMENT TO 8324450 CANADA INC | 030986 | 0268 | |
Sep 27 2013 | 8324450 CANADA INC | 8324450 Delaware LLC | ENTITY DOMICILE CHANGE | 034651 | 0817 | |
Oct 22 2013 | 8324450 CANADA INC | COREL SOFTWARE LLC | CHANGE OF NAME SEE DOCUMENT FOR DETAILS | 047675 | 0950 | |
Sep 05 2018 | COREL SOFTWARE LLC | Corel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 048067 | 0586 | |
Jul 02 2019 | CASCADE BIDCO CORP | CITIBANK, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 049678 | 0950 | |
Jul 02 2019 | COREL INC | CITIBANK, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 049678 | 0950 | |
Jul 02 2019 | CLEARSLIDE INC | CITIBANK, N A | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 049678 | 0950 | |
Jul 02 2019 | CASCADE BIDCO CORP | Cantor Fitzgerald Securities | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 049678 | 0980 | |
Jul 02 2019 | COREL INC | Cantor Fitzgerald Securities | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 049678 | 0980 | |
Jul 02 2019 | CLEARSLIDE INC | Cantor Fitzgerald Securities | SECURITY INTEREST SEE DOCUMENT FOR DETAILS | 049678 | 0980 |
Date | Maintenance Fee Events |
Feb 09 2007 | STOL: Pat Hldr no Longer Claims Small Ent Stat |
May 08 2009 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Jun 01 2011 | ASPN: Payor Number Assigned. |
Jun 06 2013 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Jul 14 2017 | REM: Maintenance Fee Reminder Mailed. |
Nov 29 2017 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Nov 29 2017 | M1556: 11.5 yr surcharge- late pmt w/in 6 mo, Large Entity. |
Date | Maintenance Schedule |
Dec 06 2008 | 4 years fee payment window open |
Jun 06 2009 | 6 months grace period start (w surcharge) |
Dec 06 2009 | patent expiry (for year 4) |
Dec 06 2011 | 2 years to revive unintentionally abandoned end. (for year 4) |
Dec 06 2012 | 8 years fee payment window open |
Jun 06 2013 | 6 months grace period start (w surcharge) |
Dec 06 2013 | patent expiry (for year 8) |
Dec 06 2015 | 2 years to revive unintentionally abandoned end. (for year 8) |
Dec 06 2016 | 12 years fee payment window open |
Jun 06 2017 | 6 months grace period start (w surcharge) |
Dec 06 2017 | patent expiry (for year 12) |
Dec 06 2019 | 2 years to revive unintentionally abandoned end. (for year 12) |