An adaptive predictive encoding apparatus for encoding a video signal by utilizing correlation between frames in both the forward and reverse directions of the time axis. A prediction signal for use in deriving prediction error values to be encoded for a frame is selected by an adaptive prediction section, in units of blocks, from a plurality of mutually differently derived prediction signals, in accordance with the degree of correlation of the block with corresponding ones of a specific preceding independently encoded frame and a specific succeeding independently encoded frame. A complementary adaptive decoding apparatus receives the encoded information and reconstructs the video signal in accordance with information supplied to the adaptive decoding apparatus by the encoding signal.
|
1. An adaptive encoding apparatus for encoding an input video signal, said video signal comprising a sequence of frames each comprising successive pixel data the apparatus comprising:
encoder means for encoding successive blocks of a frame of said video signal, each of said blocks comprising a fixed-size array of said pixel data values; means for selecting one in every N of said frames to be transferred directly to said encoder means as a reference frame, to be encoded by intra-frame encoding, where N is a fixed integer of value greater than one; and adaptive prediction means for executing adaptive prediction processing, as a dependent frame, of each frame occurring between a preceding one and a succeeding one of said reference frames in said frame sequence, by deriving for the data values of each block of a dependent frame respective prediction error values based upon an optimum prediction signal selected from a plurality of prediction signals derived using a plurality of combinations of said preceding and succeeding reference frames.
2. An adaptive predictive encoding apparatus according to
means for deriving a first prediction signal based on a combination of pixel data of said preceding and succeeding reference frames, a second prediction signal derived only from said preceding reference frame, a third prediction signal derived only from said succeeding, and a non-prediction signal derived only from said dependent frame; and predictive mode selection means for selecting, for each of said blocks, one out of four prediction modes in which said first, second and third prediction signals and said non-prediction signal are respectively used in deriving predictive error values for respective pixel data of said block, to be sent to said encoder means and encoded thereby, said selection being based upon judgement of said errors, said predictive mode selection means further supplying to said encoding means, to be encoded thereby, predictive mode data indicating predictive modes which have been selected for respective ones of the blocks.
3. An adaptive predictive encoding apparatus according to
4. An adaptive predictive encoding apparatus according to
5. An adaptive predictive encoding apparatus according to
6. An adaptive decoding apparatus for decoding a video signal encoded by the apparatus of
decoding means for receiving and decoding the encoded reference frames, the prediction error values and a prediction mode signal that identifies which of said plurality of combinations of said preceding and succeeding reference frames were used during encoding to obtain said optimum prediction signal; prediction signal generating means responsive to the decoded prediction mode signal for reconstructing said optimum prediction signal; means for combining said prediction error values and said optimum prediction signal for each dependent frame to generate display information corresponding thereto; and means for outputting the decoded reference frames and the display information for each dependent frame in proper sequence to produce a video signal. 7. A decoding system for decoding video signals that have been encoded in an encoder by arranging said video signals into spaced-apart reference frames and dependent frames located therebetween; said reference frames being output from said encoder in encoded form and used therein to implement one of a plurality of prediction modes for adaptively predicting the display information in each of said dependent frames based upon the degree of correlation between each dependent frame and the reference frames which immediately precede and follow said dependent frame; said encoder thereby generating and outputting therefrom an encoded frame signal for each dependent frame and a prediction mode signal for identifying the prediction mode used to generate said frame signal, said decoding system comprising: decoding means for receiving and decoding each of the encoded reference frames, and the frame signals and the prediction mode signals associated with each dependent frame; processing means responsive to said prediction mode signal for reconstructing the display information for each dependent frame from its respective decoded frame signal and the reference frames which precede and follow said dependent frame; and means for outputting the decoded reference frames and the reconstructed display information generated for each of said dependent frames in proper sequence to produce a video signal. 8. The decoding system in accordance with claim 7, wherein said processing means includes memory means for storing the two decoded reference frames which respectively precede and follow each dependent frame; and means responsive to the prediction mode signal for generating and combining weighted values of said display information from said two reference frames to reconstruct the predicted display information for said dependent frame.
9. The decoding system in accordance with claim 8, including means for combining said predicted display information with said decoded frame signal to produce an output representing the display information for said dependent frame. 10. The decoding system in accordance with claim 8, wherein said weighted values are generated by multiplying said display information in the preceding reference frame by a first weighting coefficient and said display information in the following reference frame by a second weighting coefficient. 11. The decoding system in accordance with claim 10, wherein, in response to a first prediction mode signal, the first and second weighting coefficients are non-zero. 12. The decoding system in accordance with claim 11 wherein the weighting coefficients are selected such that the reference frame temporally closer to the dependent frame is given a greater weight than the other reference frame. 13. The decoding system in accordance with claim 10, wherein, in response to a second prediction mode signal, the second weighting coefficient is effectively zero. 14. The decoding system in accordance with claim 10, wherein, in response to a third prediction mode signal, the first weighting coefficient is effectively zero. 15. The decoding system in accordance with claim 7, wherein, in response to a fourth prediction mode signal, the decoded frame signal is output to represent the display information of said dependent frame. 16. A method for decoding and generating a video signal from video signals that have been encoded in an encoder by arranging said video signals into spaced-apart reference frames and dependent frames located therebetween, said reference frames being output from said encoder in encoded form and used therein to implement one of a plurality of prediction modes for adaptively predicting the display information in each of said dependent frames based upon the degree of correlation between each of said dependent frames and the reference frames which immediately precede and follow each of said dependent frames, said encoder generating and outputting therefrom an encoded frame signal for each of said dependent frames and a prediction mode signal for identifying the prediction mode used to generate said encoded frame signal, said method comprising the steps of: (a) receiving and decoding each of said encoded reference frames, said encoded frame signals and said prediction mode signal associated with each dependent frame; (b) reconstructing the display information for each dependent frame from the corresponding decoded frame signal and the decoded reference frames which precede and follow said dependent frame in accordance with said associated prediction mode signal; and (c) outputting the decoded reference frames and the reconstructed display information generated for each of said dependent frames in proper sequence to generate a video signal. 17. The method of claim 16 wherein step (b) includes the steps of storing the display information of the two decoded reference frames which respectively precede and follow said dependent frame and combining weighted values of said display information with said decoded frame signal to reconstruct the display information for said dependent frame. 18. The method in accordance with claim 17, wherein said weighted values are generated by multiplying said display information of the preceding reference frame by a first weighting coefficient and said display information in the following reference frame by a second weighting coefficient. 19. The method in accordance with claim 18, wherein, in response to a first prediction mode signal, the first and second weighting coefficients are non-zero. 20. The method in accordance with claim 19, wherein the weighting coefficients are selected such that the reference frame temporally closer to the dependent frame is given a greater weight than the other reference frame. 21. The method in accordance with claim 18, wherein, in response to a second prediction mode signal, the second weighting coefficient is effectively zero. 22. The method in accordance with claim 18, wherein, in response to a third prediction mode signal, the first weighting coefficient is effectively zero. 23. The method in accordance with claim 16, wherein, in response to a fourth prediction mode signal, the decoded frame signal is added to a fixed data value to reconstruct the display information of said dependent frame. |
The present application is a FIG. 6 is a block diagram of a decoding apparatus for decoding signals in accordance with the present invention.
FIG. 7 is a schematic diagram of the adaptive prediction section of the decoding apparatus of FIG. 6. (not shown in the drawings) shown in FIG. 6, the various quantization errors etc. that are present in the final decoded data will also be present in the data that are used in denying the prediction error values. This ensures a greater accuracy of prediction than would be the case if the independent frames of the input video signal data were to be written directly into the memories 12 and then 13.
With this embodiment, evaluation for determining the prediction mode is based upon error-squared values of prediction error values that are obtained directly from the input video signal. Greater accuracy of evaluation would be obtained by using the video signal data of the dependent frames after all of the encoding processing (including transform processing, and quantization) has been executed. However this would require additional circuits for executing the inverse of such encoding, i.e. for the inverse transform processing etc., increasing the circuit scale substantially and making the apparatus more difficult to realize in practical form.
As stated above, the DC component of the current frame signal is subtracted from the current frame signal in the subtractor 39, to thereby prevent an excessively high output value being produced by the cumulative adder 31. However if not compensated for, this will tend to produce an excessively high probability that mode 4 will be selected by the minimum value selector circuit 32, i.e. the output from the cumulative adder 31 will tend to have too low a value. For that reason, a compensating offset value B is added to the output from the cumulative adder 31 in the adder 37.
On the other hand, in cases where there are only small differences between the respective values of prediction error that are being produced from the prediction signal subtractor 21, subtractor 20 and subtractor 22, it is preferable to prevent unnecessary switching between the modes 1, 2 and 3. For that reason, a slight amount of bias is given towards the selection of mode 1 (2-dimensional linear prediction) by the minimum value selector circuit 32. This is done by subtracting an offset value A from the output of the cumulative adder 29, in the subtractor 36. This has the advantage of increasing the rate of selection of mode 1, and so enabling a reduction in the amount of encoded data that are produced by encoding the prediction mode data from the minimum value selector circuit 32, if entropy encoding using for example the Huffman code is employed in the variable-length encoder section 8.
The As shown in FIG. 6, the decoding apparatus for decoding the encoded data on line 48 that are transmitted from such an adaptive predictive encoded apparatus can be implemented very simply, by using the mode prediction data that are contained in the encoded output data. After the inverse of the variable-length encoding executed by the variable-length encoder section 8 has been performed, in variable length decoder 50, followed by dequantizing in dequantizer 51 and inverse transform processing in inverse orthogonal transform section 52, each independent frame is transferred successively to a first (succeeding) frame memory 54 and then to a second frame memory for use in processing (preceding) frame memory 55 for use in processing the dependent frames, corresponding to the memories 12 and 13 of FIG. 1 3, and are outputted on line 70 without further processing. The operation of switches 61 and 62 correspond to the operation of switches 2 and 5 in the encoder of FIG. 3. During processing of the independent frames, their contacts are in the respective positions "b" shown in FIG. 6 and during processing of the dependent frames, their contacts are switched to their respective "a" positions. Each block of a dependent frame is processed by performing the inverse of the operations performed by the adaptive prediction section 4 of FIG. 3 during the encoding process in adaptive prediction section 53 of the decoder, shown schematically in FIG. 7, depending upon the position of the movable contact of switch 64 which is controlled by the associated decoded prediction mode data for that block, as follows:
(1) If the prediction mode data on line 57 indicates that the block has been encoded in mode 1, then the pixel data values of the corresponding blocks of the corresponding preceding and succeeding independent frame (read out on lines 58 and 59 from the aforementioned two frame memories 54 and 55) are respectively multiplied by the weighting values W and (1-W), the results added in adder 63, and the resultant value added in adder 65 to the current frame signal supplied on line 56 to generate the output of the decoder on line 60.
(2) If the prediction mode data on line 57 indicates that the block has been encoded in mode 2, then the pixel data values of the corresponding blocks of the corresponding preceding independent frame on line 58 are added in adder 65 to the current frame signal on line 56 to generate the output of the decoder on line 60.
(3) If the prediction mode data on line 57 indicates that the block has been encoded in mode 3, then the pixel data values of the corresponding blocks of the corresponding succeeding independent frame supplied on line 59 are added in adder 65 to the current frame signal on line 56 to generate the output of the decoder on line 60.
(It will be apparent that a single circuit can be used to implement all of the functions (1), (2) and (3) above, by appropriately setting the weighting value W to either 1 or 0 for functions (2) and (3)).
(4) If the prediction mode data on line 57 indicates that the block has been encoded in mode 4, then the fixed value on line 66 (subtracted in the subtractor 23 of FIG. 2 of the encoder apparatus) is added in adder 65 to the current frame signal on line 56 to generate the output of the decoder on line 60.
It will be apparent that the decoder apparatus shown in FIGS. 6 and 7 for receiving an encoded output signal produced by an adaptive predictive encoder apparatus according to the present invention can have a simple configuration, and can for example by implemented by slightly modifying an encoder apparatus that is described in the aforementioned related U.S. application by the assignee of the present invention.
Patent | Priority | Assignee | Title |
10063863, | Jul 18 2003 | Microsoft Technology Licensing, LLC | DC coefficient signaling at small quantization step sizes |
10116959, | Jun 03 2002 | Microsoft Technology Licesning, LLC | Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation |
10284843, | Jan 25 2002 | Microsoft Technology Licensing, LLC | Video coding |
10368065, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
10382750, | Nov 18 2005 | Apple Inc. | Region-based processing of predicted pixels |
10554985, | Jul 18 2003 | Microsoft Technology Licensing, LLC | DC coefficient signaling at small quantization step sizes |
10659793, | Jul 18 2003 | Microsoft Technology Licensing, LLC | DC coefficient signaling at small quantization step sizes |
6317459, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Digital video signal encoder and encoding method |
6351545, | Dec 14 1999 | NIO INC | Motion picture enhancing system |
6408029, | Apr 02 1998 | Intel Corporation | Method and apparatus for simplifying real-time data encoding |
6449352, | Jun 20 1995 | MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD | Packet generating method, data multiplexing method using the same, and apparatus for coding and decoding of the transmission data |
6574278, | Apr 02 1998 | Intel Corporation | Method and apparatus for performing real-time data encoding |
6584226, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Method and apparatus for implementing motion estimation in video compression |
6647425, | Jul 03 1997 | Microsoft Technology Licensing, LLC | System and method for selecting the transmission bandwidth of a data stream sent to a client based on personal attributes of the client's user |
6707852, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Digital video signal encoder and encoding method |
6904174, | Dec 11 1998 | Intel Corporation | Simplified predictive video encoder |
6937657, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Motion video signal encoder and encoding method |
7046734, | Apr 02 1998 | Intel Corporation | Method and apparatus for performing real-time data encoding |
7054494, | Nov 30 1998 | Microsoft Technology Licensing, LLC | Coded block pattern decoding with spatial prediction |
7072396, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Motion video signal encoder and encoding method |
7092576, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Bitplane coding for macroblock field/frame coding type information |
7099515, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Bitplane coding and decoding for AC prediction status information |
7127114, | Nov 30 1998 | Microsoft Technology Licensing, LLC | Coded block pattern encoding with spatial prediction |
7139313, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Digital video signal encoder and encoding method |
7154951, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Motion video signal encoder and encoding method |
7200275, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
7215384, | Apr 02 1998 | Intel Corporation | Method and apparatus for simplifying field prediction motion estimation |
7224731, | Jun 28 2002 | ZHIGU HOLDINGS LIMITED | Motion estimation/compensation for screen capture video |
7231091, | Sep 21 1998 | Intel Corporation | Simplified predictive video encoder |
7263127, | Apr 02 1998 | Intel Corporation | Method and apparatus for simplifying frame-based motion estimation |
7280700, | Jul 05 2002 | Microsoft Technology Licensing, LLC | Optimization techniques for data compression |
7289673, | Nov 30 1998 | Microsoft Technology Licensing, LLC | Decoding macroblock type and coded block pattern information |
7317839, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Chroma motion vector derivation for interlaced forward-predicted fields |
7352905, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Chroma motion vector derivation |
7379607, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
7408990, | Nov 30 1998 | Microsoft Technology Licensing, LLC | Efficient motion vector coding for video compression |
7426308, | Jul 18 2003 | Microsoft Technology Licensing, LLC | Intraframe and interframe interlace coding and decoding |
7499495, | Jul 18 2003 | Microsoft Technology Licensing, LLC | Extended range motion vectors |
7529302, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Four motion vector coding and decoding in bi-directionally predicted interlaced pictures |
7555167, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
7567617, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Predicting motion vectors for fields of forward-predicted interlaced video frames |
7577200, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Extended range variable length coding/decoding of differential motion vector information |
7590179, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Bitplane coding of prediction mode information in bi-directionally predicted interlaced pictures |
7599438, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Motion vector block pattern coding and decoding |
7606308, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Signaling macroblock mode information for macroblocks of interlaced forward-predicted fields |
7606311, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Macroblock information signaling for interlaced frames |
7609763, | Jul 18 2003 | Microsoft Technology Licensing, LLC | Advanced bi-directional predictive coding of video frames |
7616692, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Hybrid motion vector prediction for interlaced forward-predicted fields |
7620106, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Joint coding and decoding of a reference field selection and differential motion vector information |
7623574, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Selecting between dominant and non-dominant motion vector predictor polarities |
7630438, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Direct mode motion vectors for Bi-directionally predicted interlaced pictures |
7646810, | Jan 25 2002 | Microsoft Technology Licensing, LLC | Video coding |
7664177, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Intra-coded fields for bi-directional frames |
7664871, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Methods and apparatus for communication media commands and data using the HTTP protocol |
7680185, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Self-referencing bi-directionally predicted frames |
7738554, | Jul 18 2003 | Microsoft Technology Licensing, LLC | DC coefficient signaling at small quantization step sizes |
7761585, | Mar 14 1997 | Microsoft Technology Licensing, LLC | Techniques for automatically detecting protocols in a computer network |
7826530, | Jan 14 2005 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | Use of out of order encoding to improve video quality |
7852936, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Motion vector prediction in bi-directionally predicted interlaced field-coded pictures |
7924920, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Motion vector coding and decoding in interlaced frame coded pictures |
7925774, | May 30 2008 | ZHIGU HOLDINGS LIMITED | Media streaming using an index file |
7949775, | May 30 2008 | ZHIGU HOLDINGS LIMITED | Stream selection for enhanced media streaming |
8031777, | Nov 18 2005 | Apple Inc | Multipass video encoding and rate control using subsampling of frames |
8064520, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Advanced bi-directional predictive coding of interlaced video |
8189666, | Feb 02 2009 | Microsoft Technology Licensing, LLC | Local picture identifier and computation of co-located information |
8233535, | Nov 18 2005 | Apple Inc | Region-based processing of predicted pixels |
8254455, | Jun 30 2007 | Microsoft Technology Licensing, LLC | Computing collocated macroblock information for direct mode macroblocks |
8290288, | Nov 30 1998 | Microsoft Technology Licensing, LLC | Encoding macroblock type and coded block pattern information |
8295343, | Nov 18 2005 | Apple Inc | Video bit rate control method |
8370887, | May 30 2008 | ZHIGU HOLDINGS LIMITED | Media streaming with enhanced seek operation |
8374245, | Jun 03 2002 | Microsoft Technology Licensing, LLC | Spatiotemporal prediction for bidirectionally predictive(B) pictures and motion vector prediction for multi-picture reference motion compensation |
8379722, | Jul 19 2002 | Microsoft Technology Licensing, LLC | Timestamp-independent motion vector prediction for predictive (P) and bidirectionally predictive (B) pictures |
8406300, | Jan 25 2002 | Microsoft Technology Licensing, LLC | Video coding |
8428374, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
8582903, | Nov 30 1998 | Microsoft Technology Licensing, LLC | Efficient macroblock header coding for video compression |
8625669, | Sep 07 2003 | Microsoft Technology Licensing, LLC | Predicting motion vectors for fields of forward-predicted interlaced video frames |
8638853, | Jan 25 2002 | Microsoft Technology Licensing, LLC | Video coding |
8687697, | Jul 18 2003 | Microsoft Technology Licensing, LLC | Coding of motion vector information |
8774280, | Jul 19 2002 | Microsoft Technology Licensing, LLC | Timestamp-independent motion vector prediction for predictive (P) and bidirectionally predictive (B) pictures |
8780997, | Nov 18 2005 | Apple Inc | Regulation of decode-side processing based on perceptual masking |
8781240, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
8819754, | May 30 2008 | ZHIGU HOLDINGS LIMITED | Media streaming with enhanced seek operation |
8873630, | Jun 03 2002 | Microsoft Technology Licensing, LLC | Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation |
8917768, | Jul 18 2003 | Microsoft Technology Licensing, LLC | Coding of motion vector information |
9049451, | Nov 18 2005 | Apple Inc. | Region-based processing of predicted pixels |
9077960, | Aug 12 2005 | Microsoft Technology Licensing, LLC | Non-zero coefficient block pattern coding |
9088785, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
9148668, | Jul 18 2003 | Microsoft Technology Licensing, LLC | Coding of motion vector information |
9185427, | Jun 03 2002 | Microsoft Technology Licensing, LLC | Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation |
9313509, | Jul 18 2003 | Microsoft Technology Licensing, LLC | DC coefficient signaling at small quantization step sizes |
9538189, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
9571854, | Jun 03 2002 | Microsoft Technology Licensing, LLC | Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation |
9706201, | Nov 18 2005 | Apple Inc. | Region-based processing of predicted pixels |
9774852, | Dec 17 2001 | Microsoft Technology Licensing, LLC | Skip macroblock coding |
9888237, | Jan 25 2002 | Microsoft Technology Licensing, LLC | Video coding |
Patent | Priority | Assignee | Title |
4383272, | Apr 13 1981 | MULTIMEDIA PATENT TRUST C O | Video signal interpolation using motion estimation |
4651207, | Mar 05 1984 | ANT Nachrichtentechnik GmbH | Motion adaptive interpolation of television image sequences |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Dec 28 1992 | Victor Company of Japan Limited | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Jun 12 1996 | ASPN: Payor Number Assigned. |
Jun 22 1998 | M184: Payment of Maintenance Fee, 8th Year, Large Entity. |
Jun 07 2002 | M185: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Feb 20 1999 | 4 years fee payment window open |
Aug 20 1999 | 6 months grace period start (w surcharge) |
Feb 20 2000 | patent expiry (for year 4) |
Feb 20 2002 | 2 years to revive unintentionally abandoned end. (for year 4) |
Feb 20 2003 | 8 years fee payment window open |
Aug 20 2003 | 6 months grace period start (w surcharge) |
Feb 20 2004 | patent expiry (for year 8) |
Feb 20 2006 | 2 years to revive unintentionally abandoned end. (for year 8) |
Feb 20 2007 | 12 years fee payment window open |
Aug 20 2007 | 6 months grace period start (w surcharge) |
Feb 20 2008 | patent expiry (for year 12) |
Feb 20 2010 | 2 years to revive unintentionally abandoned end. (for year 12) |