The present invention relates to an image processing technique, and in particular to a method for restoring a compressed image by using a hybrid motion compensation discrete cosine transform (hybrid MC/DCT) mechanism, including: a step of defining a smoothing functional having a smoothing degree of an image and reliability for an original image by pixels having an identical property in image block units; and a step of computing a restored image by performing a gradient operation on the smoothing functional in regard to the original image, thereby preventing the blocking artifacts and the ringing effects in regard to the pixels having an identical property in image blocks. In one embodiment, the method includes obtaining a pixel value in a current block and at least one adjacent pixel value, obtaining a difference value between the pixel value in the current block and the adjacent pixel value, and obtaining a smoothing value of the current image based on the difference value. A pixel value around a boundary of the block is smoothed based on a threshold value and the smoothing value.
|
0. 27. A method of decoding a current image, comprising:
obtaining a pixel value in a current block and at least one adjacent pixel value;
obtaining a difference value between the pixel value in the current block and the adjacent pixel value;
obtaining a smoothing value of the current image based on the difference value; and
smoothing a pixel value around a boundary of the block based on a threshold value and the smoothing value; and wherein
the threshold value is based on quantization information of at least a partially restored portion of the current image.
0. 1. A method for restoring a compressed image of an image processing system, comprising:
a step for defining a smoothing functional having a smoothing degree of an image and reliability for an original image by pixels having an identical property in image block units; and
a step for computing a restored image by performing a gradient operation on the smoothing functional in regard to the original image;
wherein the smoothing functional M(f) comprises a sum of a smoothing functional MVB(f) for pixels positioned at the boundary of a block in a vertical direction, a smoothing functional MVW(f) for pixels positioned inside the block in a horizontal direction, a smoothing functional MHB(f) for pixels positioned at the boundary of a block in a horizontal direction, a smoothing functional MHW(f) for pixels positioned inside the block in a horizontal direction, a smoothing functional MT(f) for pixels moved and compensated in the temporal section, “f” indicating the original image.
0. 2. The method according to
0. 3. The method according to
MVB(f)=∥QVBf∥2+αVB∥g−f∥w12 MHB(f)=∥QHBf∥2+αHB∥g−f∥w22 MVW(f)=∥QVWf∥2+αVW∥g−f∥w32 MHW(f)=∥QHWf∥2+αHW∥g−f∥w42 MT(f)=∥QTf∥2+αT∥g−f∥w52 QVB, QVW, QHB, QHW, QT indicating high pass filters for smoothing the respective pixels, αVB, αVW, αHB, αHW, αT being regularization parameters, g being a reconstructed image, and W1, W2, W3, W4, W5 indicating diagonal matrixes for determining whether each group has an element.
0. 4. The method according to
0. 5. The method according to
Q2p(m,n) indicating a quantization variable of a macro block including an (m,n)th pixel of a two-dimensional image.
0. 6. The method according to
0. 7. The method according to
0. 8. The method according to
0. 9. The method according to
fk+1=fk+β[Ag−Bfk], A=αVBW1+αHBW2+αVWW3+αHWW4+αTW5 B=(QTVBQVB+QTHBQHB+QTVWQVW+QTHWQHW+QTTQT)+A and, β is a relaxation parameter having a convergence property, and computed at the range of
an eigen value λ(A) of the matrix A being replaced by a fixed value.
0. 10. The method according to
0. 11. The method according to
0. 12. The method according to
0. 13. The method according to
0. 14. The method according to
0. 15. The method according to
0. 16. An apparatus for restoring a compressed image of an image processing system, comprising:
a decoder for decoding a coded image signal, and for outputting information of the restored image, such as the decoded image, a quantization variable, a macro block type and a motion vector; and
a post processing unit for including the information of the restored image inputted from the image decoder, for defining a smoothing functional including a sum of a smoothing functional MVB(f) for pixels positioned at the boundary of a block in a vertical direction, a smoothing functional MVW(f) for pixels positioned inside the block in a horizontal direction, a smoothing functional MHB(f) for pixels positioned at the boundary of a block in a horizontal direction, a smoothing functional MHW(f) for pixels positioned inside the block in a horizontal direction, a smoothing functional MT(f) for pixels moved and compensated in the temporal section, “f” indicating the original image, and for performing a gradient operation on the smoothing functional in regard to the original image,
the smoothing functional including a regularization parameter having weight of reliability for the original image.
0. 17. A method for restoring a compressed image of an image processing system, comprising:
a step for defining a smoothing functional having a smoothing degree of an image and reliability for an original image by pixels having an identical property in image block units;
a step for computing a restored image by performing a gradient operation on the smoothing functional in regard to the original image; and
a step for computing an iterative solution in regard to the restored image, after computing the restored image.
0. 18. The method according to
0. 19. The method according to
0. 20. The method according to
MVB(f)=∥QVBf∥2+αVB∥g−f∥w12 MHB(f)=∥QHBf∥2+αHB∥g−f∥w22 MVW(f)=∥QVWf∥2+αVW∥g−f∥w32 MHW(f)=∥QHWf∥2+αHW∥g−f∥w42 MT(f)=∥QTf∥2+αT∥g−f∥w52 QVB, QVW, QHB, QHW, QT indicating high pass filters for smoothing the respective pixels, αVB, αVW, αHB, αHW, αT being regularization parameters, g being a reconstructed image, and W1, W2, W3, W4, W5 indicating diagonal matrixes for determining whether each group has an element.
0. 21. The method according to
0. 22. The method according to
Q2p(m,n) indicating a quantization variable of a macro block including an (m,n)th pixel of a two-dimensional image.
0. 23. The method according to
0. 24. The method according to
0. 25. The method according to
fk+1=fk+β[Ag−Bfk], A=αVBW1+αHBW2+αVWW3+αHWW4+αTW5
B=(QTVBQVB+QTHBQHB+QTVWQVW+QTHWQHW+QTTQT)+A and, β is a relaxation parameter having a convergence property, and computed at the range of
an eigen value λ(A) of the matrix A being replaced by a fixed value.
0. 26. An apparatus for restoring a compressed image of an image processing system, comprising:
a decoder for decoding a coded image signal, and for outputting information of the restored image, such as the decoded image, a quantization variable, a macro block type and a motion vector; and
a post processing unit for including the information of the restored image inputted from the image decoder, for defining a smoothing functional including a smoothing degree of the image and reliability of an original image block unit, and for performing a gradient operation on the smoothing functional in regard to the original image,
the smoothing functional including a regularization parameter having weight of reliability for the original image.
0. 28. The method of
0. 29. The method of
0. 30. The method of
0. 31. The method of
0. 32. The method of
|
|||||||||||||||||
Here, “g” and “f” indicate row vectors re-arranged in a stack-order, namely a scanning order, and “n” indicates a quantization error. When it is presumed that a size of the image is M×M, the original image (f), the reconstructed image (g) and (n) are column vectors having a size of M×1.
An original pixel for the original image (f) is represented by f(i,j). Here, “i” and “j” indicate a position of the pixel in the image.
The 8×8 pixels in the block are classified into the pixels having an identical property. That is, the pixels are divided in accordance with their position, vertical direction, horizontal direction and smoothing variation in the temporal section. Accordingly, it is defined that a set of the pixels positioned at a boundary of the block in a vertical direction is CVB, a set of the pixels positioned inside the block in the vertical direction is CVW, a set of the pixels positioned at a boundary of a block in a horizontal direction is CHB, a set of the pixels positioned inside the block in the horizontal direction is CHW, and a set of the pixels moved and compensated in the temporal section is CT. The sets CVB, CVW, CHB, CHW, CT are represented by the following expressions.
CVB={f(i,j): i mod 8=0,1, and j=0,1, . . . , M−1}
CVW={f(i,j): i mod 8=0,1, and j=0,1, . . . , M−1} (2)
CHB={f(i,j): j mod 8=0,1, and i=0,1, . . . , M−1}
CHW={f(i,j): j mod 8=0,1, and i=0,1, . . . , M−1}
CT={f(i,j): f(i,j)εMBinter or f(i,j)εMBnot coded}
Here, the set CT is a set of the pixels having a macro block type of “inter” or “not coded” in order to remove temporal redundancy information.
The smoothing functional M(f) for using the regularization restoration method from the above-defined sets CVB, CVW, CHB, CHW, CT is defined as follows.
M(f)=MVB(f)+MHB(f)+MVW(f)+MHW(f)+MT(f) (3)
Here, MVB(f) is a smoothing functional for the set CVB, MHB(f) is a smoothing functional for CHB, MVW(f) is a smoothing functional for the set CVW, MHW(f) is a smoothing functional for the set CHW, and MT(f) is a smoothing functional for the set CT. The smoothing fuctionals are respectively defined as follows.
MVB(f)=∥QVBf∥2+αVB∥g−f∥w12
MHB(f)=∥QHBf∥2+αHB∥g−f∥w22
MVW(f)=∥QVWf∥2+αVW∥g−f∥w32
MHW(f)=∥QHWf∥2+αHW∥g−f∥w42
MT(f)=∥QTf∥2+αT∥g−f∥w52
Here, first terms in each expression indicate a smoothing degree for the original pixel (reference pixel) and adjacent pixel, and second terms indicate reliability for the original pixel and the restored pixel. “∥.∥” indicates the Euclidean norm. QVB, QVW, QHB, QHW, QT indicate high pass filters for smoothing the pixels in the sets CVB, CVW, CHB, CHW, CT.
The first term at the right side is represented by the following expression.
The smoothing functionals represented by Expression (4) are quadratic equations, respectively. Thus, local minimizers of each smoothing functional become global minimizers.
αVB, αHB, αVW, αHW, αT included in the second terms of Expression (4) are regularization parameters in regard to each set, indicate a ratio of the smoothing degree and reliability, and imply an error element. W1, W2, W3, W4, W5 indicate diagonal matrixes having a size of M×M in order to determine whether each set has an element, and have a value of “1”, or “0” according to whether each pixel is included in a corresponding set. That is, if the respective pixels are included in the corresponding sets, the value of the diagonal elements is “0”. If not, the value of the diagonal elements is
Thereafter, the regularization parameters, αVB, αHB, αVW, αHW, αT are approximated as follows.
Approximation of Regularization Parameters
Approximation of the regularization parameters is a major element determining performance of the smoothing functional. In order to reduce the computation amount, presumptions are made as follows.
Under the above presumptions, in case a set theoretic is applied, each regularization parameter is approximated as follows.
Here, Q2p(m,n) is a quantization variable of a macro block including a (m,n)th pixel of a two-dimensional image.
In Expression (6), denominator terms of the respective regularization parameters are a sum of the energy for the quantization noise of the elements included in each group. As described above, the values of the regularization parameters may be easily computed by applying the set theoretic under the two presumptions.
Computing Pixels to be Restored From Smoothing Functional
Only the original image needs to be computed. However, the smoothing functional includes a square term of the original image. Accordingly, in order to compute the original image, a gradient operation is carried out on the smoothing functional in regard to the original image. A result value thereof is “0”, and represented by the following expression.
∇fM(f)=2QTVBQVB+2QTHBQHB+
2QTVW
QVW+2QTHWQHW+2QT
TQT−2αVBWT1W1(
g−f)−2αHBWT2W2(
g−f)−2αVWWT3W3(
g−f)−2αHWWT4W4(
g−f)−2αT(g−f)=0 (7)
Here, a superscript “T” indicates a transposition of the matrix.
A restored image similar to the original image (f) can be obtained by Expression (7). However, operation of an inverse matrix must be performed, and thus the computation amount is increased. Thus, in accordance with the present invention, the restored image is computed by an iterative technique which will now be explained.
Iterative Technique
When Expression (7) is iterated k times, an iterative solution fk+1 is represented by the following expression.
fk+1=fk+β[Ag−Bfk],
A=αVBW1+αHBW2+αVWW3+αHWW4+αTW5 (8)
B=(QTVBQVB+QTHBQHB+QTVWQVW+QTHWQHW+QTTQT)+A
In Expression (8), “β” is a relaxation parameter having a convergence property. Expression (8) can be represented by the following expression by computing consecutive iterative solutions.
(fk+1−fK)=(I−B)(fk−fk-1) (9)
Here, “I” is an identity matrix, and the matrix B has a positive definite property. Therefore, when the following condition is satisfied, the iterative solutions are converged.
∥I−B∥<1 (10)
Expression (10) can be summarized as follows.
In Expression (11), “λ(A)” depicts an eigen value of the matrix A. A considerable amount of computation is required to compute the eigen value λ(A). However, the high pass filters have a certain shape determined according to the positions of the respective pixels, regardless of the image. Accordingly, before computing Expression (8), the eigen value λ(A) can be replaced by a fixed value. The value may be computed by a power method which has been generally used in interpretation of numerical values.
For example, a computation process of an eigen value of an iterative solution will now be explained.
xk+1=Kxk
Here, “xk” is a vector of M×1, and “K” is a positive-definite symmetric M×M matrix. The eigen value λ′ of the matrix K is approximated as follows.
In the above expression, if “k” is to infinity, the eigen value λ′ is approximated to a real value.
Thus, the iterative solution represented by Expression (8) is computed. The next thing to be considered is a time of finishing the iterative technique, in order to determine the number of iteration. Here, two standards are set as follows.
Firstly, a predetermined threshold value is set before starting iteration, an image obtained after iteration, namely a partially-restored image is compared with the previously-set threshold value, and it is determined whether the iteration technique is continuously performed according to a comparison result.
Secondly, the iteration technique is performed as many as a predetermined number, and then finished.
According to the first standard, a predetermined threshold value is set in performing iteration, and thus a wanted value is obtained. However, although the iteration number is increased, it may happen that the predetermined threshold value is not reached. On the other hand, the second standard is performed by experience, but can reduce a computation amount. Therefore, the two standards may be selectively used according to the design specification.
In the step S4, the DCT is performed on the restored image corresponding to the iterative solution fk+1 obtained in the step S3. An (u,v)th DCT coefficient of the two-dimensional restored image is expressed as Fk+1(u,v), and must exist in the following section in accordance with a property of the quantization process.
G(u,v)−Qp≦Fk+1(u,v)≦G(u,v)+Qp (12)
Here, “Qp” is a maximum quantization error as explained above, and “G(u,v)” is a two-dimensional DCT coefficient obtained by performing the DCT on the reconstructed image (g). The DCT coefficients Fk+1(u,v) and G(u,v) are represented as follows. In Expression (13), “B” indicates a block DCT.
Fk+1(u,v)=(Bfk+1)(u,v), and G(u,v)=(Bg)(u,v) (13)
In the step S6, a section of the DCT coefficient of the restored image is set as in Expression (12). Accordingly, in case the DCT coefficient Fk+1(u,v) of the restored image is not in the predetermined section, it must be projected as follows. A projection process is carried out in the step S7, and represented by Expression (14).
P(Fk+1(u,v))=G(u,v)−Qp, if Fk+1(u,v)<G(u,v)−Qp
P(Fk+1(u,v))=G(u,v)+Qp, if Fk+1(u,v)>G(u,v)−Qp 14
P(Fk+1(u,v))=Fk+1(u,v), otherwise.
Expression (14) will now be described.
When Fk+1(u,v) is smaller than G(u,v)−Qp, the projected restored image P(Fk+1(u,v)) is mapped to G(u,v)−Qp. In case Fk+1(u,v) is greater than G(u,v)+Qp, the projected restored image P(Fk+1(u,v)) is mapped to G(u,v)+Qp. Otherwise, the projected restored image P(Fk+1(u,v)) is mapped as it is.
In the step S8, the inverse DCT is performed on the mapped image P(Fk+1(u,v)) in the spatial section. The finally restored image is represented by Expression (14).
fk+1=BTPBfk+1 (15)
Here, “B” indicates the DCT, “P” indicates mapping, and “BT” indicates the inverse DCT.
The restored image is stored in a frame memory in the post processing unit 220 (Step S9). The post processing unit 220 performs motion compensation based on the motion vector MV (Step S10). The motion and compensation image is employed for generation of the regularization parameter for a succeeding image and the iteration technique.
The post processing unit 220 outputs the restored motion and compensation image as a video signal to a display (not shown) (Step S11).
As discussed earlier, the present invention can restrict a section of the restored image for the respective pixels by using the various regularization parameters. In addition, the present invention prevents flickering which may occur in the dynamic image compression technique.
Consequently, the present invention adaptively prevents the blocking artifacts and the ringing effects for the pixels having an identical property in image block units, and thus can be widely used for the products of the hybrid MC-DCT mechanism.
As the present invention may be embodied in several forms without departing from the spirit or essential characteristics thereof, it should also be understood that the above-described embodiment is not limited by any of the details of the foregoing description, unless otherwise specified, but rather should be construed broadly within its spirit and scope as defined in the appended claims, and therefore all changes and modifications that fall within the meets and bounds of the claims, or equivalences of such meets and bounds are therefore intended to be embraced by the appended claims.
| Patent | Priority | Assignee | Title |
| Patent | Priority | Assignee | Title |
| 4719885, | Jan 31 1986 | Hitachi, Ltd. | Electronic control fuel injection device |
| 5122875, | Feb 27 1991 | GENERAL ELECTRIC COMPANY, A CORP OF NY | An HDTV compression system |
| 5475434, | Aug 17 1993 | Goldstar Co. Ltd. | Blocking effect attenuation apparatus for high definition television receiver |
| 5557330, | Oct 26 1994 | U S BANK NATIONAL ASSOCIATION, AS COLLATERAL AGENT | Encoding video signals using selective pre-filtering |
| 5589829, | Oct 26 1994 | U S BANK NATIONAL ASSOCIATION, AS COLLATERAL AGENT | Decoding variable-length encoded signals |
| 5590064, | Oct 26 1994 | Intel Corporation | Post-filtering for decoded video signals |
| 5825934, | Apr 23 1991 | Canon Kabushiki Kaisha | Adaptive image coding method and apparatus |
| 5898457, | Dec 10 1993 | Ricoh Company, LTD | TV conference communication device and method of controlling the same |
| 6026232, | Jul 13 1995 | TIME WARNER ENTERTAINMENT CO , L P | Method and system to replace sections of an encoded video bitstream |
| 6115503, | Jul 04 1996 | Siemens Aktiengesellschaft | Method and apparatus for reducing coding artifacts of block-based image encoding and object-based image encoding |
| 6115505, | Sep 05 1997 | Fujitsu Limited | Area detection apparatus |
| 6144700, | May 14 1996 | QUARTERHILL INC ; WI-LAN INC | Method and apparatuses for removing blocking effect in a motion picture decoder |
| 6148109, | May 28 1996 | SUN PATENT TRUST | Image predictive coding method |
| 6192081, | Oct 26 1995 | MEDIATEK, INC | Apparatus and method for selecting a coding mode in a block-based coding system |
| 6222641, | Jul 01 1998 | Electronics for Imaging, Inc. | Method and apparatus for image descreening |
| 6222881, | Oct 18 1994 | Intel Corporation | Using numbers of non-zero quantized transform signals and signal differences to determine when to encode video signals using inter-frame or intra-frame encoding |
| 6246802, | Sep 30 1996 | NEC PERSONAL COMPUTERS, LTD | Image data processor |
| 6295326, | Mar 01 2000 | CALLAHAN CELLULAR L L C | Kalman filter based equalization for digital multicarrier communications systems |
| 6295376, | Jun 09 1997 | Hitachi, LTD | Image sequence coding method and decoding method |
| 6310962, | Aug 20 1997 | Samsung Electronics Co., Ltd.; SAMSUNG ELECTRONICS CO , LTD | MPEG2 moving picture encoding/decoding system |
| 6343141, | Oct 08 1996 | Alcatel-Lucent USA Inc | Skin area detection for video image systems |
| 6360014, | Sep 26 1997 | MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD | Image decoding method, image decoding apparatus, and data recording medium |
| 6381275, | Jun 28 1996 | Mitsubishi Denki Kabushiki Kaisha | Image coding apparatus and image decoding apparatus |
| 6415055, | Dec 12 1994 | Sony Corporation | Moving image encoding method and apparatus, and moving image decoding method and apparatus |
| 6738528, | May 22 1998 | Matsushita Electric Industrial Co., Ltd. | Block noise detector and block noise eliminator |
| 20030218781, | |||
| 20050243375, | |||
| 20070070423, | |||
| JP5308623, | |||
| JP8084342, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Aug 20 2007 | LG Electronics Ltd. | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Oct 19 2010 | ASPN: Payor Number Assigned. |
| Date | Maintenance Schedule |
| Jun 29 2013 | 4 years fee payment window open |
| Dec 29 2013 | 6 months grace period start (w surcharge) |
| Jun 29 2014 | patent expiry (for year 4) |
| Jun 29 2016 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Jun 29 2017 | 8 years fee payment window open |
| Dec 29 2017 | 6 months grace period start (w surcharge) |
| Jun 29 2018 | patent expiry (for year 8) |
| Jun 29 2020 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Jun 29 2021 | 12 years fee payment window open |
| Dec 29 2021 | 6 months grace period start (w surcharge) |
| Jun 29 2022 | patent expiry (for year 12) |
| Jun 29 2024 | 2 years to revive unintentionally abandoned end. (for year 12) |