The present invention relates to a method for recovering a compressed image for an image processing technique and an apparatus therefor. In the present invention, a cost function is defined in consideration with a directional characteristic of the pixels which will be recovered and a plurality of pixels of the recovering pixels. In addition, a regularization parameter variable having a certain weight is obtained from the cost function, and the regularization parameter variable is approximated using the compressed pixel for thereby obtaining a recovering pixel. The regularization parameter variable has a weight of a reliability with respect to the original image and a weight of a smoothing degree of the original image. In the methods and apparatuses for filtering, a filtering method is selected from filtering methods having different filtering strengths based on whether a pixel being filleted is in an intra-coded portion of an image.

Patent
   RE42677
Priority
Nov 03 1998
Filed
Jan 25 2006
Issued
Sep 06 2011
Expiry
Oct 29 2019

TERM.DISCL.
Assg.orig
Entity
Large
0
25
EXPIRED
0. 27. A method of filtering an image, comprising:
selecting, with a filter apparatus, a filtering method from filtering methods having different filtering strengths based on whether a pixel being filtered is in an intra-coded portion of an image; and
filtering, with the filter apparatus, the pixel using the selected filtering method, the selected filtering method adjusting a degree of filtering based on a difference value, the difference value being based on the pixel being filtered and a neighboring pixel.
0. 44. A method of filtering an image, comprising:
checking, with a filter apparatus, whether a first pixel to be filtered is in an intra-coded portion of an image and whether a neighboring pixel is in the intra-coded portion of the image;
selecting, with the filter apparatus, a filtering method having a filtering strength based on a result of the checking step, the selected filtering method adjusting a degree of filtering based on a difference value, the difference value being based on the pixel being filtered and a neighboring pixel; and
filtering, with the filter apparatus, the first pixel with the filtering method.
0. 1. A method for recovering a compressed motion picture, comprising the steps of:
defining a cost function having a smoothing degree of an image and a reliability with respect to an original image in consideration of the directional characteristics of the pixels which will be recovered and a plurality of pixels near the pixels which will be recovered;
obtaining a regularization parameter variable having a weight value of the reliability with respect to the original image based on a cost function; and
approximating the regularization parameter variable using the compressed pixel and obtaining a pixel which will be recovered,
wherein said regularization parameter variable is a weight value with respect to reliability and is determined based on a difference between the original pixel and the compressed pixel and a difference value between the original pixel and the neighboring pixel.
0. 2. The method of claim 1, wherein said cost function includes another cost function for setting an interrelationship of a time region with respect to the recovering pixel when the pixel which will be recovered is in an inter macro block.
0. 3. The method of claim 1, wherein said cost function includes another cost function which is defined based on a smoothing degree which is obtained by computing a difference between the recovering pixel and the neighboring pixel, a reliability of the original image obtained by computing a difference between the original image and the compressed image, and an interrelationship of a time region of the pixels of the block having a motion information.
0. 4. The method of claim 1, wherein said plurality of neighboring pixels are the pixels which are neighboring in the upper, lower, left and right side directions of the recovering pixels.
0. 5. The method of claim 1, wherein said difference value between the original pixel and the compressed pixel is approximated based on a quantizing maximum difference, and a difference value between the original pixel and the neighboring pixel is approximated based on a difference value between the compressed pixel and the neighboring compressed pixel.
0. 6. The method of claim 1, after the step for obtaining the recovering pixel, further comprising a step for performing a DCT with respect to the recovering pixels, projecting the recovering pixels in accordance with pixel value which will be processed, and performing a reverse DCT with respect to the projected images, and in said projecting step, a recovering image is projected at a subset for setting a range of DCT coefficients of a compressed image, and a maximum quantizing difference of the macro block is included in the subset.
0. 7. The method of claim 1, wherein in said step for approximating the regularization parameter variable, a quantizing maximum difference of a macro block unit is a quantizing variable, a quantizing difference is uniformly allocated to each pixel in a corresponding macro block, and the non-uniform values between two pixels of the original image are statistically similar to the non-uniform values between two pixels of the compressed image.
0. 8. The method claim 1, wherein said regularization parameter variable includes a weight value of a smoothing degree of the original image based on the cost function.
0. 9. The method of claim 8, wherein when the pixels of the current macro block are the same as the pixels of the previously transmitted macro block, the recovered pixel values are substituted for the current pixel values with respect to the macro block of the previous image.
0. 10. The method of claim 8, wherein in said step for approximating the regularization parameter variable, a quantizing difference of each pixel is set based on a function of a quantizing variable set by the unit of a macro block, and a weight value is added to the pixel based on the pixel position.
0. 11. In a method for recovering a compressed motion image for processing an original pixel f(i,j) based on a DCT by the unit of macro blocks of a M×M size, quantizing the DCT-processed coefficient, transmitting together with motion vector information, reversely quantizing and reversely DCT-processing the compressed pixel g(i,j) and recovering an image similar to the original image, a method for recovering a compressed motion picture, comprising the steps of:
defining a cost function M(i,j) having a smoothing degree of an image and a reliability with respect to an original image as a pixel unit in consideration of a directional characteristic between the pixels which will be recovered and the pixels neighboring the pixels which will be recovered;
adaptively searching a regularization parameter variable having a weight of a reliability with respect to the original image from the cost function M(i,j); and
obtaining a projected pixel P(F(u,v)) using a projection method for mapping the pixels which will be recovered in accordance with a range value of the pixels which will be recovered,
wherein said regularization parameter variable is a weight value with respect to reliability and is determined based on a difference between the original pixel and the compressed pixel and a difference value between the original pixel and the neighboring pixel.
0. 12. The method of claim 11, wherein said cost function M(i,j) is formed of a cost function MHL(f(i,j)) which represents a smoothing degree and a reliability with respect to an original pixel f(i,j) and a left side neighboring pixel f(i,j−1), a cost function MHR(f(i,j)) which represents a smoothing degree and a reliability with respect to the original pixel f(ij)and a right side neighboring pixel f(i,j+1), a cost function MVT(f(i,j)) which represents a smoothing degree and a reliability with reflect to the original pixel f(i,j) and an upper side neighboring pixel f(i−1,j), a cost function MVD(f(i,j)) which represents a smoothing degree and a reliability with respect to the original pixel f(i,j) and a lower side neighboring pixel f(i+1,j), and a cost function MT(f(i,j)) for setting an interrelationship of a time region with respect to the original pixel.
0. 13. The method of claim 12, wherein each cost function is obtained according to the following equations:

MHL(f(i,j))=[f(i,j)−f(i,j−1)]2HL[g(i,j)−f(i,j)]2

MHR(f(i,j))=[f(i,j)−f(i,j−1)]2HR[g(i,j)−f(i,j)]2

MVT(f(i,j))=[f(i,j)−f(i,j−1)]2VT[g(i,j)−f(i,j)]2

MVD(f(i,j))=[f(i,j)−f(i,j+1)]2VD[g(i,j)−f(i,j)]2

MT(f(i,j))=[f(i,j)−fMC(i,j)]2T[g(i,j)−f(i,j)]2
where fMC(i,j) represents a motion compensated pixel, αHL, αHR, αVT, αVD and αT represent a regulation parameter variable with respect to each cost function.
0. 14. The method of claim 13, wherein the pixel f(i,j) which will be recovered is obtained based on the following equation when the pixel is included in an inter macro block,
f ( i , j ) = f ( i , j - 1 ) + f ( i , j + 1 ) + f ( i - 1 , j ) + f ( i + 1 , j ) + f MC ( i , j ) + α TOT g ( i , j ) 5 + α TOT
where, αTOTHLHRVTVDT, and the pixel f(i,j) which will be recovered is obtained based on the following equation when the pixel is included in an intra macro block,
f ( i , j ) = f ( i , j - 1 ) + f ( i , j + 1 ) + f ( i - 1 , j ) + f ( i + 1 , j ) + α TOT g ( i , j ) 4 + α TOT
where αTOTHLHRVTVD.
0. 15. The method of claim 13, wherein said regularization parameter variables αHL, αHR, αVT, αVD, αT are obtained by approximations as follows:
α HL = [ g ( i , j ) - g ( i , j - 1 ) ] 2 Q p l 2 , α HR = [ g ( i , j ) - g ( i , j + 1 ) ] 2 Q p l 2 α VT = [ g ( i , j ) - g ( i - 1 , j ) ] 2 Q p l 2 , α VD = [ g ( i , j ) - g ( i + 1 , j ) ] 2 Q p l 2 α T = [ g ( i , j ) - f MC ( i , j ) ] 2 Q p l 2
where Qpl represents a quantizing variable of the l-th macro block.
0. 16. The method of claim 11, wherein in said step for obtaining the projected pixel P(F(u,v)), when (u,v)-th value F(u,v) of two-dimensional DCT coefficient of the original image is smaller than G(u,v)−Qpl, the projected pixel P(F(u,v)) is mapped to G(u,v)−Qpl, and when the value F(u,v) is larger than G(u, v)+Qpl, the projected pixel P(F(u, v)) is mapped to G(u, v)+Qpl, otherwise, the projected pixel P(F(u,v)) is mapped to F(u,v), where G(u,v) represents (u,v)th value of the two-dimensional DCT coefficient of the compression image, and Qpl represents a quantizing maximum difference of the l-th macro block.
0. 17. The method of claim 11, further comprising the following steps which are repeatedly performed by k-times:
defining a cost function M(i,j) having a smoothing degree of an image and a reliability with respect to the original image by the unit of pixels in consideration with a directional characteristic between the pixels which will be recovered and the pixels neighboring the pixels which will be recovered;
adaptively searching a regularization parameter variable having a weight value of a reliability with respect to the original image from the cost function M(i,j); and
obtaining a projected pixel P(F(u,v) using a projection method for mapping the recovering pixel in accordance with a range value of the pixel which will be recovered, for thereby finally obtaining a recovering image.
0. 18. In a method for recovering a compressed motion image for processing an original pixel f(i,j) based on a DCT by the unit of macro blocks of a M×M size, quantizing the DCT-processed coefficient, transmitting together with motion vector information, reversely quantizing and reversely DCT-processing the compressed pixel g(i,j) and recovering an image similar to the original image, a method for recovering a compressed motion picture, comprising the steps of:
defining a cost function M(i,j) having a smoothing degree of an image and a reliability with respect to an original image as a pixel unit in consideration of a directional characteristic between the pixels which will be recovered and the pixels neighboring the pixels which will be recovered;
adaptively searching a regularization parameter variable having a weight of a reliability with respect to the original image from the cost function M(i,j); and
obtaining a finally recovered image of a spatial region by obtaining a block DCT coefficient based on a block DCT and obtaining a projected pixel P(F(u,v)) by a projection method for mapping the pixels which will be recovered in a range value of the pixel for processing the block DCT coefficient, and performing a reverse DCT,
wherein said regularization parameter variable is a weight value with respect to reliability and is determined based on a difference between the original pixel and the compressed pixel and a difference value between the original pixel and the neighboring pixel.
0. 19. An apparatus for recovering a compressed motion picture, comprising:
an image decoding unit for outputting an information with respect to an image which will be recovered such as a decoded image, a quantized variable, a macro block type, and a motion type by decoding a coded image signal; and
a block process eliminating filter for defining a cost function based on a smoothing degree of an image and a reliability with respect to an original pixel in consideration of a directional characteristic between the neighboring pixel and the pixel which will be processed based on the pixels which will be recovered using an information with respect to the image which will be recovered inputted from the image decoding unit, adaptively searching a regularization parameter variable which provides a weight of a reliability with respect to the original image for each cost function, and recovering an original pixel using a projection method for mapping the pixels which will be recovered in accordance with a range value of the pixels which will be processed,
wherein said regularization parameter variable is a weight value with respect to reliability and is determined based on a difference between the original pixel and the compressed pixel and a difference value between the original pixel and the neighboring pixel.
0. 20. The apparatus of claim 19, further comprising:
a DCT unit for performing a DCT with respect to an image recovered by the block process eliminating filter;
a vector projection unit for projecting a pixel which will be recovered in accordance with a pixel value after the DCT process is performed; and
an IDCT unit for performing a reverse DCT with respect to the image projected by the vector projection unit.
0. 21. In a method for recovering a compressed motion image for processing an original pixel f(i,j) based on a DCT by the unit of macro blocks of a M×M size, quantizing the DCT-processed coefficient, transmitting together with motion vector information, reversely quantizing and reversely DCT-processing the compressed pixel g(i,j) and recovering an image similar to the original image, a method for recovering a compressed motion picture, comprising the steps of:
defining a cost function M(i,j) having a smoothing degree of an image and a reliability with respect to an original image as a pixel unit in consideration with a directional characteristic between the pixels which will be recovered and the pixels neighboring the pixels which will be recovered; and
adaptively searching a regularization parameter variable having a weight of a reliability with respect to the original image from the cost function M(i,j) and a weight value of a smoothing degree of the original image,
wherein said regularization parameter variable is a weight value with respect to reliability and is determined based on a difference between the original pixel and the compressed pixel and a difference value between the original pixel and the neighboring pixel.
0. 22. The method of claim 21, wherein said cost function is obtained based on the following equations:

ML(f(i,j))=αL(f(i,j))[f(i,j)−f(i,j−1)]2+(1−αL(f(i,j)))[g(i,j)−f(i,j)]2

MR(f(i,j))=αR(f(i,j))[f(i,j)−f(i,j+1)]2+(1−αR(f(i,j)))[g(i,j)−f(i,j)]2

MU(f(i,j))=αU(f(i,j))[f(i,j)−f(i−1,j)]2+(1−αU(f(i,j)))[g(i,j)−f(i,j)]2

MD(f(i,j))=αD(f(i,j))[f(i,j)−f(i−1,j)]2+(1−αD(f(i,j)))[g(i,j)−f(i,j)]2
where αL, αR, αU, αD are regularization parameter variables with respect to each cost function.
0. 23. The method of claim 22, wherein when the pixel of the current macro block is the same as the pixel of the previously transmitted macro block, in said pixel f(i,j) which will be recovered, the pixel value which is previously recovered with respect to the macro block of the previous image is substituted for the current pixel value, and otherwise the following Equation is obtained:
f ( i , j ) = α L f ( i , j - 1 ) + α R f ( i , j + 1 ) + α U f ( i - 1 , j ) + α D f ( i + 1 , j ) + ( 4 + α TOT ) g ( i , j ) 4
where αTOTLRUD.
0. 24. The method of claim 22, wherein said regularization parameter variables αL, αR, αU, αD are approximated as follows:
α L = K L Q P 2 [ g ( i , j ) - g ( i , j - 1 ) ] 2 + K L Q P 2 α R = K R Q P 2 [ g ( i , j ) - g ( i , j + 1 ) ] 2 + K R Q P 2 α U = K U Q P 2 [ g ( i , j ) - g ( i - 1 , j ) ] 2 + K U Q P 2 α D = K D Q P 2 [ g ( i , j ) - g ( i + 1 , j ) ] 2 + K D Q P 2
where KLQp2, KRQp2, KUQp2, KDQp2 are functions of the quantizing variable Qp, and constants KL, KR, KU, KD are weight values with respect to the regularization parameter variables αL, αR, αU, αD, and have different values based on whether the neighboring pixel is positioned at the block boundary or in the interior of the block.
0. 25. The method of claim 24, wherein the weight values KL, KR, KU, KD are expressed as follows, assuming that i and j of the pixel f(i,j) are 8, respectively,
KL={9, if j mod 8=0; 1, otherwise}
KR={9, if j mod 8=7; 1, otherwise}
KU={9, if i mod 8=0; 1, otherwise}
KD={9, if i mod 8=7; 1, otherwise}.
0. 26. An apparatus for recovering a compressed motion picture, comprising:
an image decoding unit for outputting an information with respect to an image which will be recovered, a quantized variable, a macro block type, and a motion type by decoding a coded image signal; and
a block process eliminating filter for defining a cost function based on a smoothing degree of an image and a reliability with respect to an original pixel in consideration of a directional characteristic between a neighboring pixel and the pixel which will be processed based on the pixels which will be recovered using an information with respect to the image which will be recovered inputted from the image decoding unit, and adaptively searching a regularization parameter variable which has a weight of a reliability with respect to the original image from each cost function and a weight of a smoothing degree of the original image for thereby recovering an original pixel,
wherein said regularization parameter variable is a weight value with respect to reliability and is determined based on a difference between the original pixel and the compressed pixel and a difference value between the original pixel and the neighboring pixel.
0. 28. The method of claim 27, wherein the selecting step selects a filtering method based on whether the pixel being filtered is in an intra-coded macro block.
0. 29. The method of claim 27, wherein the neighboring pixel is a pixel adjacent to the pixel being filtered.
0. 30. The method of claim 27, further comprising:
determining the difference value.
0. 31. The method of claim 30, wherein the determining step determines the difference between the pixel being filtered and the neighboring pixel as the difference value.
0. 32. The method of claim 27, wherein the selected filtering method filters the pixel based on a quantization parameter used in processing a portion of an image including the pixel.
0. 33. The method of claim 32, wherein the portion of an image including the pixel is a macroblock.
0. 34. The method of claim 27, wherein the selected filtering method includes determining at least one boundary value based on a quantization parameter of a portion of the image including the pixel.
0. 35. The method of claim 34, wherein the portion of the image including the pixel is a macroblock.
0. 36. The method of claim 34, wherein the selected filtering method includes determining more than one boundary value based on the quantization parameter.
0. 37. The method of claim 36, wherein the portion of the image including the pixel is a macroblock.
0. 38. The method of claim 27, wherein the selected filtering method filters the pixel based on a quantization parameter used in processing a portion of an image including the pixel.
0. 39. The method of claim 38, wherein the portion of an image including the pixel is a macroblock.
0. 40. The method of claim 27, wherein the selected filtering method includes determining at least one boundary value based on a quantization parameter of a portion of the image including the pixel.
0. 41. The method of claim 40, wherein the portion of the image including the pixel is a macroblock.
0. 42. The method of claim 40, wherein the filtering methodology includes determining more than one boundary value based on the quantization parameter.
0. 43. The method of claim 42, wherein the portion of the image including the pixel is a macroblock.
0. 45. The method of claim 44, wherein the selecting step selects the filtering method from filtering methods having different filtering strengths based on the result of the checking step.
0. 46. The method of claim 44, wherein the intra-coded portion of the image is an intra-coded macro block.
0. 47. The method of claim 27, further comprising:
outputting a filtered image including the filtered pixel.
0. 48. The method of claim 44, further comprising:
outputting a filtered image including the filtered pixel.

an embodiment of PREFERRED
where, g, f, and n have a size of MM×1 rearranged in a scanning sequence, and n represents a quantizing difference.

In order to process the original image f by the unit of pixels, the original pixels f(i,j) having a certain position information(i,j) is adapted. The recovered pixel g(i,j) may be expressed using the original pixel(i,j) and a quantizing difference n(i,j) with respect to the original pixel(i,j).
g(i,j)=f(i,j)+n(i,j)  (2)

As seen Equation 2, a smoothing
where MHL represents a cost function having a relationship between the pixel f(i,j) and the left side neighboring pixel f(i,j−1), MHR(f(i,j)) represents a cost function having a relationship between the pixel f(i,j) and the right side neighboring pixel f(i,j+1), MVT(f(i,j)) represents a cost function having a relationship between the pixel f(i,j) and the upper side neighboring pixel f(i−1,j), MVD(f(i,j)) represents a cost function having a relationship between the pixel f(i,j) and the lower side neighboring pixel f(i+1,j), and MT(f(i,j)) represents a cost function having a relationship of the time region.

The cost function having a smoothing degree and reliability may be expressed as the following equation 4.
MHL(f(i,j))=[f(i,j)−f(i,j−1)]2HL[g(i,j)−f(i,j)]2
MHR(f(i,j))=[f(i,j)−f(i,j+1)]2HR[g(i,j)−f(i,j)]2
MVT(f(i,j))=[f(i,j)−f(i−1,j)]2VT[g(i,j)−f(i,j)]2
MVD(f(i,j))=[f(i,j)−f(i+1,j)]2VD[g(i,j)−f(i,j)]2
MT(f(i,j))=[f(i,j)−fMC(i,j)]2T[g(i,j)−f(i,j)]2  (4)

As seen in Equation 4, the first term of the right side of each cost function represents a smoothing degree with respect to the original pixel and the neighboring pixel, and the second term of the right side represents a reliability with respect to the original pixel and the recovered pixel.

The first term of the right side of the cost function MHL(f(i,j)) represents a square value of the difference between the original pixel f(i,j) and the left side neighboring pixel f(i,j−1) and represents a uniformity degree, namely, a smoothed degree of the original pixel f(i,j) and the left side neighboring pixel f(i,j−1) based on the error component between the original pixel f(i,j) and the left side neighboring pixel f(i,j−1). In addition, the second term of the right side of the cost function MHL(f(i,j)) represents a square value of the difference between the original pixel f(i,j) and the compressed pixel g(i,j) and represents a value for comparing whether a certain difference exists between the compressed pixel g(i,j) and the original pixel f(i,j) based on a difference component between the original pixel f(i,j) and the compressed pixel g(i,j) and represents a reliability of the original pixel f(i,j) and the compressed pixel g(i,j).

In addition, the first term of the right side of MHR(f(i,j)) represents a smoothing degree of the original pixel f(i,j) and the right side neighboring pixel f(i,j+1), and the second term of the right side represents a reliability of the original pixel f(i,j) and the compressed pixel g(i,j). The first term of the right side of the cost function MVT(f(i,j)) represents a smoothing degree of the original pixel f(i,j) and the upper side neighboring pixel f(i−1,j), and the second term of the right side represents a reliability of the original pixel, and the compressed pixel g(i,j). The first term of the right side of the cost function MVT(f(i,j)) represents a smoothing degree of the original pixel f(i,j) and the lower side neighboring pixel f(i+1,j), and the second term of the right side represents a reliability of the original pixel f(i,j) and the compressed pixel g(i,j). MT(f(i,j)) represents a cost function for setting a relationship of the time region.

The values of αHL, αHR, αVT, αVD αT of the second term of the right side represents a regularization parameter and a ratio of a smoothing degree and reliability. These values represent a difference component. In addition, these values represent a weight value with respect to the reliability. As these values are increased, the reliability is enhanced. Since the smoothing degree and the reliability are opposite to each other, the ratio of the smoothing degree and reliability is determined when the regularization parameter is determined. Each regularization parameter may be expressed as the following Equation 5.

α HL = [ f ( i , j ) - f ( i , j - 1 ) ] 2 [ g ( i , j ) - f ( i , j ) ] 2 , α HR = [ f ( i , j ) - f ( i , j + 1 ) ] 2 [ g ( i , j ) - f ( i , j ) ] 2 ( 5 ) α VT = [ f ( i , j ) - f ( i - 1 , j ) ] 2 [ g ( i , j ) - f ( i , j ) ] 2 , α VD = [ f ( i , j ) - f ( i + 1 , j ) ] 2 [ g ( i , j ) - f ( i , j ) ] 2 α T = [ f ( i , j ) - f MC ( i , j ) ] 2 [ g ( i , j ) - f ( i , j ) ] 2

In the above Equation 5, the denominators of the above-equations represents a difference between the original pixel and the compressed pixel, and the numerator represents a difference between the original pixel and the neighboring pixel.

Computation of Recovering Pixels Based on Cost Function

It is needed to obtain the recovering pixels which is the original pixels. However, the cost function includes a square with respect to the original pixel. Therefore, the cost function is partially differentiated with respect to the original pixel, so that it is possible to obtain the original pixels based on the differentiated values. The cost function M(f(i,j)) may be differentiated based on Equation 3.

M ( f ( i , j ) ) f ( i , j ) = M HL ( f ( i , j ) ) f ( i . j ) + M HR ( f ( i , j ) ) f ( i , j ) + M VT ( f ( i , j ) ) f ( i , j ) + M VD ( f ( i , j ) ) f ( i , j ) + M T ( f ( i , j ) ) f ( i , j ) = 0 ( 6 )

Each term of the right side of the cost function with respect to the neighboring pixels is as follows.

M HL ( f ( i , j ) ) f ( i , j ) = 2 [ f ( i , j ) - f ( i , j - 1 ) ] - 2 α HL [ g ( i , j ) - f ( i , j ) ] ( 7 ) M HR ( f ( i , j ) ) f ( i , j ) = 2 [ f ( i , j ) - f ( i , j + 1 ) ] - 2 α HR [ g ( i , j ) - f ( i , j ) ] M VT ( f ( i , j ) ) f ( i , j ) = 2 [ f ( i , j ) - f ( i - 1 , j ) ] - 2 α VT [ g ( i , j ) - f ( i , j ) ] M VD ( f ( i , j ) ) f ( i , j ) = 2 [ f ( i , j ) - f ( i + 1 , j ) ] - 2 α VD [ g ( i , j ) - f ( i , j ) ] M T ( f ( i , j ) ) f ( i , j ) = 2 [ f ( i , j ) - f MC ( i , j ) ] - 2 α T [ g ( i , j ) - f ( i , j ) ]

The values of Equation 7 are substituted for Equation 6, and the pixels which will be finally recovered are in the following Equation 8.

f ( i , j ) = f ( i , j - 1 ) + f ( i , j + 1 ) + f ( i - 1 , j ) + f ( i + 1 , j ) + f MC ( i , j ) + α TOT g ( i , j ) 5 + α TOT ( 8 ) α TOT = α HL + α HR + α VT + α VD + α T

The pixels expressed by Equation 8 are the pixels included in the inter macro block. However, the pixels of the macro block coded into the intra macro type based on Equation 6 is

M T ( f ( i , j ) ) f ( i , j ) = 0
because there is not a motion information on tile time axis. Therefore, the pixels included in the intra macro block may be expressed in the following Equation 9.

f ( i , j ) = f ( i , j - 1 ) + f ( i , j + 1 ) + f ( i - 1 , j ) + f ( i + 1 , j ) + α TOT g ( i , j ) 4 + α TOT ( 9 )
αTOTHLHRVTVD

Therefore, the pixels included in the inter macro block are obtained based on

    • where B represents a DCT process, and Q represents a quantizing process.

The DCT coefficient of the original image and the DCT coefficient of the compressed image have the following interrelationship as seen in Equation 12.
G(u,v)−Qpl≦F(u,v)≦G(u,v)+Qpl  (12)

    • where G(u,v) represents a (u,u)-th value of the two-dimensional DCT coefficient of the compressed image, F(u,v) represents a (u,v)-th value of the two-dimensional DCT coefficient of the original image, Qpl represents the quantizing maximum difference of the l-th macro block, and each DCT coefficient value represents a subset for setting the range of the DCT coefficient of the recovered images. Therefore, the recovered images must be projected based on the subset of Equation 12, and this process is performed in Step ST6 as seen in the following Equation 13.
      P(F(u,v))=G(u,v)−Qpl if F(u,v)<G(u,v)−Qpl
      P(F(u,v))=G(u,v)+Qpl if F(u,v)>G(u,v)+Qpl
      P(F(u,v))=F(u,v) otherwise  (13)

The Equation 13 will be explained in detail.

If F(u,v) is smaller than G(u,v)−Qpl, the projected recovering image P(F(u,v) is mapped based on G(u,v)−Qpl, and if F(u,v) is larger than G(u,v)−Qpl, the projected recovering image P(F(u,v)) is mapped based on G(u,v)+Qpl, otherwise P(F(u,v)) is directly mapped based on the projected recovering image F(u,v).

The mapped image P(F(u,v)) is reversely DCT-processed in the spacious region in Step ST7, and the finally recovered image may be expressed by the following Equation 14.
f′=BTPBf=BTPBK(g)  (14)

    • where K(g) represents a computation of the recovering pixels of Equation 8 or 9, BK(g) represents a block DCT coefficient, PBK(g) represents a projected block DCT coefficient, and BTPBK(g) represents that the projected block DCT coefficient is recovered in the spacious region. The recovered image is stored in the image memory and is outputted.

In the present invention, it is possible to eliminate a block artifact and ring effect based on an non-uniform degree and reliability of the recovered image using a plurality of information from the decoder.

Repetition Technique

If the block artifact and ring effect are not fully eliminated from the recovered pixels,

Namely, the block artifacts and ring effects are eliminated from the recovered images by an adaptive decoding operation, so that a real time process is implemented in the digital video apparatus. In particular, it is possible to enhance the resolution in the compression images which require a low bit

    • where ML represents a cost function having an interrelationship between the pixel f(i,j) and the left side neighboring pixel f(i,j−1), MR(f(i,j)) represents a cost function having an interrelationship between the pixel f(i,j) and the right side neighboring pixel f(i,j+1), MU(f(i,j)) represents a cost function having an interrelationship between the pixel f(i,j) and the upper side neighboring pixel f(i−1,j), and MD(f(i,j)) is a cost function having an interrelationship between the pixel f(i,j) and the lower side neighboring pixel f(i+1,j).

Next, the cost functions including a smoothing degree and reliability are defined. The regularization parameter variable is included in only the portion (the second term of the right side in Equation 4) of the reliability with respect to the original pixel and recovered pixel. Differently from this construction, in another embodiment of the present invention, the regularization parameter variable is included in the portion which represents a reliability of the original pixel and recovered pixel as well as is included in the portion which represents the smoothing degree with respect to the original pixel and the neighboring pixel. In addition, the smoothing degree and the reliability of the pixel are opposite each other
MR(f(i,j))=αR(f(i,j))[f(i,j)−f(i,j+1)]2+(1−αR(f(i,j)))[g(i,j)−f(i,j)]2


MD(f(i,j))=αD(f(i,j))[f(i,j)−f(i +1,j)]2+(1−αD(f(i,j)))[g(i,j)−f(i,j)]2  (19)

As seen in Equation 19, the first term of the right side represents a smoothing degree with respect to the original pixel and the neighboring pixel, and the second term of the right side represents a reliability with respect to the original pixel and the recovered pixel. Here, αL, αR, αU, αD represent a regularization parameter variable with respect to each cost function and represent a ratio of a smoothing degree and reliability as a difference component. For example, αL represents a weight value with respect to the smoothing degree, and 1−αL represents a weight value with respect to the reliability. Therefore, as the regularization parameter variable is increased, the smoothing degree is increased, and the reliability is decreased. Since the regularization includes the right side first term and the left side term of the cost function, it is possible to implement more stable smoothing

Next, as seen in Equation 22, the recovering pixel includes a regularization parameter variable α, and each regularization parameter variable is obtained as follows.

The regularization parameter variable is obtained based on Equation 19. Namely, since the smoothing degree and reliability are opposite to each other, the regularization parameter variable may be arranged according to Equation 24 as follows based on a ratio of the smoothing degree and the reliability. Equation 24 may be expressed as follows.

1 - α L α L = [ f ( i , j ) - f ( i , j - 1 ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2 ( 24 ) 1 - α R α R = [ f ( i , j ) - f ( i , j + 1 ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2 1 - α U α U = [ f ( i , j ) - f ( i - 1 , j ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2 1 - α D α D = [ f ( i , j ) - f ( i + 1 , j ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2

In order to obtain the regularization parameter variable expressed as Equation 24, the pixels f(i,j), f(ij−1), f(i,j+1), f(i−1,j), f(i+1,j) must be approximated based on the compressed pixels g(i,j), g(i,j−1), g(i,j+1), g(i−1,j), g(i+1,j) which may be actually used. For implementing the above-described operation, the following three cases are assumed.

First, a quantization difference of each pixel is a function of a quantization variable Qp which is set by the unit of macro blocks.

Second, since the block artifacts generating at a block boundary has a certain non-uniformity degree which is larger than the ring effect occurring in the interior of the block, the difference with respect to the pixels positioned at the block boundary is more largely reflected compared to the pixels positioned in the interior of the block. Namely, a weight value is provided to the difference based on the position of the pixels.

Equation 24 is approximated to Equation 25 based on the above-described two assumptions.

1 - α L α L = [ f ( i , j ) - f ( i , j - 1 ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2 [ g ( i , j ) - g ( i , j - 1 ) ] 2 Φ ( Q p ) ( 25 ) 1 - α R α R = [ f ( i , j ) - f ( i , j + 1 ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2 [ g ( i , j ) - g ( i , j + 1 ) ] 2 Φ ( Q p ) 1 - α U α U = [ f ( i , j ) - f ( i - 1 , j ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2 [ g ( i , j ) - g ( i - 1 , j ) ] 2 Φ ( Q p ) 1 - α D α D = [ f ( i , j ) - f ( i + 1 , j ) ] 2 [ f ( i , j ) - g ( i , j ) ] 2 [ g ( i , j ) - g ( i + 1 , j ) ] 2 Φ ( Q p )

    • where Φ(Qp) is a function of the quantizing variable Qp and is different based on the position of a pixel.
      Therefore, with consideration of the position of each pixel in the function Φ(Qp), Φ(Qp) may be expressed as KLQP2 with respect to αL, and Φ(Qp) is expressed as KRQp2 with respect to αR, and Φ(Qp) is expressed as KUQp2, with respect to αU, and Φ(Qp) is expressed as KDQp2 with respect to αD. Here, constants KL, KR, KU, KD are weight values and are different based on whether the neighboring pixel is positioned at the block boundary or in the interior of the block. With consideration to the position of each pixel, type regularization parameter variable is approximated based on the following Equation 26.

α L = K L Q P 2 [ g ( i , j ) - g ( i , j - 1 ) ] 2 + K L Q P 2 ( 26 ) α R = K R Q P 2 [ g ( i , j ) - g ( i , j + 1 ) ] 2 + K R Q P 2 α U = K U Q P 2 [ g ( i , j ) - g ( i - 1 , j ) ] 2 + K U Q P 2 α D = K D Q P 2 [ g ( i , j ) - g ( i + 1 , j ) ] 2 + K D Q P 2

Assuming that one block is formed of 8×8 number of pixels, namely, assuming that I and j of f(i,j) is 8, respectively, the weight values KL, KR, KU, KD may be expressed as follows.

    • KL={9, if j mod 8=0; 1, otherwise}
    • KR={9, if j mod 8=7; 1, otherwise}
    • KU={9, if i mod 8=0; 1, otherwise}
    • KD={9, if i mod 8=7; 1, otherwise}

For example, in the Equation related to KL, if the residual is 0 when dividing j by 8, KL is 9, and otherwise, KL is 1.

When the approximated regularization parameter values are substituted for Equation 22, it is possible to obtain a resultant value f(i,j).

FIG. 5 is a flow chart illustrating a method for recovering a compressed image for an image processing system according to another embodiment of the present invention.

In Step ST10, it is judged whether the pixels of the current macro block are the same as the pixels of the previously transmitted macro block based on the COD value. If they are same, in Step ST11, the recovering pixel values are substituted for the pixel values which are previously recovered based on Equation 23. If they are not the same, in Step ST12, the regularization parameter variables αL, αR, αU, αD are obtained based on Equation 26, and the recovering pixel f(i,j) is obtained based on Equation 22 in Step ST13.

As described above, in the present invention, a certain weight is provided to the regularization parameter variable, which will be approximated, based on the position of the pixels in consideration with the reliability and smoothing degree as well as the regularization parameter variables, so that it is possible to obtain a value which is near the actual pixel value. Therefore, in the present invention, it is not needed to perform a projection method and a repetition method. In addition, in the present invention, the computation amount and time are significantly decreased.

The invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims invention.

Hong, Min-Cheol

Patent Priority Assignee Title
Patent Priority Assignee Title
5283646, Apr 09 1992 Polycom, Inc Quantizer control method and apparatus
5488570, Nov 24 1993 Intel Corporation Encoding and decoding video signals using adaptive filter switching criteria
5563813, Jun 01 1994 Industrial Technology Research Institute Area/time-efficient motion estimation micro core
5611000, Feb 22 1994 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Spline-based image registration
5748795, Feb 07 1995 Canon Kabushiki Kaisha Image decoder using adjustable filtering
5790131, May 15 1996 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P System and method for lossy compression of data with output file size control
5878166, Dec 26 1995 AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD Field frame macroblock encoding decision
5940536, Sep 05 1995 Matsushita Electric Industrial Co., Ltd. Ringing detector and filter
6041145, Nov 02 1995 MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD Device and method for smoothing picture signal, device and method for encoding picture and device and method for decoding picture
6058210, Sep 15 1997 Xerox Corporation Using encoding cost data for segmentation of compressed image sequences
6108455, May 29 1998 STMicroelectronics, Inc. Non-linear image filter for filtering noise
6167164, Mar 10 1997 SAMSUNG ELECTRONICS CO , LTD One-dimensional signal adaptive filter for reducing blocking effect and filtering method
6178205, Dec 12 1997 Cisco Technology, Inc Video postfiltering with motion-compensated temporal filtering and/or spatial-adaptive filtering
6195632, Nov 25 1998 Panasonic Intellectual Property Corporation of America Extracting formant-based source-filter data for coding and synthesis employing cost function and inverse filtering
6226050, Apr 04 1997 SAMSUNG ELECTRONICS CO , LTD Signal adaptive filtering method for reducing ringing noise and signal adaptive filter
6259823, Feb 15 1997 SAMSUNG ELECTRONICS CO,, LTD Signal adaptive filtering method and signal adaptive filter for reducing blocking effect and ringing noise
6385245, Sep 23 1997 NXP B V Motion estimation and motion-compensated interpolition
6529638, Feb 01 1999 RAKUTEN, INC Block boundary artifact reduction for block-based image compression
6594400, Aug 31 1998 LG Electronics Inc. Method of removing block phenomenon and ring phenomenon in compression coded image
6631162, Jul 16 1997 SAMSUNG ELECTRONICS CO , LTD Signal adaptive filtering method, signal adaptive filter and computer readable medium for storing program therefor
6665346, Aug 01 1998 SAMSUNG ELECTRONICS CO , LTD ; Korea Advanced Institute of Science and Technology Loop-filtering method for image data and apparatus therefor
7272186, Jun 15 2001 LG Electronics, Inc. Loop filtering method in video coder
20050147319,
20050201633,
WO9904497,
/
Executed onAssignorAssigneeConveyanceFrameReelDoc
Jan 25 2006LG Electronics Inc.(assignment on the face of the patent)
Date Maintenance Fee Events
Nov 22 2011ASPN: Payor Number Assigned.


Date Maintenance Schedule
Sep 06 20144 years fee payment window open
Mar 06 20156 months grace period start (w surcharge)
Sep 06 2015patent expiry (for year 4)
Sep 06 20172 years to revive unintentionally abandoned end. (for year 4)
Sep 06 20188 years fee payment window open
Mar 06 20196 months grace period start (w surcharge)
Sep 06 2019patent expiry (for year 8)
Sep 06 20212 years to revive unintentionally abandoned end. (for year 8)
Sep 06 202212 years fee payment window open
Mar 06 20236 months grace period start (w surcharge)
Sep 06 2023patent expiry (for year 12)
Sep 06 20252 years to revive unintentionally abandoned end. (for year 12)