Disclosed are methods, apparatuses, and systems for encoding and decoding an image. The present invention provides an intra prediction unit that receives an input image, calculates a correlation between a luma area block and chroma area block in the input image in intra prediction, removes high frequency ingredients by low-pass filtering an encoded luma pixel based on the calculated correlation, and generates a prediction block by applying an lm chroma mode, which is an extended chroma mode technique, to the luma pixel removed therefrom the high frequency ingredients.
|
6. An image decoding apparatus, comprising:
an intra predictor that receives a bitstream, analyzes a flag included in the bitstream, removes high frequency ingredients by selectively low pass filtering luma pixel based on a set value of a correlation flag included in the analyzed flag, and predicts a chroma pixel by applying an lm chroma mode,
wherein the lm chroma mode which is an extended chroma mode is applied to the luma pixel removed of the high frequency ingredients.
1. An image encoding apparatus, comprising:
an intra predictor receiving an input image, calculating a correlation between a luma area and chroma areas in intra prediction, removing high frequency ingredients by low-pass filtering the luma area including an encoded luma pixel based on the calculated correlation, and generating a prediction block by predicting a chroma pixel by applying an lm chroma mode,
wherein the lm chroma mode comprises: an extended chroma mode technique applied to the filtered encoded luma pixel.
10. An image encoding method employed by an image encoding apparatus that receives an input image, encodes the input image based on intra prediction, and outputs a bitstream, the method comprising:
calculating a correlation between a luma area block and a chroma area block in the input image during intra prediction;
if the correlation is smaller than a pre-set threshold value, removing, by the image encoding apparatus, high frequency ingredients by low-pass filtering an encoded luma pixel;
if the correlation is smaller than the pre-set threshold value, generating a prediction block by applying an lm chroma mode, which is an extended chroma mode technique, to the low-pass filtered luma pixel or, if the correlation is equal to or greater than the pre-set threshold value, generating a prediction block by applying the lm chroma mode to the luma pixel that is not low-pass filtered; and,
generating a bitstream of a rgb image exhibiting high correlations between color planes by the prediction block and outputting the bitstream.
13. An image decoding method employed by an image encoding apparatus that receives a bitstream, encodes the bitstream based on intra prediction, and outputs a reconstructed image, the method comprising:
receiving, by the image decoding apparatus, the bitstream and obtaining a luma area block;
obtaining and analyzing a flag included in the bitstream and selecting a mode for performing intra prediction from a plurality of pre-set modes;
if a mode selected based on the analyzed flag is an lm chroma mode, analyzing the setting of a correlation flag included in the flag;
if the correlation flag is set, removing high frequency ingredients of the luma pixel by low-pass filtering the luma pixel;
if the correlation flag is set, generating a prediction block by applying the lm chroma mode, which is an extended chroma mode technique, to the low-pass filtered luma pixel and, if no correlation flag is set, generating a prediction block by applying the lm chroma mode to the luma pixel that is not low-pass filtered; and
outputting the reconstructed image by the prediction block.
15. An imaging system comprising:
an image encoding apparatus that receives an input image, calculates a correlation between a luma area block and a chroma area block in the input image during intra prediction, removes high frequency ingredients by low-pass filtering the encoded luma pixel based on the calculated correlation, generates a prediction block by applying an lm chroma mode, which is an extended chroma mode technique, to the low-pass filtered luma pixel removed of the high frequency ingredients, and generates and outputs a bitstream of a rgb image exhibiting correlations between color planes by the prediction block; and
an image decoding apparatus that receives the bitstream and obtaining a luma area block, analyzes a flag included in the bitstream, removes high frequency ingredients of the luma pixel by selectively low-pass filtering the luma pixel based on the setting of a correlation flag of the analyzed flag, generates a prediction block by applying the lm chroma mode, which is an extended chroma mode technique, to the luma pixel removed therefrom the high frequency ingredients, and outputs reconstructed image by the prediction block.
2. The image encoding apparatus of
a chroma predictor, which comprises a plurality of pre-set mode predictors including a lm chroma mode predictor that generates the prediction block by applying the lm chroma mode with regard to the encoded luma pixel, wherein the mode predictor individually performs intra prediction to generate a plurality of prediction blocks;
a rate-distortion optimizer, which receives the plurality of prediction blocks and determines an optimal prediction block from the plurality of prediction blocks according to pre-set rules; and
a mode selector, which sets up a flag corresponding to a mode for generating the optimal prediction block from the plurality of modes and outputs the flag together with the optimal prediction block.
3. The image encoding apparatus of
a correlation calculator, which calculates a correlation between a luma area block and a chroma area block and determines whether the calculated correlation is smaller than a pre-set threshold value;
a low pass filter, which, if the calculated correlation is smaller than the pre-set threshold value, low-pass filters luma pixel value included in the luma area block; and
an extended chroma predictor, which, if the calculated correlation is equal to or greater than the pre-set threshold value, generates the prediction block by applying the lm chroma mode regarding the luma pixel value applied by the correlation calculator and, if the calculated correlation is smaller than the pre-set threshold value, generates the prediction block by applying the lm chroma mode to the luma pixel removed of the high frequency ingredients of the encoded luma pixel.
4. The image encoding apparatus of
5. The image encoding apparatus of
wherein, when the flag is set, the mode selector adds a correlation flag indicating whether to perform a low-pass filtering, to the flag.
7. The image decoding apparatus of
a flag analyzer, which analyzes a flag included in the bitstream;
a mode selector, which selects a mode for performing intra prediction from a plurality of pre-set modes including the lm chroma mode based on the analyzed flag; and
a chroma predictor, which comprises a plurality of pre-set mode predictors including an lm chroma mode predictor that generates a prediction block including predicted chroma pixel by applying the lm chroma mode to the luma pixel, wherein the mode prdictor individually performs intra prediction to generate a plurality of prediction blocks.
8. The image decoding apparatus of
a correlation flag determiner, which determines whether a correlation flag is set to the flag analyzed by the flag analyzer;
a low pass filter, which, if the correlation flag is set, low-pass filters the luma pixel; and
an extended chroma predictor, which, if no correlation flag is set, generates the prediction block by applying the lm chroma mode to the luma pixel applied by the correlation calculator and, if the correlation flag is set, generates the prediction block by applying the lm chroma mode to the luma pixel low-pass filtered by the low pass filter.
9. The image decoding apparatus of
11. The image encoding method of
performing intra prediction with respect to the luma pixel in a plurality of pre-set modes including the lm chroma mode and generating a plurality of prediction blocks;
determining an optimal prediction block from the plurality of prediction blocks according to pre-set rules; and
setting up a flag corresponding to a mode for generating the optimal prediction block from the plurality of modes and outputting the flag with the optimal prediction block.
12. A non-transitory computer readable recording medium having recorded thereon a computer program for implementing the image encoding method of
14. A non-transitory computer readable recording medium having recorded thereon a computer program for implementing the image decoding method of
16. The imaging system of
a chroma predictor, which comprises a plurality of pre-set mode predictors including an lm chroma mode predictor that generates the prediction block by applying the lm chroma mode to the encoded luma pixel, wherein the mode predictor individually performs intra prediction to generate a plurality of prediction blocks;
a rate-distortion optimizer, which receives the plurality of prediction blocks and determines an optimal prediction block from the plurality of prediction blocks according to pre-set rules; and
a mode selector, which sets up a flag corresponding to a mode for generating the optimal prediction block from the plurality of modes and outputs the flag with the optimal prediction block.
17. The imaging system of
a correlation calculator, which calculates a correlation between the luma area block and the chroma area block and determines whether the calculated correlation is smaller than a pre-set threshold value;
a low pass filter, which, if the calculated correlation is smaller than the pre-set threshold value, low-pass filters luma pixel; and
an extended chroma predictor, which, if the calculated correlation is equal to or greater than the pre-set threshold value, generates the prediction block by applying the lm chroma mode to the luma pixel applied by the correlation calculator and, if the calculated correlation is smaller than the pre-set threshold value, generates the prediction block by applying the lm chroma mode to the luma pixel low-pass filtered by the low pass filter.
18. A non-transitory computer readable recording medium having recorded thereon a computer program for implementing the image encoding method of
|
This application is the U.S. national stage application of International Patent Application No. PCT/KR2014/009222, filed Sep. 30, 2014, which claims priority to Korean Application No. 10-2013-0149075, filed Dec. 3, 2013, the disclosures of each of which are incorporated herein by reference in their entirety.
The present disclosure relates to methods of encoding and decoding image, apparatus and system therefor, and more particularly, to methods, apparatus, and a system for encoding and decoding an image for improving efficiencies of predictions between color planes of a RGB image in high efficiency video coding (HEVC).
Following the standardized high efficiency video coding (HEVC), a HEVC range extension is currently being standardized for developing an extended technique for supporting 4:2:2 and 4:4:4 color samplings and YUV images and RGB images corresponding to 10 bit, 12 bit, and 14 bit depths. Here, unlike an YUV image, a RGB image exhibits high correlations between respective color planes R, G, and B. Based on the same, an extended chroma mode (referred to hereinafter as ‘ECM’) has been suggested by J. Kim at a recent MPEG standardization conference (J. Kim, “RCE1: The performance of extended chroma mode for non 4:2:0 format, JCTVC-M0097, 13th JCT-VC Meeting, Incheon, Korea, April 2013)
The ECM suggested by J. Kim is a technique for predicting pixel values in a chroma (or B, R) area with reference to an encoded luma (or G) pixel in case of an intra prediction for chroma (U, V; B, R in the RGB coding) pixel. The algorithm employs a technique suggested by J. Chen during the standardization of the HEVC, but is not employed by the HEVC (J. Chen, V. Seregin, W-J Han, J. Kim, B. Jeon, “CE6.a.4: chroma intra prediction by reconstructed luma samples”, JCTVC-E266, 5th Meeting, Geneva, Switzerland, March 2011). However, unlike the YUV 4:2:0 format of the HEVC, the RGB 4:4:4 format exhibits high correlations between color planes. Therefore, if the technique suggested by J. Chen is modified in correspondence to the RGB 4:4:4 format instead of the YUV format and applied to the RGB 4:4:4 format, compression efficiency may be significantly improved. In other words, a high coding gain may be obtained. Therefore, the technique may be applied to the HEVC range extension.
However, as a result of analysis of a large number of RGB images, a common RGB image exhibits low correlations between color planes at high frequency areas of the color planes. In other words, due to low correlations between color planes at high frequency areas, high frequency ingredients for luma area may interfere chroma pixel prediction during an application of the ECM, thereby deteriorating compression efficiency.
TABLE 1
G/B
G/R
Test Sequence
LL
LH
HL
HH
LL
LH
HL
HH
Traffic
0.926
0.634
0.614
0.549
0.963
0.837
0.810
0.777
Kimono1
0.948
−0.105
0.136
−0.008
0.977
−0.011
0.303
0.045
EBULupoCandlelight
0.820
0.859
0.869
0.899
0.904
0.944
0.929
0.940
EBURainFruits
0.909
0.972
0.970
0.955
0.892
0.971
0.969
0.956
VenueVu
0.720
0.885
0.853
0.553
0.597
0.840
0.604
0.229
DucksAndLegs
0.906
0.227
−0.090
−0.154
0.963
0.238
0.125
−0.186
OldTownCross
0.981
0.456
0.265
0.025
0.984
0.617
0.466
0.243
ParkScene
0.919
0.247
0.443
0.117
0.972
0.406
0.462
0.126
Overall
0.891
0.522
0.508
0.367
0.907
0.605
0.584
0.391
Table 1 shows results of calculating correlations between four frequency bands between color planes (G-B and G-R) by using first frames of 8 RGB experimental images currently used in the HEVC range extension and shows correlations between color planes corresponding to respective frequency bands in HEVC range extension RGB experimental images. Table 1 is published in “Color plane interpolation using alternating projections,” (B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau), IEEE Trans. Video Processing, Vol. 11, No. 9, pp. 997-1013, September 2002. The technique disclosed therein can analyze frequencies of a RGB image based on mathematical analysis of correlations between color planes R, G, and B, where each correlation has a range from −1 to 1. The higher the correlation indicates the corresponding color planes having the higher correlation to each other.
In Table 1, the LL frequency band indicates low frequency ingredients of an image, the LH and HL frequency bands indicate horizontal and vertical edge ingredients of the image, and the HH frequency band indicates diagonal edge ingredients of the image, that is, high frequency ingredients. Except an EBULupoCandlelight image and an EBURainFruits image, images generally exhibit high correlations in low frequency areas LL and low correlations in high frequency areas HH. Referring to Table 1, RGB images exhibit high correlations (0.90 average) in LL frequency bands and low correlations (0.38 average) in HH frequency bands. As shown in Table 1, high frequency ingredients having low correlations compared to the other frequency ingredients having generally high correlations may deteriorate compression efficiency when pixel values between color planes are predicted by using the ECM.
The present invention provides methods of encoding and decoding an image for improving efficiency of predicting pixel values between color planes by filtering high frequency ingredients exhibiting low correlations between the color planes.
The present invention also provides apparatuses for encoding and decoding an image for accomplishing the technical goal.
The present invention also provides a system for encoding and decoding an image for accomplishing the technical goal.
According to an aspect of the present invention, there is provided an image encoding apparatus including an intra prediction unit that receives an input image, calculates a correlation between a Luma area and Chroma areas during intra prediction, removes high frequency ingredients by low-pass filtering an encoded luma pixel based on the calculated correlation, and generates a prediction block by predicting the chroma pixel by applying an LM chroma mode, which is an extended chroma mode technique to the luma pixel removed therefrom the high frequency ingredients.
The intra prediction unit includes a chroma predicting unit, which includes a plurality of pre-set mode prediction units including a LM chroma mode prediction unit that generates the prediction block by applying the LM chroma mode to the encoded luma pixel, wherein the mode prediction unit individually perform intra prediction to generate a plurality of prediction blocks; a rate-distortion optimizing unit, which receives the plurality of prediction blocks and determines an optimal prediction block from among the plurality of prediction blocks according to pre-set rules; and a mode selecting unit, which sets up a flag corresponding to a mode for generating the optimal prediction block from among the plurality of modes and outputs the flag together with the optimal prediction block.
The LM chroma mode prediction unit includes a correlation calculating unit, which calculates a correlation between the Luma area block and the Chroma area block and determines whether the calculated correlation is smaller than a pre-set threshold value; a low pass filter, which, if the calculated correlation is smaller than the pre-set threshold value, low-pass filters the luma pixel; and an extended chroma prediction unit, which, if the calculated correlation is equal to or greater than the pre-set threshold value, generates the prediction block by applying the LM chroma mode to the luma pixel applied by the correlation calculating unit and, if the calculated correlation is smaller than the pre-set threshold value, generates the prediction block by applying the LM chroma mode to the luma pixel low-pass filtered by the low pass filter.
According to another aspect of the present invention, there is provided an image decoding apparatus including an intra prediction unit that receives a bitstream, analyzes a flag included in the bitstream during an intra prediction, removes high frequency ingredients by selectively low pass filtering a luma pixel based on a set value of a correlation flag included in the analyzed flag, and generates a prediction block regarding Chroma areas by applying an LM chroma mode, which is an extended chroma mode technique to the luma pixel removed therefrom the high frequency ingredients.
The intra prediction unit includes a flag analyzing unit, which analyzes a flag set to the bitstream; a mode selecting unit, which selects a mode for performing intra prediction from among a plurality of pre-set modes including the LM chroma mode based on the analyzed flag, and a chroma predicting unit, which includes a plurality of pre-set mode prediction units including a LM chroma mode prediction unit that generates the prediction block by applying the LM chroma mode to the luma pixel, wherein the mode prediction unit individually perform intra prediction to generate a plurality of prediction blocks.
The LM chroma mode prediction unit includes a correlation flag determining unit, which determines whether a correlation flag is set to the flag analyzed by the flag analyzing unit; a low pass filter, which, if the correlation flag is set, low-pass filters the luma pixel; and an extended chroma prediction unit, which, if no correlation flag is set, generates the prediction block by applying the LM chroma mode with regard to the luma pixel applied by the correlation calculating unit and, if the correlation flag is set, generates the prediction block by applying the LM chroma mode to the luma pixel low-pass filtered by the low pass filter.
If the correlation flag is set, the chroma predicting unit low-pass filters the luma pixel by using the low pass filter, calculates the low-pass filtered the luma pixel according to Equation Greconlpf|x,y|=0.5 Grecon|x,y|+0.125 Grecon|x+1,y|+0.125 Grecon|x−1,y|+0.125 Grecon|x,y+1|+0.125 Grecon|x,y−1|, applies the LM chroma mode to the low-pass filtered luma pixel, and generates the prediction block according to an Equation Cpred [x,y]=αGreconlpf[x,y]+β (here, Greconlpf denotes a low pass filtered luma pixel, and α and β are weights inferred by using the luma pixel at same locations as pixel values around the Chroma blocks).
According to another aspect of the present invention, there is provided an image encoding method employed by an image encoding apparatus that receives an input image, encodes the input image based on intra prediction, and outputs a bitstream, the method including calculating a correlation between a Luma area block and a Chroma area block in the input image during intra prediction; if the correlation is smaller than a pre-set threshold value, removing, by the image encoding apparatus, high frequency ingredients by low-pass filtering the encoded luma pixel; if the correlation is smaller than the pre-set threshold value, generating a prediction block by applying an LM chroma mode, which is an extended chroma mode technique, with regard to the low-pass filtered luma pixel or, if the correlation is equal to or greater than the pre-set threshold value, generating a prediction block by applying the LM chroma mode to the luma pixel that are not low-pass filtered; and, generating a bitstream of a RGB image exhibiting high correlations between color planes by using the prediction block and outputting the bitstream.
According to another aspect of the present invention, there is provided an image decoding method employed by an image encoding apparatus that receives a bitstream, encodes the bitstream based on intra prediction, and outputs a restored image, the method including receiving, by the image decoding apparatus, the bitstream and obtaining a Luma area block; obtaining and analyzing a flag included in the bitstream and selecting a mode for performing intra prediction from among a plurality of pre-set modes; if a mode selected based on the analyzed flag is a LM chroma mode, analyzing the setting of a correlation flag included in the flag; if the correlation flag is set, removing high frequency ingredients of the luma pixel by low-pass filtering the luma pixel; if the correlation flag is set, generating a prediction block by applying the LM chroma mode, which is an extended chroma mode technique, with regard to the low-pass filtered luma pixel and, if no correlation flag is set, generating a prediction block by applying the LM chroma mode to the luma pixel that are not low-pass filtered; and outputting the reconstructed image by using the prediction block.
According to another aspect of the present invention, there is provided an imaging system including an image encoding apparatus that receives an input image, calculates a correlation between a Luma area block and a Chroma area block in the input image during intra prediction, removes high frequency ingredients by low-pass filtering the encoded luma pixel based on the calculated correlation, generates a prediction block by applying an LM chroma mode, which is an extended chroma mode technique, with regard to the low-pass filtered the luma pixel removed therefrom the high frequency ingredients, and generates and outputs a bitstream of a RGB image exhibiting high correlations between color planes by using the prediction block; and an image decoding apparatus that receives the bitstream and obtaining a Luma area block, analyzes a flag included in the bitstream, removes high frequency ingredients of the luma pixel by selectively low-pass filtering the luma pixel based on the setting of a correlation flag of the analyzed flag, generates a prediction block by applying the LM chroma mode, which is an extended chroma mode technique, to the luma pixel removed therefrom the high frequency ingredients, and outputs the reconstructed image by using the prediction block.
Therefore, according to methods, apparatuses, and a system for encoding and decoding an image according to the present invention, during intra prediction, a correlation between color planes is calculated in an extended chroma mode, low-pass filtering is selectively applied with regard to a Luma area block, and a prediction block is generated by performing a LM chroma mode. Therefore, encoding or decoding may be performed in the LM chroma mode after low-pass filtering is selectively performed based on a correlation, and thus coding efficiency may be improved.
The attached drawings for illustrating preferred embodiments of the present invention are referred to in order to gain a sufficient understanding of the present invention, the merits thereof, and the objectives accomplished by the implementation of the present invention.
Hereinafter, the present invention will be described in detail by explaining preferred embodiments of the invention with reference to the attached drawings. Like reference numerals in the drawings denote like elements.
Throughout the specification, when a portion “includes” an element, another element may be further included, rather than excluding the existence of the other element, unless otherwise described. As used herein, the terms “unit”, “element”, “module”, etc. should be understood as units in which at least one function or operation is performed and which may be embodied in the form of hardware, software, or a combination of hardware and software.
As described above, an extended chroma mode (ECM) is a technique for predicting pixel values in a chroma area with reference to an encoded luma area during an intra prediction for chroma pixel. By using the technique, a very high coding gain may be obtained with respect to a RGB image exhibiting high correlations between color planes. The ECM is included to an intra prediction mode for chroma pixel and is configured to be selected via a competition with conventional intra prediction modes for chroma pixel.
Since a RGB image is encoded in the order of G, B, and R, a color plane B and a color plane R may be encoded in the intra prediction mode for chroma pixel. Equation 1 below is a mathematical expression of the ECM technique regarding a RGB image.
Cpred[x,y]=αGrecon[x,y]+β [Equation 1]
Here, Cpred denotes a pixel value of a B block or a R block to be predicted, Grecon denotes a decoded pixel value of an encoding-completed G block, and [x, y] denote locations of pixels. α and β are weights and are inferred by using pixel values of the G block at same locations as pixel values around the B block or the R block.
As shown in Equation 1, the ECM is a technique for predicting chroma pixel values by using pixel values of an encoded luma pixel. However, the ECM technique may be useful only when correlations between color planes are high.
Referring to
Therefore, according to the present invention, when pixel values are predicted by using the ECM to improve accuracy of a prediction between color planes, efficiency of the prediction between the color planes may be improved by removing high frequency ingredients of a luma (G) area having a low correlation in advance. Furthermore, to handle changes of correlations of an intra high frequency area, the present invention suggests a prediction unit (PU) adaptive filtering technique.
Referring to
The image encoding apparatus 100 encodes an input image in an intra mode or an inter mode and outputs a bitstream. In the below embodiments of the present invention, an intra prediction may be used as a synonym of an in-screen prediction, whereas an inter prediction may be used as a synonym of an inter-screen prediction. To determine the optimal prediction method regarding a prediction unit, an intra prediction mode and an inter prediction mode may be selectively applied with respect to the prediction unit. The image encoding apparatus 100 may generate a prediction block regarding an original block of an input image and encodes a difference between the original block and the prediction block.
In an intra prediction mode, the intra predicting unit 120 (an intra prediction unit may be used as a synonym) supports a chroma mode for predicting chroma pixel values (B, R) with reference to pixel values of an encoded Luma (G) pixel during intra predictions of chroma (B, R) screen images. In particular, the intra predicting unit 120 according to the present invention provides not only chroma modes provided by a conventional intra prediction unit, which include a planar mode, a vertical mode, a horizontal mode, and a DC mode, but also an LMchroma mode, which is an ECM suggested by J. Kim. However, the intra predicting unit 120 according to the present invention may calculate a correlation between chroma (B, R) pixels adjacent to a Luma (G) pixel when a prediction block is generated, and remove high frequency region of Luma (G) pixel by using low pass filter (LPF) based on the calculated correlation, and then apply LMchroma mode to the Luma (G) pixel. In other words, the intra predicting unit 120 may perform low pass filtering to Luma (G) pixel based on a correlation between the Chroma areas (B, R) and the Luma (G) area, which are adjacent to each other, before applying the LMChroma mode to Luma (G) pixel.
Furthermore, the intra predicting unit 120 generates prediction blocks in all available modes, selects the most efficient mode among the all available modes, and outputs a prediction block generated in the selected mode. Detailed descriptions of the intra predicting unit 120 will be given below.
In an inter prediction mode, the motion predicting unit 111 searches for an area of a reference image stored in the reference image buffer 190, which is the best-matched to an input block, and calculates a motion vector during a motion prediction. The motion compensating unit 112 generates a prediction block by performing motion compensation by using the motion vector.
The subtractor 125 generates a residual block based on a difference between the input block and the generated prediction block. The transforming unit 130 performs transformation with respect to the residual block and outputs a transformation coefficient. Furthermore, the quantizing unit 140 quantizes the input transformation coefficient according to quantization parameters and outputs a quantized coefficient. The entropy encoding unit 150 entropy-encodes the input quantized coefficient based on probability distribution and outputs a bitstream.
Since inter prediction encoding, that is, inter-screen prediction encoding is performed in the HEVC, it is necessary to decode and store a currently encoded image to be used as a reference image. Therefore, a quantized coefficient is inverse quantized by the inverse quantizing unit 160 and is inverse transformed by the inverse transforming unit 170. The inverse-quantized and inverse-transformed coefficient is added to a prediction block by the adder 175, and thus a restored block is generated.
The reconstructed block is filtered by the filter unit 180, where the filter unit 180 may apply at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or a reconstructed picture. The filter unit 180 may also be referred to as an adaptive in-loop filter. The deblocking filter may remove block distortions formed at boundaries between blocks. The SAO may add a suitable offset value to a pixel value for compensating a coding error. The ALF may be performed based on values obtained by comparing a reconstructed image to an original image and may be performed only when high efficiency is required. The reconstructed block filtered by the filter unit 180 is stored in the reference image buffer 190.
In the HEVC, in addition to encoding of macro block in the related art, three new units, which are a coding unit (CU), a prediction unit (PU), and a transformation unit (TU), are additionally defined. From among the new units, when a block-by-block splitting operation is performed and coding units are determined, the prediction unit is determined with respect to all coding units that are no longer split. According to the present invention, when the intra predicting unit 120 predicts a chroma pixel value (B, R) via prediction between color planes during prediction of pixel values by using prediction units, high frequency ingredients of a luma (G) pixel that is arranged at a same location as chroma (B, R) pixels to be encoded are removed by using a low pass filter (LPF). Therefore, the present invention may provide a filtering technique adaptive to changes of correlations of a luma (G) pixel from which high frequency regions are removed.
Although the intra predicting unit 120 of
In
Like a conventional intra prediction unit, the chroma predicting unit 121 includes a planar mode prediction unit PM, a vertical mode prediction unit VM, a horizontal mode prediction unit HM, and a DC mode prediction unit DC mode. The planar mode prediction unit PM, the vertical mode prediction unit VM, the horizontal mode prediction unit HM, and the DC mode prediction unit DC mode perform intra predictions by using intra prediction modes known in the art, that is, a planar mode, a vertical mode, a horizontal mode, and a DC mode. Since those predictions are known in the art, detailed descriptions thereof will be omitted.
Furthermore, the chroma predicting unit 121 according to the present invention further includes a LMchroma mode prediction unit LMCM for the ECM.
When the chroma predicting unit 121 performs chroma predictions in all modes set up by using the planar mode prediction unit PM, the vertical mode prediction unit VM, the horizontal mode prediction unit HM, the DC mode prediction unit DC mode, and the LMchroma mode prediction unit LMCM, the rate-distortion optimizing unit 122 receives pixel values of prediction blocks chroma-predicted in a plurality of modes and determines an optimal mode.
The mode selecting unit 123 receives mode information regarding the optimal mode determined by the rate-distortion optimizing unit 122, sets up a flag corresponding to the optimal mode, and transmits the set up flag with predicted pixel values regarding a corresponding block.
In
The low pass filter LPF receives the block including Luma (G) pixel, removes high frequency region of the Luma (G) pixel, and transmits the low-pass filtered Luma (G) pixel values to the extended chroma prediction unit ECM.
The extended chroma prediction unit ECM predicts Chroma (B, R) pixel values by applying the Luma (G) pixel values transmitted from the correlation calculating unit COC or the low pass filter LPF to Equation 1.
Therefore, the LMchroma mode prediction unit LMCM off
Equation 2 is a mathematical expression of a block prediction technique performed by the LPF LMchroma mode LPFM.
Cpred[x,y]=αGreconlpf[x,y]+β [Equation 2]
(here, Greconlpf denotes a pixel value of a low pass filtered G block, and, like in Equation 1, α and β are weights and are inferred by using pixel values of the G block at same locations as pixel values around the B block or the R block.)
Greconlpf[x,y]=0.5 Grecon[x,y]+0.125 Grecon[x+1,y]+0.125Grecon[x−1,y]+0.125 Grecon[x,y+1]+0.125 Grecon[x,y−1] [Equation 3]
Equation 3 is an example of calculating a pixel value Greconlpf of a low pass filtered G block by using four neighboring pixel values.
The intra predicting unit 120 may provide not only conventional intra prediction modes, which are a planar mode, a vertical mode, a horizontal mode, and a DC mode, but also an LMchroma mode, which is an ECM suggested by J. Kim, and a low pass filter (LPF) LMchroma mode, which is a mode for applying an ECM after removing high frequency ingredients of a Luma (G) area by using a LPF based on a correlation between the Luma (G) area and the Chroma (B, R) areas even in the LMchroma mode according to the present invention. Furthermore, after block prediction is individually performed in each of the modes, an optimal mode is selected, and information regarding the selected optimal mode is transmitted together with pixel values regarding a block that are predicted in the optimal mode. Therefore, efficiency of predicting pixel values between color planes may be significantly improved.
Table 2 shows an example of flags according to an intra chroma prediction mode according to the present invention.
TABLE 2
chroma_pred_from_
chroma_pred_from_
luma_enabled_
luma_enabled_
flag = 1
flag = 0
intra_chroma_pred_mode
prefix
suffix
prefix
suffix
6 (DM_CHROMA)
0
n/a
n/a
n/a
5 (proposed method)
10
1
0
n/a
4 (ECM)
10
0
0
n/a
0 (PLANAR)
11
00
1
00
1 (VERTICAL)
11
01
1
01
2 (HORIZONTAL)
11
10
1
10
3 (DC)
11
11
1
11
As shown in Table 2, as an intra chroma prediction mode indicates an LMchroma mode and a LPF LMchroma mode prediction unit may be performed even in an LMchroma mode, a flag may be set up by adding a suffix 0 (when filtering is not applied) or a suffix 1 (when filtering is applied) based on whether a low pass filtering is performed in the LMchroma mode (prefix: 10) in case of binarization of intra prediction as shown in the second and third columns of Table 2. Although only the prefix ‘10’ bit was used as a flag for encoding a mode in an ECM mode in the related art, in the present invention, both the prefix ‘10’ bit and the suffix 1 bit are used to additionally use the LPF LMchroma mode, where the suffix 1 is used as a correlation flag. However, set locations and values of flags may vary.
To describe the intra prediction method of
Furthermore, the generated prediction blocks are transmitted to the rate-distortion optimizing unit 122, and the rate-distortion optimizing unit 122 analyzes pixel values of the prediction blocks and determine an optimal mode (operation S140). When the optimal mode is determined by the rate-distortion optimizing unit 122, the mode selecting unit 123 selects the determined optimal mode (operation S150). Next, a flag is set up according to pre-set rules based on the selected optimal mode (operation S160). When a flag is set up, the mode selecting unit 123 outputs the set up flag together with a selected prediction block (operation S 170).
Referring to
The image decoding apparatus 200 receives a bitstream output by an encoding apparatus, decodes the bitstream in an intra mode or an inter mode, and outputs a reconstructed image. In an intra mode, a prediction block is generated by using an in-screen prediction mode. In an inter mode, a prediction block is generated by using an inter-screen prediction mode. The image decoding apparatus 200 obtains a residual block from an input bitstream, generates a prediction block, and generates a block reconstructed by adding the residual block to the prediction block, that is, a reconstructed block.
The entropy decoding unit 210 decodes the input bitstream based on probability distribution and outputs a quantized coefficient. The quantized coefficient is inverse quantized by the inverse quantizing unit 220 and is inverse transformed by the inverse transforming unit 230. As a result of the inverse quantization and the inverse transformation of the quantized coefficient, a residual block is generated.
In an intra prediction mode, the intra predicting unit 240 generates a prediction block by performing a spatial prediction by using pixel values of blocks neighboring a current block. Here, in particular, the intra predicting unit 240 according to the present invention analyzes the flag of an input bitstream and, if the flag is set to the LMchroma mode, it is determined whether the correlation flag is set to the LPF LMchroma mode. If it is determined that the correlation flag is set to the LPF LMchroma mode, high frequency ingredients of the decoding-completed luma (G) area is removed by using a low pass filter (LPF) first. Next, an ECM technique for predicting chroma (B, R) pixels with reference to the Luma (G) pixel from which the high frequency ingredients are removed is applied.
In an inter prediction mode, the motion compensating unit 250 generates a prediction block by performing motion compensation by using a motion vector and a reference image stored in the reference image buffer 270.
The residual block and the prediction block are combined by the adder 255, and the combined block is processed by the filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, a SAO, and an ALF to a restored block or a restored picture. The filter unit 260 outputs a reconstructed image. The reconstructed image is stored in the reference image buffer 270 and may be used for an inter prediction.
An intra predicting unit 240 of
The flag analyzing unit 241 analyzes a flag included in a bitstream and transmits information regarding an analyzed mode to the mode selecting unit 242. The mode selecting unit 242 selects one of a planar mode prediction unit PM, a vertical mode prediction unit VM, a horizontal mode prediction unit HM, a DC mode prediction unit DC mode, and a LMchroma mode prediction unit LMCM included in the chroma predicting unit 243 based on the mode analyzed by the flag analyzing unit 241, activates the selected prediction unit, and transmits the received bitstream to the activated prediction unit.
The chroma predicting unit 243 includes the planar mode prediction unit PM, the vertical mode prediction unit VM, the horizontal mode prediction unit HM, the DC mode prediction unit DC mode, and the LMchroma mode prediction unit LMCM, each of which is activated according to a selection of the mode selecting unit 242 and generates a prediction block according to pre-set rules.
In particular, the LMchroma mode LMCM may predict the Chroma (B, R) pixel values with direct reference to the Luma (B) pixel values based on the correlation flag. Alternatively, the LMchroma mode LPFM may operate in the LPF LMchroma mode for removing high frequency ingredients of the decoded luma (G) area by using a low pass filter (LPF) and predicting Chroma (B, R) pixel values with reference to the Luma (G) pixel values from which the high frequency ingredients are removed.
Table 3 shows a result of analyzing performance of a LPF LM chroma mode according to the present invention by applying the LPF LM chroma mode to first frames of the eight RGB experimental images of Table 1. Table 3 was obtained by using the HM10.0_RExt2.0 program, where the HEVC range extension common test conditions (D. Flynn, K. Sharman, “Common test conditions and software reference configurations for HEVC range extensions”, JCTVC-L 1006, 12th Meeting, Geneva, Switzerland, January 2013.) were applied for setting experiment variables. Furthermore, performance of the suggested algorithm technique has been tested in All-Intra (AI) mode by using 8 RGB 4:4:4 test images, where a LPF used in the test was 2D-separatble LPF (ho=[1 2 1]/4). As shown in Table 3, compared to the conventional LM chroma mode, when an intra chroma prediction mode according to the present invention was performed, average bitrate-gains of 0.4%, 0.6%, and 0.8% could be obtained at respective color planes G, B, and R at the All Intra HE Super-High-tier compared to the ECM, which is the conventional LM chroma mode. Therefore, an overall average bitrate-gain of 0.6% could be obtained. Furthermore, encoding speed was increased by about 8%, whereas decoding speed was increased by 1%. In particular, since the decoding speed (complexity), which is most sensitive to speed increase, an average bitrate-gain of 0.6% could be obtained without increasing encoding and decoding speeds according to the technique according to the present invention. Therefore, the technique according to the present invention may be considered as an efficient technique.
TABLE 3
All Intra HE Super-High-tier
G
B
R
Traffic
0.0%
−0.2%
−0.6%
Kimono1
−0.6%
−0.9%
−0.9%
EBULupoCandlelight
−0.5%
−0.7%
−0.9%
EBURainFruits
0.6%
0.0%
−0.5%
VenueVu
−0.8%
−1.3%
−1.2%
DucksAndLegs
−0.5%
−0.3%
−0.1%
OldTownCross
−0.6%
−0.8%
−0.8%
ParkScene
−0.7%
−0.9%
−0.9%
Overall
−0.4%
−0.6%
−0.8%
Enc Time [%]
108%
Dec Time [%]
101%
In
The low pass filter LPF receives the Luma (G) area block, removes high frequency ingredients of the Luma (G) area block, and transmits the low-pass filtered pixel values to the extended chroma prediction unit ECM.
The extended chroma prediction unit ECM predicts the Chroma (B, R) pixel values by applying the Luma (G) pixel values transmitted from the correlation flag determining unit CFA or the low pass filter LPF to Equation 1.
Therefore, the LMchroma mode prediction unit LMCM off
Although it is described above that the correlation flag determining unit CFA is included in the LMchroma mode prediction unit LMCM for convenience of explanation, the correlation flag determining unit CFA may be included in the flag analyzing unit 241. If the correlation flag determining unit CFA is included in the flag analyzing unit 241, the mode selecting unit 242 may transmit a Luma (G) area block to either the low pass filter LPF or the extended chroma prediction unit ECM in the LMchroma mode prediction unit LMCM based on a correlation flag analyzed by the flag analyzing unit 241.
To describe the intra prediction method of
As described above, according to methods, apparatuses, and systems for encoding and decoding an image according to the present invention, in an ECM for predicting Chroma (B, R) pixel with reference to a Luma (G) pixel during an intra prediction, an LM Chroma mode may apply an ECM technique after a low-pass filtering is selectively applied based on a correlation between color planes, thereby improving coding efficiency.
The present invention can also be embodied as computer readable codes on a computer readable recording medium. The computer readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.
Choi, Jang Won, Choe, Yoon Sik
Patent | Priority | Assignee | Title |
11252421, | Sep 30 2013 | IDEAHUB INC | Method, device and system for encoding and decoding image |
11425419, | Dec 03 2013 | IDEAHUB INC | Method, apparatus, and system for encoding and decoding image using LM chroma prediction |
11683506, | Sep 30 2013 | IDEAHUB INC. | Method, device and system for encoding and decoding image |
Patent | Priority | Assignee | Title |
7227585, | Dec 30 2003 | Synaptics Incorporated | Luminance and chrominance separation system |
9948938, | Jul 21 2011 | Texas Instruments Incorporated | Methods and systems for chroma residual data prediction |
20090297054, | |||
20130336591, | |||
20140078394, | |||
20150003524, | |||
20150063460, | |||
JP4989048, | |||
KR101998007479, | |||
KR102007007232, | |||
KR102012000314, | |||
KR102012004128, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Sep 30 2014 | UNIVERSITY-INDUSTRY FOUNDATION (UIF), YONSEI UNIVERSITY | (assignment on the face of the patent) | / | |||
Jun 21 2016 | CHOE, YOON SIK | UNIVERSITY-INDUSTRY FOUNDATION UIF , YONSEI UNIVERSITY | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 046257 | /0321 | |
Jun 26 2016 | CHOI, JANG WON | UNIVERSITY-INDUSTRY FOUNDATION UIF , YONSEI UNIVERSITY | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 046257 | /0321 | |
May 14 2021 | UNIVERSITY INDUSTRY FOUNDATION OF YONSEI UNIVERSITY | IDEAHUB INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056358 | /0816 |
Date | Maintenance Fee Events |
Feb 03 2022 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Mar 07 2022 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Date | Maintenance Schedule |
Dec 18 2021 | 4 years fee payment window open |
Jun 18 2022 | 6 months grace period start (w surcharge) |
Dec 18 2022 | patent expiry (for year 4) |
Dec 18 2024 | 2 years to revive unintentionally abandoned end. (for year 4) |
Dec 18 2025 | 8 years fee payment window open |
Jun 18 2026 | 6 months grace period start (w surcharge) |
Dec 18 2026 | patent expiry (for year 8) |
Dec 18 2028 | 2 years to revive unintentionally abandoned end. (for year 8) |
Dec 18 2029 | 12 years fee payment window open |
Jun 18 2030 | 6 months grace period start (w surcharge) |
Dec 18 2030 | patent expiry (for year 12) |
Dec 18 2032 | 2 years to revive unintentionally abandoned end. (for year 12) |