The present invention relates to an interpolation method for enlarging a digital image or predicting a moving vector of a compressed image system as a sub-pixel unit when the image digitized through a CCD (Charge Coupled Device) camera ect. has a low resolution in a video phone or video conference or general digital video system, particularly the present invention can be adapted to a post processor of a compressed digital image in order to improve the image quality, and can be used for finding a moving vector of a moving picture compressed type, accordingly the present invention is capable of improving the image quality.
|
0. 20. A method for a moving picture compression with a video system by generating an interpolated pixel data, the method comprising:
generating a set of high-resolution interpolated pixel data from a set of original pixel data from an original image; and
finding a sub-pixel motion vector using the set of high-resolution interpolated pixel data,
the high-resolution interpolated pixel data for a particular pixel is generated by performing operations using the video system, the operations comprising:
selecting, by the video system, original pixel data including more than three pixels of the original image;
obtaining, by the video system, at least a first filter coefficient and a second filter coefficient, the first filter coefficient and the second filter coefficient being configured to interpolate the original pixel data;
applying, by the video system, the first filter coefficient to the selected original pixel data to produce first interpolated pixel data, the first filter coefficient including weighting factors having at least three different numerical values, wherein said applying the first filter coefficient to the selected original pixel data comprises:
multiplying each of the weighting factors and the selected original pixel data to produce weighted pixel data, and
summing the weighted pixel data to produce the first interpolated pixel data;
multiplying, by the video system, the second filter coefficient and the first interpolated pixel data to produce second interpolated pixel data; and
identifying, by the video system, the high-resolution interpolated pixel data for the particular pixel as the second interpolated pixel data.
0. 1. A filtering control method for improving the image quality of a bi-linear interpolated image when recovering a high resolution image from a low resolution image, the method comprising performing operations using at least one processor, the operations comprising:
restoring a requested high resolution image f by finding an added filter coefficient Q of a PSF(P) and a bi-linear interpolation filter B from an equation f=Pg=PBz=Qz, wherein f is the high resolution image as requested, P is the PSF (Point Spread Function), g is the high resolution image found by the bi-linear interpolation method, and z is the low resolution image;
wherein the high resolution image f can be restored by performing an added function M(f) definition process for finding the PSF(H) from an equation g=Bz=Hf+n, wherein B, H are bi-linear interpolation filters, and n is a noise component generated by the assumed H; and
wherein the added function M(f) is defined as M(f)=∥g−Hf∥2+α∥Cf∥2, wherein α is a regularization parameter, and C is a two-dimensional high frequency filter for finding mitigation of the original image.
0. 2. The filtering control method for improving the image quality of the bi-linear interpolated image according to
0. 3. The filtering control method for improving image quality of the b-linear interpolated image according to
0. 4. A filtering control method for improving the image quality of a bi-linear interpolated image when recovering a high resolution image from a low resolution image, the method comprising performing operations using at least one processor, the operations comprising:
restoring a requested high resolution image f by finding an added filter coefficient Q of a PSF(P) and a bi-linear interpolation filter B from an equation f=Pg=PBz=Qz, wherein f is the high resolution image as requested, P is the PSF (Point Spread Function), g is the high resolution image found by the bi-linear interpolation method, and z is the low resolution image;
wherein the high resolution image f can be restored by performing an added function M(f) definition process for finding the PSF(H) from an equation g=Bz=Hf+n, wherein B, H are bi-linear interpolation filters, and n is a noise component generated by the assumed H;
wherein the high resolution image f is restored by finding a PSF(P) of a f=Pg function after finding the PSF(H) from the added function M(f); and
wherein the PSF(H) is found by using an equation
G(k,l) is the component in the k,l frequency region of the bi-linear interpolated image, and F(k,l) is the component in the k,l frequency region of the high resolution image.
0. 5. A filtering control method for improving the image quality of a bi-linear interpolated image when recovering a high resolution image from a low resolution image, the method comprising performing operations using at least one processor, the operations comprising:
restoring a requested high resolution image f by finding an added filter coefficient Q of a PSF(P) and a bi-linear interpolation filter B from an equation f=Pg=PBz=Qz, wherein f is the high resolution image as requested, P is the PSF (Point Spread Function), g is the high resolution image found by the bi-linear interpolation method, and z is the low resolution image;
wherein the PSF(P) can be found by getting an IFT (Inverse Fourier Transform) by an equation
H(k,l) is a component in the k,l frequency region of the PSF(H), and C is a two-dimensional high frequency filter.
0. 6. The filtering control method for improving the image quality of the bi-linear interpolated image according to
0. 7. A filtering control method for improving the image quality of a bi-linear interpolated image when recovering a high resolution image from a low resolution image, the method comprising performing operations using at least one processor the operations comprising:
defining an added function M(f) for finding a PSF(H) from an equation g=Bz=Hf+n (wherein B, H are bi-linear filters, n is a noise component generated by an assumed H when the H is a PSF (Point Spread Function), f is a requested high resolution image, z is a low resolution image, and g is a high resolution image gotten by the bi-linear interpolation method);
finding a PSF(P) of a f=Pg function after finding the PSF (H) from the defined added function M(f); and
restoring the requested high resolution image f by finding an added filter coefficient Q of the PSF(P) and interpolation filter B from the equation f=Pg=PBZ=Qz;
wherein the added function M(f) is defined as M(f)=∥g−Hf∥2+α∥Cf∥2, wherein α is a regularization parameter, and C is a two-dimensional high frequency filter for finding the mitigation of the original image.
0. 8. The filtering control method for improving the image quality a of the bi-linear interpolated image according to
0. 9. The filtering control method for improving image quality of the bi-linear interpolated image according to
0. 10. A filtering control method for improving the image quality of a bi-linear interpolated image when recovering a high resolution image from a low resolution image, the method comprising performing operations using at least one processor the operations comprising:
defining an added function M(f) for finding a PSF(H) from an equation g=Bz=Hf+n (wherein B, H are bi-linear filters, n is a noise component generated by an assumed H when the H is a PSF (Point Spread Function), f is a requested high resolution image, z is a low resolution image, and g is a high resolution image gotten by the bi-linear interpolation method);
finding a PSF(P) of a f=Pg function after finding the PSF (H) from the defined added function M(f); and
restoring the requested high resolution image f by finding an added filter coefficient Q of the PSF(P) and interpolation filter B from the equation f=Pg=PBZ=Qz;
wherein the PSF(H) is found by an equation
H(k,l)=(G(k,l)/F(−k,l), wherein G(k,l) is the component in the k,l frequency region of the bi-linear interpolated image, and F(k,l) is the component in the k,l frequency region of the high resolution image.
0. 11. A filtering control method for improving the image quality of a bi-linear interpolated image when recovering a high resolution image from a low resolution image, the method comprising performing operations using at least one processor, the operations comprising:
defining an added function M(f) for finding a PSF(H) from an equation g=Bz=Hf+n (wherein B, H are bi-linear filters, n is a noise component generated by an assumed H when the H is a PSF (Point Spread Function), f is a requested high resolution image, z is a low resolution image, and g is a high resolution image gotten by the bi-linear interpolation method);
finding a PSF(P) of a f=Pg function after finding the PSF(H) from the defined added function M(f); and
restoring the requested high resolution image f by finding an added filter coefficient Q of the PSF(P) and interpolation filter B from the equation f=Pg=PBZ=Qz;
wherein the PSF(P) is found by using an IFT (Inverse Fourier Transform) by an equation
H(k,l) is a component in the k,l frequency region of the PSF(H), and C is a two-dimensional high frequency filter.
0. 12. The filtering control method for improving the image quality of the bi-linear interpolated image according to
0. 13. A method for generating an interpolated pixel data, the method comprising generating a set of the interpolated pixel data from a set of original pixel data from an original image, wherein interpolated pixel data for a particular pixel is generated by performing operations using at least one processor, the operations comprising:
selecting original pixel data for more than three pixels of the original image;
obtaining at least a first filter coefficient and a second filter coefficient, the first filter coefficient and the second filter coefficient being configured to interpolate the original pixel data;
applying the first filter coefficient to the selected original pixel data to produce first interpolated pixel data, the first filter coefficient including weighting factors having at least three different numerical values, wherein applying the first filter coefficient to the selected original pixel data comprises:
multiplying each of the weighting factors and the selected original pixel data to produce weighted pixel data, and
summing the weighted pixel data to produce the first interpolated pixel data;
multiplying the second filter coefficient and the first interpolated pixel data to produce second interpolated pixel data; and
identifying the interpolated pixel data as the second interpolated pixel data.
0. 14. The method of claim 13, wherein the second filter coefficient is a matrix that includes one or more individual numeric values.
0. 15. The method of claim 13, wherein the first filter coefficient and the second filter coefficient each comprise at least one integer value.
0. 16. The method of claim 13, wherein a value of the first filter coefficient and a value of the second filter coefficient are one.
0. 17. The method of claim 13, wherein a value of the second filter coefficient is one.
0. 18. The method of claim 13, wherein the original image is obtained from a low-resolution imaging system.
0. 19. The method of claim 13, wherein the second filter coefficient is a point spread function (P) and the first filter coefficient is a bi-linear interpolation filter (B).
0. 21. The method of claim 20, wherein the second filter coefficient is a matrix that includes one or more individual numeric values.
0. 22. The method of claim 20, wherein the first filter coefficient and the second filter coefficient each comprise at least one integer value.
0. 23. The method of claim 20, wherein the original image is obtained from a low-resolution imaging system.
0. 24. The method of claim 20, wherein the second filter coefficient is a point spread function (P) and the first filter coefficient is a bi-linear interpolation filter (B).
0. 25. The method of claim 20, wherein the sub-pixel motion vector includes a 1/3 unit motion vector.
|
Herein, the B, H, n are the bi-linear interpolation filters, H is the spatially invariant PSF defining the relation between the original high resolution image and high resolution image gotten by the interpolation method, and the n is a noise component generated by the assumed H.
Herein, when the noise component is neglected and a direct inverse is used in order to find the PSF(H), the PSF(H) can be described as below equation 2 in the frequency region.
Herein, the H(k,l) is the component in the k,l frequency region of the PSF(H), the G (k,l) is the component in the k,l frequency region of the bi-linear interpolated image. In addition, the F (k,l) is the component in the k,l frequency region of the high resolution image.
Meanwhile, the high resolution image f to be restored is unknown, the PSF(H) can be gotten from the bi-linear interpolated high resolution image through a statistical processing after performing an under-sample processing of various images as various value.
Herein, the high resolution image is gotten by using the PSF(H) found from the direct inverse. In other words, there is a system stabilization problem because the high resolution image gotten from the PSF(H) by using the direct inverse is overshoot in the region where the k,l have ‘0’ value (in general, high frequency region) in the frequency region, accordingly the regularization image restoration for improving the system stabilization is used to solve the problem.
The regularization image restoration method is used for restoring the image or finding a certain PSF, an added function M(f) for finding the PSF(H) by using the regularization image restoration method can be described as below equation 3.
M(f)=∥g−Hf∥2+α∥Cf∥2 [Equation 3]
Herein, the first term of the right side of Equation 3 illustrates the credibility of the bi-linear interpolated image, the second term of the right side illustrates increase of the stability of the system by providing the mitigation to the restored image.
In addition, the ∥.∥ means a norm, the α is a regularization parameter for determining the credibility and mitigation of the original image. In addition, the C is the two-dimensional high frequency filter for determining the mitigation of the original image, in the present invention a two-dimensional Gaussian filter is used as the C.
When a gradient operator is adapted to Equation 3 in order to get the high resolution image, it can be described as below equation 4.
□fM(f)=−2HT(g−Hf)+2αCTCf=0 [Equation 4]
Herein, the T means a transpose of a matrix.
Meanwhile, conventionally a repetition method is used in order to get the high resolution image and regularization parameter, but it is not suited to the moving picture processing because the method causes lots of computational complexity.
Accordingly, in the present invention, the regularization parameter α is fixed as ‘1’, and the high resolution image f can be found as below equation 5.
When the PSF(P) is found by Equation 5, PSF(P)=H/(HTH+CTC) requires the lots of computational complexity for calculating an inverse matrix, however the PSF(P) in Equation 5 is a block-circulant matrix, accordingly it can be easily calculated in the frequency region.
Accordingly, the PSF(P) can be found finally as below Equation 6.
Herein, the ‘*’ means a complex-conjugate.
The PSF(P) can be found by using an IFT (Inverse Fourier Transform) from Equation 6.
The requested high resolution image f can be found as below Equation 7 by using the found PSF(P) and Equation 1.
f=Pg=PBz=Qz [Equation 7]
The PSF(P) is the spatially invariant function, the bi-linear interpolation filter B can be easily found by the conventional technology, accordingly the added filter coefficient Q of the PSF(P) and bi-linear interpolation filter B can be found.
Herein, in order to reduce the computational complexity, the number of a kernel of the PSF(P) is set in accordance with the up-sampling value.
When the up-sampling value is 2 in the present invention, the number of the kernel is limited as 3, when the up-sampling value is 3, the number of the kernel is limited as 4.
When the up-sampling value is 2, it can be used in an application segment for enlarging the size of the image as twice at a post processor of the compressed digital image and in finding of a sub-pixel moving vector in a H.263 moving picture compressed method.
In addition, when the up-sampling value is 3, it can be used in using of a ⅓ unit moving vector in a H.26L moving picture compressed method.
Herein, the H.263 and H.26L are moving picture compressed standards presented in the ITU-T (International Telecommunications Union-Telecommunication).
As described above, the present invention can be used for improving the image quality at the post processor of the compressed digital image by using the interpolation method for getting the interpolated high resolution image from the low resolution image when the resolution of the digital image lowers due to the low resolution image sensor.
In addition, the interpolation method of the present invention can improve the image quality by finding the moving vector of the moving picture compressed type.
Hong, Min-Cheol, Soh, Yoon-Seong
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
4550437, | Jun 19 1981 | Hitachi, Ltd. | Apparatus for parallel processing of local image data |
5208872, | Mar 30 1990 | The United States of America as represented by the United States | Programmable remapper with single flow architecture |
5258938, | Nov 30 1990 | Norio, Akamatsu; Alpine Electronics, Inc. | Interpolating method using bit-shift and addition/subtraction operations |
5274469, | Dec 23 1991 | Eastman Kodak Company | Sample rate converter circuit for image data |
5396592, | May 30 1991 | SONY CORPORATION A CORP OF JAPAN | Image signal interpolating circuit for calculating interpolated values for varying block sizes |
5659364, | Dec 24 1993 | Matsushita Electric Industrial Co., Ltd. | Motion vector detection circuit |
5696848, | Mar 09 1995 | Intellectual Ventures Fund 83 LLC | System for creating a high resolution image from a sequence of lower resolution motion images |
5821915, | Oct 11 1995 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Method and apparatus for removing artifacts from scanned halftone images |
5838387, | Dec 20 1996 | Intel Corporation | Digital video scaling engine |
5875268, | Sep 27 1993 | Canon Kabushiki Kaisha | Image processing with low-resolution to high-resolution conversion |
5880767, | Sep 11 1996 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Perceptual image resolution enhancement system |
5917963, | Sep 21 1995 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
5949914, | Mar 17 1997 | GEOEYE SOLUTIONS HOLDCO INC ; DIGITALGLOBE, INC | Enhancing the resolution of multi-spectral image data with panchromatic image data using super resolution pan-sharpening |
5991464, | Apr 03 1998 | Odyssey Technologies | Method and system for adaptive video image resolution enhancement |
6058248, | Apr 21 1997 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Computerized method for improving data resolution |
6061477, | Apr 18 1996 | Sarnoff Corporation | Quality image warper |
6072907, | May 28 1997 | Xerox Corporation | Method and apparatus for enhancing and thresholding images |
6075926, | Apr 21 1997 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Computerized method for improving data resolution |
6154574, | Nov 19 1997 | Samsung Electronics Co., Ltd. | Digital focusing method and apparatus in image processing system |
6236433, | Sep 29 1998 | Intel Corporation | Scaling algorithm for efficient color representation/recovery in video |
6252576, | Aug 06 1998 | DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT | Hardware-efficient system for hybrid-bilinear image scaling |
6263120, | Nov 11 1997 | Sharp Kabushiki Kaisha | Image data interpolation processing method |
6285804, | Dec 21 1998 | Sharp Kabushiki Kaisha | Resolution improvement from multiple images of a scene containing motion at fractional pixel values |
6331902, | Oct 14 1999 | LIBRE HOLDINGS, INC | System and method for digital color image processing |
6496608, | Jan 15 1999 | Qualcomm Incorporated | Image data interpolation system and method |
6529638, | Feb 01 1999 | RAKUTEN, INC | Block boundary artifact reduction for block-based image compression |
6567568, | Jan 26 1998 | MINOLTA CO , LTD | Pixel interpolating device capable of preventing noise generation |
6665344, | Jun 29 1998 | Zenith Electronics Corporation | Downconverting decoder for interlaced pictures |
6714593, | Oct 21 1997 | Robert Bosch GmbH | Motion compensating prediction of moving image sequences |
6912004, | Sep 15 1998 | Microsoft Technology Licensing, LLC | Method and system for processing images |
6950469, | Sep 17 2001 | Nokia Technologies Oy | Method for sub-pixel value interpolation |
8428396, | Dec 04 2007 | Huawei Technologies Co., Ltd.; Tsinghua University; HUAWEI TECHNOLOGIES CO , LTD | Image interpolation method and apparatus, and method for obtaining interpolation coefficients |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Dec 04 2015 | LG Electronics Inc. | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Date | Maintenance Schedule |
Apr 02 2022 | 4 years fee payment window open |
Oct 02 2022 | 6 months grace period start (w surcharge) |
Apr 02 2023 | patent expiry (for year 4) |
Apr 02 2025 | 2 years to revive unintentionally abandoned end. (for year 4) |
Apr 02 2026 | 8 years fee payment window open |
Oct 02 2026 | 6 months grace period start (w surcharge) |
Apr 02 2027 | patent expiry (for year 8) |
Apr 02 2029 | 2 years to revive unintentionally abandoned end. (for year 8) |
Apr 02 2030 | 12 years fee payment window open |
Oct 02 2030 | 6 months grace period start (w surcharge) |
Apr 02 2031 | patent expiry (for year 12) |
Apr 02 2033 | 2 years to revive unintentionally abandoned end. (for year 12) |