A system and method that detects when pixels from one color space would exceed the range of a second color space, and restrict the range of the pixels without distorting their brightness, while preserving their color. The method may provide the ability to restrict the range of pixels in the first color space that may ordinarily transform to “illegal” values with erroneous brightness in the second color space, to a sub-space containing the entire second color space, while preserving the brightness of the pixels. The parameters used to define the sub-space may be programmable. The first color space may be YCrCb and the second color space may be rgb.

Patent
   8085277
Priority
Jan 30 2004
Filed
Oct 19 2004
Issued
Dec 27 2011
Expiry
Nov 25 2025
Extension
402 days
Assg.orig
Entity
Large
1
22
EXPIRED
1. A method that keeps pixels from a first color space in the range of a second color space, wherein the pixels have a color and brightness, the method comprising:
detecting when pixels in the first color space exceed the range of pixels in the second color space with a circuit;
restricting the pixels in the first color space to the range of the second color space;
defining a sub-space within the first color space using a set of parameters; and
determining whether a pixel is outside the sub-space or inside the sub-space, wherein the sub-space approximates the range of the second color space and comprises values from the first color space that are outside the second color space; and
wherein defining the sub-space within the first color space further comprises determining values in the first color space that correspond to the boundaries of the sub-space comprising values from the first color space that are outside the second color space.
10. A system that keeps pixels from a first color space in the range of a second color space, wherein pixels have a color and brightness, the system comprising:
at least one processor capable of detecting when pixels in the first color space exceed the range of pixels in the second color space;
the at least one processor capable of restricting the pixels in the first color space to the range of the second color space; and
wherein brightness of the restricted pixels is preserved, wherein the at least one processor:
defines a sub-space within the first color space using a set of parameters, wherein defining the sub-space within the first color space further comprises determining values in the first color space that correspond to the boundaries of the sub-space; and
determines whether a pixel is outside the sub-space or inside the sub-space, wherein the sub-space approximates the range of the second color space and comprises values from the first color space that are outside the second color space; and
wherein the sub-space in the first color space is larger than the unit cube in rgb color space.
16. A memory having stored thereon, a computer program having at least one code section that keeps pixels from a first color space in the range of a second color space, wherein pixels have a color and brightness, the at least one code section being executable by a machine for causing the machine to perform steps comprising:
detecting when pixels in the first color space exceed the range of pixels in the second color space;
restricting the pixels in the first color space to the range of the second color space; and
wherein brightness of the restricted pixels is preserved, wherein the code for restricting the pixels comprises:
code for defining a sub-space within the first color space using a set of parameters, wherein defining the sub-space within the first color space further comprises determining values in the first color space that correspond to the boundaries of the sub-space; and
code for determining whether a pixel is outside the sub-space or inside the sub-space, wherein the sub-space approximates the range of the second color space and comprises values from the first color space that are outside the second color space.
2. The method according to claim 1 wherein the restricting comprises, when a pixel is inside the sub-space keeping the pixel unchanged, wherein the pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.
3. The method according to claim 1 wherein the restricting comprises, when a pixel is outside the sub-space:
forcing the pixel to a new pixel in the sub-space, wherein the brightness and color of the new pixel is not visually deviated from the brightness and color of the pixel from the first color space; and
wherein the new pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.
4. The method according to claim 1 wherein the set of parameters is programmable.
5. The method of claim 4, wherein the sub-space in the first color space is larger than a unit cube in rgb color space.
6. The method according to claim 1 wherein the first color space is YCrCb color space.
7. The method according to claim 1 wherein the second color space is rgb color space.
8. The method of claim 1, wherein if the pixel is outside the sub-space, clipping the pixel to the sub-space.
9. The method of claim 1, wherein the sub-space in the first color space is larger than a unit cube in rgb color space.
11. The system according to claim 10 wherein, the at least one processor keeps the pixel unchanged when the pixel is inside the second color space, wherein the pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.
12. The system according to claim 10 wherein, when a pixel is outside the sub-space, the at least one processor:
forces the pixel to a new pixel in the sub-space, wherein the brightness and color of the new pixel is not visually deviated from the brightness and color of the pixel from the first color space; and
wherein the new pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.
13. The system according to claim 10 wherein the first color space is YCrCB color space.
14. The system according to claim 10 wherein the second color space is rgb color space.
15. The system of claim 10, wherein if the pixel is outside the sub-space, clipping the pixel to the sub-space.
17. The memory according to claim 16 wherein the code for restricting the pixels further comprises code for keeping the pixel unchanged when the pixel is inside the second color space, wherein the pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.
18. The memory according to claim 16 wherein the code for restricting the pixels further comprises, when a pixel is outside the sub-space:
code for forcing the pixel to a new pixel in the sub-space, wherein the brightness and color of the new pixel is not visually deviated from the brightness and color of the pixel from the first color space; and
wherein the new pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.
19. The memory of claim 16, wherein if the pixel is outside the sub-space, clipping the pixel to the sub-space.
20. The memory of claim 16, wherein the sub-space in the first color space is larger than a unit cube in rgb color space.

This patent application makes reference to, claims priority to and claims benefit from U.S. Provisional Patent Application Ser. No. 60/540,699, entitled “YCrCb to RGB Color Clip,” filed on Jan. 30, 2004, the complete subject matter of which is hereby incorporated herein by reference, in its entirety.

[Not Applicable]

[Not Applicable]

In video processing, transmitted video is almost always represented in a three-dimensional color space. Video may be represented in the YCrCb, where Y is the luminance or brightness component and Cr and Cb are the chrominance or color components. Video may also be represented in the RGB color space, where R is the red color component, G is the green color component, and B is the blue color component. While most video processing applications are in YCrCb space, most televisions displays utilize the RGB color space. Thus, when a television receives a transmission that in the YCrCb space, a conversion may take place to the RGB space. However, the YCrCb color space is larger than the RGB color space. If the YCrCb and the RGB color spaces were to be represented by cubes, the RGB cube would be entirely inside the YCrCb cube. Colors that are in the YCrCb color space, but that are outside the RGB color space are referred to as “out-of-gamut” and are not valid colors to the human eye. Conventional video processing can lead to generating colors that are out-of-gamut. One way of resolving this issue and transforming an out-of-gamut color space to an in-gamut color space is by “clipping” the out-of-gamut colors that are out of the RGB color space.

FIG. 1 illustrates a block diagram of a projection of a valid RGB cube to a (Cr, Y) plane. For a given Y value, many Cr values are out-of-gamut. Also, Y values in the range [0,16] or [235,255] have Cr=0, and in the YCrCb color space, pixels with a Y value above 235 appear white, and pixels with a Y value below 16 appear black. If a pixel outside the RGB gamut is converted to RGB with a standard matrix conversion, values will exceed the legal RGB ranges of [0,255]. As an example, an out-of-gamut value of YCrCb =[16, 0, 0] converts to RGB=[neg, 154, neg]. If the negative values are simply clipped to zero, the resulting color is bright green. But (Y==16) should result in black. The problem in converting from the YCrCb color space to the RGB color space is that a color that is represented by YCrCb may not fall within the RGB cube, and a decision needs to be made on what the television is supposed to display using RGB representation.

After converting YCrCb =[16, 0, 0] to RGB=[neg, 154, neg], the negative values may be easily clipped. However, the positive green value has substantially increased. Clipping the negative values, therefore, may not preserve luminance and may instead add positive luminance to the output. Thus, a pixel with a luminance of 16, which may have a color very close to black, when clipped to the RGB cube, ends up much brighter than it should be. Similarly, pixels in the region outside the RGB cube and above the line where the luminance is 235 should be white, but with clipping them to the RGB cube, the pixels end up with a color instead of being white, is an off-white color. The white pixels may become a little green or red or gray, but not white. Clipping causes noticeable artifacts around objects on the TV screen, especially, around the black and the white regions where other colors leak in.

Clipping may work moderately when the clipping produces a small error. However, clipping causes problems when large errors result such as, for example, when a pixel falls in the black and white regions, where Y values are below 16 and above 235, respectively. Because many video devices output YCrCb signals directly, a method is desired to keep the YCrCb values in a legal range, without the complexity of converting to the RGB space, clipping, and then converting back to YCrCb. In addition, simply “clipping” the out-of-gamut values back to the RGB space is acceptable for small excursions, but it can cause visible distortion for signals that are far out-of-gamut. The eye is most sensitive to luminance than hue, and least sensitive to saturation. As a result, it is most favorable to preserve luminance, and sacrifice saturation.

Although it is possible to accurately limit saturation to a color space, mathematically precise methods involve coordinate transforms that can be prohibitively expensive for low-cost applications. Because small excursions and small changes to hue are not objectionable, it is desirable to have an approximate solution with reasonable quality at a low cost.

Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings.

Aspects of the present invention may be seen in a system and method that keeps pixels from a first color space in the range of a second color space, wherein pixels have a color and brightness, and where the brightness of the restricted pixels is preserved. The method may comprise detecting when pixels in the first color space exceed the range of pixels in the second color space, and restricting the pixels in the first color space to the range of the second color space. Detecting when pixels in the first color space exceed the range of pixels in the second color space may comprise defining a sub-space within the first color space using a set of parameters, and determining whether a pixel is outside the sub-space, or inside the sub-space, wherein the sub-space approximates the range of the second color space. The set of parameters defining the sub-space may be programmable.

When a pixel is inside the sub-space, the method may comprise keeping the pixel unchanged, wherein the pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.

Whenever a pixel is located outside the sub-space, that pixel may be forced to a new pixel inside the sub-space, wherein the brightness and color of the new pixel is not visually deviated from the brightness and color of the pixel from the first color space, and wherein the new pixel inside the sub-space has brightness and color that do not visually deviate when mapped to the second color space.

The system comprises at least one processor capable of performing the method that keeps pixels from a first color space in the range of a second color space, wherein pixels have a color and brightness, and where the brightness of the restricted pixels is preserved.

These and other features and advantages of the present invention may be appreciated from a review of the following detailed description of the present invention, along with the accompanying figures in which like reference numerals refer to like parts throughout.

FIG. 1 illustrates a block diagram of a projection of a valid RGB cube to a (Cr,Y) plane.

FIG. 2 illustrates an exemplary video network system having a color clip circuit, in accordance with an embodiment of the present invention

FIG. 3 illustrates a block diagram of an exemplary projection of a RGB color cube to the Y-Cr plane in the YCrCb color space, in accordance with an embodiment of the present invention.

FIG. 4 illustrates a block diagram of an exemplary projection of a RGB color cube to the Y-Cb plane in the YCrCb color space, in accordance with an embodiment of the present invention.

Aspects of the present invention relate to processing of video signals. More specifically, certain embodiments of the invention relate to a method and system that performs adequate color clipping to preserve luminance of pixels in the YCrCb color space to restrict the pixels to a valid range in the RGB color space by detecting when the pixels in the YCrCb color space would exceed the range (or gamut) of the RGB color space. Although the following discusses the invention in terms of the YCrCb color space and the RGB color space, it should be understood that the following may be applied to other color spaces such as, for example, YIQ, YPrPb, YUV, etc.

FIG. 2 illustrates an exemplary video network system 100 having a color clip circuit, in accordance with an embodiment of the present invention. The color clip circuit may be located in the compositor (comp) block 101.

FIG. 3 illustrates a block diagram of an exemplary projection of a RGB color cube to the Y-Cr plane in the YCrCb color space, in accordance with an embodiment of the present invention. A RGB cube 301 may be projected to a (Cr, Y) plane 303, where the Cr values may be bound to the designated Area S 305, thus effectively clipping the YCrCb color space to a RGB color space. The designated area S 305 may be bounded by slopes C, D, E and F and points H and J.

FIG. 4 illustrates a block diagram of an exemplary projection of a RGB color cube to the Y-Cb plane in the YCrCb color space, in accordance with an embodiment of the present invention. The RGB cube may also be projected to the (Cb, Y) plane 403, in a similar manner as the projection to the (Cr, Y) plane 303, where the Cb axis is perpendicular to the (Cr, Y) plane 303. The Cb values may be bound to a designated region 405, but with different slopes and a different point. As a result, the whole RGB cube 301 may be contained within a YCrCb space defined by the slopes C, D, E, and F, and points H and J, and the slopes and points in the (Cb, Y) plane. In alternative embodiments of the present invention, different color spaces may have different projections and may be approximated by different slopes and points.

A pixel may have a (YCrCb) value inside the RGB color space 301, which would be valid and may not be affected by the transformation from one color space to the other. However, another pixel may have a (YCrCb) value outside the RGB cube 301. If a pixel is outside the legal RGB cube, it may be desirable to correct the pixel in a way that preserves luminance. Hence, if the pixel is white or black, the correction may ensure that it remains white or black, respectively. Area S (305 for Cr, and 405 for Cb) may only be an approximation of the true RGB gamut (301). An embodiment of the present invention may only perform a partial correction to area S. The approximate correction may not be exact, but it may be close enough to give visually acceptable performance. Also, the approximation error may be very small in the black and white regions (regions where the Y values are below 16 and above 235) where errors may be most noticeable.

While the following derivations and calculations may derive parameters for standard definition (SD) television as define in ITU Rec. 601, the parameters in an embodiment of this invention may be programmable for operation with a wide variety of color spaces that may be used in the video industry.

In accordance with an embodiment of the present invention, restricting pixels in the YCrCb color space to the range of the RGB color space may be calculated as follows:
Let Y′=(Y−16), Cb′=(Cb−128), Cr′=(Cr−128)
R=1.164 Y′+1.596 Cb′
G=1.164 Y′−0.813 Cb′−0.391 Cr
B=1.164 Y′+2.018 Cr
Y′=0.257*R+0.504*G+0.098*B

Negative clips may be avoided by keeping R, G, and B from clipping negative. Whenever R<0, this implies 1.596 V′<−1.164 Y′. The rule may be, therefore, that in order to keep R from clipping negative, the following must be true for Cb′: Cb′>−1.164/1.596 Y′=−0.729 Y′.

In order to keep B from clipping negative, the following relationship may be satisfied for Cr′: Cr′>−1.164/2.018 Y′=−0.577 Y′.

If K=2Cb′+Cr′, it may be approximated that G=1.164Y′−0.4K. In this embodiment, therefore, the rule may be that, in order to keep G from clipping negative and greater than zero, it must be true that K<2.91 Y′.

Since it may be difficult to correct green clips and preserve hue, Cr and Cb values may be measured independently. G=1.164Y′−0.813Cb′−0.391Cr′>0. Since Cb′>−0.729Y′, therefore 1.164Y′+0.813(0.729)Y′−0.391Cr′>0 and Cr′>4.493Y′. Similarly, 1.164Y′−0.813V′+0.391(0.577)Y′>0 and Cr′>1.709Y′.

Positive clips may be adjusted as follows.
Let Y*=219−Y′→Y′=219−Y*

Any equation that begins with “255=1.164Y′+ . . . ” may be re-written as “0=1.1643638(219−Y*)−255+ . . . ” Or, “0=1.164Y*+ . . . ”. The value of R may be restricted from exceeding 255 by solving the equation 1.164Y*+1.596 Cb′<0. Therefore, the value of R may be restricted from exceeding 255 if Cb′<0.729 Y*. The same type of equations may be applied to all other positive clips.

In another embodiment of the present invention, the high definition (HD) television coefficients, as defined in ITU Rec. 709, may be calculated as follows.
R=1.164 Y′+1.793 Cb
G=1.164 Y′−0.534 Cb′−0.213Cr
B=1.164 Y′+2.115 Cr
Y′=0.183*R+0.614*G+0.062*B
Cb′>−1.164/1.793Y′=−0.649Y′
Cr′>−1.164/2.1 5Y′=−0.550Y′
1.164Y′−0.534Cb′−0.213Cr>0
Cr′>(1.164Y′+0.534*0.649Y′)/0.213=7.09 Y′
Cb′>(1.164Y′+0.213*0.550Y′)/0.534=2.40 Y′

A color clip block in accordance with an embodiment of the present invention may be adapted so that it may be enabled/disabled at any time without causing a discontinuity in an associated display. Separate enables may be available for positive and negative clipping. In addition, the slopes C, D, E, and F, and points H and J in the (Cr, Y) plane, and similar slopes and points for the (Cb, Y) plane, may be programmable.

A YCrCb to RGB color clip may be implemented with pseudo-code, requiring one multiply for Cr, one multiply for Cb and adders. Multiplier coefficients may be 10 bits. The pseudo-code may be as follows:

void clip(y,u,v) {
 ty = MAX(*y, 16) − 16; // note that y has a useful range of 16–235
 tyx = 219.0 − ty;
 if (tyx < 0.0) {tyx = 0.0;}
 tu = u − 128;
 tv = u − 128;
 ty += 0.5;
 tyx += 0.5;
  // clip U
  if (tu > 0) {
   if (ty < 25) { // “25” is point “J” in FIG. 2
       if (tu > (4.5 * ty))    {tu = 4.5 * ty;} // slope A
   }
   else {
       if (tu > (0.58 * tyx)) (tu = 0.58 * tyx;} // slope B
   }
  }
  else {
   if (tyx < 25) {
       if (−tu > (4.5 * tyx))   (tu = −4.5 * tyx;}
   }
   else {
       if (−tu > (0.58 * ty))   {tu = −0.58 * ty;}
   }
  }
  // clip V is similar to U, with different coefficients
  {...}
u = tu + 128;
v = tv + 128;
}

Accordingly, the present invention may be realized in hardware, software, or a combination of hardware and software. The present invention may be realized in a centralized fashion in at least one computer system, or in a distributed fashion where different elements may be spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein may be suitable. A typical combination of hardware and software may be a general-purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein. In a computer system, a processor and a memory unit may be utilized to achieve the methods described herein.

The present invention may also be embedded in a computer program product, which may comprise all the features enabling the implementation of the methods described herein, and which when loaded in a computer system may be able to carry out these methods. A computer program in the present context may mean any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.

While the present invention has been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present invention without departing from its scope. Therefore, it is intended that the present invention not be limited to the particular embodiment disclosed, but that the present invention will include all embodiments falling within the scope of the appended claims.

Schoner, Brian

Patent Priority Assignee Title
8237747, Apr 04 2005 Koninklijke Philips Electronics N V Method of converting signals for multi-primary color display
Patent Priority Assignee Title
5233684, Jun 26 1990 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Method and apparatus for mapping a digital color image from a first color space to a second color space
5732205, Dec 30 1994 Intel Corporation Color conversion using 4.5 bit palette
5818613, Dec 04 1995 Microsoft Technology Licensing, LLC System and method for color space conversion
5867169, Apr 17 1996 Pixar Method and apparatus for manipulating color values in a computer graphics system
5946113, Mar 29 1996 Open Invention Network, LLC System and method for color space conversion
6326977, Nov 03 1998 Sharp Kabushiki Kaisha Rendering of YCBCR images on an RGS display device
6510242, Oct 26 1999 Sharp Laboratories of America, Inc.; Sharp Laboratories of America, Inc Color image enhancement during YCbCr to RGB color conversion
6707459, Feb 01 2001 Apple Inc Adjustment of color values for optimized image processing
6735334, Oct 29 1999 Canon Kabushiki Kaisha Optimizing processing gamuts for color conversion chains
6825853, Mar 26 2002 Lucent Technologies Inc. Apparatus and method for color space conversion in video images
7057671, Jan 30 2002 AT & T Corp Image and video processing with chrominance attenuation
7129957, Dec 20 2002 Samsung Electronics Co., Ltd. Color signal processing device and method
7130462, Jan 15 2002 Seiko Epson Corporation Output and store processed image data
20030052895,
20030072015,
20030184559,
20040207882,
20050031199,
20090085924,
EP1333684,
JP2003092690,
JP2005252393,
////////
Executed onAssignorAssigneeConveyanceFrameReelDoc
Oct 18 2004SCHONER, BRIANBroadcom CorporationASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0156110190 pdf
Oct 19 2004Broadcom Corporation(assignment on the face of the patent)
Feb 01 2016Broadcom CorporationBANK OF AMERICA, N A , AS COLLATERAL AGENTPATENT SECURITY AGREEMENT0378060001 pdf
Jan 19 2017BANK OF AMERICA, N A , AS COLLATERAL AGENTBroadcom CorporationTERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS0417120001 pdf
Jan 20 2017Broadcom CorporationAVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0417060001 pdf
May 09 2018AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LIMITEDMERGER SEE DOCUMENT FOR DETAILS 0471960687 pdf
Sep 05 2018AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LIMITEDCORRECTIVE ASSIGNMENT TO CORRECT THE PROPERTY NUMBERS PREVIOUSLY RECORDED AT REEL: 47630 FRAME: 344 ASSIGNOR S HEREBY CONFIRMS THE ASSIGNMENT 0488830267 pdf
Sep 05 2018AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LIMITEDCORRECTIVE ASSIGNMENT TO CORRECT THE EFFECTIVE DATE OF MERGER TO 9 5 2018 PREVIOUSLY RECORDED AT REEL: 047196 FRAME: 0687 ASSIGNOR S HEREBY CONFIRMS THE MERGER 0476300344 pdf
Date Maintenance Fee Events
Jun 30 2015M1551: Payment of Maintenance Fee, 4th Year, Large Entity.
Jun 30 2015M1554: Surcharge for Late Payment, Large Entity.
Aug 19 2019REM: Maintenance Fee Reminder Mailed.
Feb 03 2020EXP: Patent Expired for Failure to Pay Maintenance Fees.


Date Maintenance Schedule
Dec 27 20144 years fee payment window open
Jun 27 20156 months grace period start (w surcharge)
Dec 27 2015patent expiry (for year 4)
Dec 27 20172 years to revive unintentionally abandoned end. (for year 4)
Dec 27 20188 years fee payment window open
Jun 27 20196 months grace period start (w surcharge)
Dec 27 2019patent expiry (for year 8)
Dec 27 20212 years to revive unintentionally abandoned end. (for year 8)
Dec 27 202212 years fee payment window open
Jun 27 20236 months grace period start (w surcharge)
Dec 27 2023patent expiry (for year 12)
Dec 27 20252 years to revive unintentionally abandoned end. (for year 12)