A system for generating a digital output signal representing a captured image includes a sensor for capturing the image and generating a sensor output signal. A gain control amplifier is coupled to the sensor and receives the sensor output signal. The gain control amplifier has controls for applying various levels of gain to the sensor output signal. An analog-to-digital converter is coupled to the gain control amplifier and generates the digital output signal representing the captured image. A processor is coupled to the analog-to-digital converter and the gate control amplifier. The processor provides a control signal to the gain control amplifier for adjusting the level of gain applied by the amplifier.
|
0. 39. A method of processing an image, comprising:
amplifying a first portion of the image to a first gain level;
amplifying a second portion of the image to a second gain level; and
updating the first gain level and the second gain level in a gain map.
0. 23. An apparatus adapted to process an image, comprising:
an amplifier adapted to apply more than one gain level to the image; and
a processor coupled to the amplifier, wherein the processor is capable of adjusting the more than one gain level applied by the amplifier in real time, and wherein the processor is capable of updating a gain map comprising settings applied by the amplifier.
17. A method for enhancing the dynamic range of a sensor output signal representing a captured image, the method comprising the steps of:
amplifying the sensor output signal in response to gain settings contained in a gain map, wherein each gain setting is associated with a particular region of the captured image; and
updating the gain settings contained in the gain map in response to changes in the sensor output signal.
22. An apparatus for capturing an image and generating a signal representing the captured image, comprising:
means for amplifying the signal in response to gain settings contained in a gain map, wherein each gain setting is associated with a particular region of the captured image;
means for updating the gain settings contained in the gain map; and
means for generating a control signal indicating a particular gain setting to be applied to a portion of the signal representing the captured image.
1. A camera comprising:
a sensor configured to capture an image and generate a sensor output signal representing the captured image;
an amplifier coupled to receive the sensor output signal, wherein the amplifier is configured to apply multiple gain levels to the sensor output signal; and
a processor coupled to the amplifier, wherein the processor is configured to provide a control signal to the amplifier to adjust the gain levels applied by the amplifier, the processor adapted to adjust the gain level of one portion of the captured image in response to a value of a previous portion of the captured image.
0. 33. An apparatus capable of processing an image, comprising:
an image capture device coupled to the apparatus and capable of providing the image;
an amplifier coupled to the image capture device, wherein the amplifier is adapted to apply at least two gain levels, each to a different region of the image; and
a processor coupled to the amplifier, wherein the processor is adapted to provide a signal to the amplifier to adjust the at least two gain levels and apply one gain level to a first portion of the image and a second gain level to a second portion of the image, and wherein the processor is capable of generating a gain map containing gain levels applied by the amplifier.
11. An apparatus for capturing an image, comprising
a camera, including:
a sensor configured to capture the image and generate a sensor output signal representing the captured image;
an amplifier coupled to receive the sensor output signal, wherein the amplifier has controls to apply multiple gain levels to the sensor output signal; and
a processor coupled to the camera, wherein the processor is configured to receive the sensor output signal, and wherein the processor is configured to provide a control signal to the amplifier to adjust the gain level applied by the amplifier so that the gain level of one portion of the captured image is different than the gain level of another portion of the captured image, and wherein the processor generates a gain map containing gain settings applied to the sensor output signal by the amplifier.
2. The camera of
3. The camera of
4. The camera of
5. The camera of
7. The camera of
8. The camera of
9. The camera of
10. The camera of
0. 12. The apparatus of
13. The apparatus of claim 12 11 wherein the gain map is a two dimensional array of gain settings, each gain setting indicating a particular gain applied by the amplifier to a region of the captured image.
14. The apparatus of
15. The apparatus of claim 12 11 wherein the camera further includes a register coupled to the processor and the amplifier.
16. The apparatus of
18. The method of
19. The method of
20. The method of
21. The method of
0. 24. The apparatus of
0. 25. The apparatus of
0. 26. The apparatus of
0. 27. The apparatus of
0. 28. The apparatus of
0. 29. The apparatus of
0. 30. The apparatus of
0. 31. The apparatus of
0. 32. The apparatus of
0. 34. The apparatus of
0. 35. The apparatus of
0. 36. The apparatus of
0. 37. The apparatus of
0. 38. The apparatus of
0. 40. The method of
0. 41. The method of
0. 42. The method of
0. 43. The method of
|
This is a continuation of application application Ser. No. 08/638/528, filed Apr. 26, 1996, U.S. Pat. No. 5,614,948. The present application Ser. No. 09/491,672, is a reissue of U.S. Pat. No. 5,712,682 having Ser. No. 08/770,386, which is a continuation of application Ser. No. 08/638,528, filed Apr. 26, 1996 (U.S. Pat. No. 5,614,948). The present application is also related to pending application No. 10/769,418, which is a continuation of the present application.
1. Field of the Invention
The present invention relates to image processing. More specifically, a system for capturing and processing video images.
2. Background
Digital cameras are used in a variety of applications requiring image capture and image processing. Many applications require cameras which are economical, yet generate a high quality video signal. Typical solid state sensors used in digital cameras have a dynamic range (or light intensity range) of 1000:1 or greater. However, many existing digital cameras utilize an inexpensive 8 bit or 6 bit analog-to-digital (A/D) converter to generate a digital output signal representing the captured image. The use of an 8 bit A/D converter limits the dynamic range of the output signal by providing a maximum of 256 possible luminance levels. Therefore, the dynamic range capabilities of the sensor (1000:1) are compressed to 256:1 by the A/D converter. This reduction in dynamic range results in “clipping” of the image; i.e., loss of image detail in bright areas and dark areas of the image. When clipping occurs, dark areas of the image become black (e.g., luminance level 0) and bright areas of the image become white (e.g., luminance level 255).
Existing cameras attempt to compensate for this dynamic range reduction by using an automatic gain control (AGC) amplifier having different gain settings. The gain setting of the AGC amplifier is determined based on the total luminance entering the camera. Since a single luminance level is determined, the selected gain setting is applied uniformly to the entire image.
An example of an existing camera is illustrated in
AGC amplifier 18 typically has several different gain settings which may be applied to the sensor output signal. The required gain setting for a particular image is selected based on the total light entering camera 10. If the total light level is low, the gain setting is increased. Similarly, if the total light level is high, the gain setting is decreased. The particular gain setting selected is applied uniformly to the entire image.
A/D converter 20 generates a digital video output signal on a signal line 22. As discussed above, the dynamic range of the digital video output signal is limited by the capacity of A/D converter 20.
Problems associated with dynamic range reduction may be solved by utilizing a 10 bit or 12 bit A/D converter to preserve the dynamic range of the sensor output signal. However, 10 bit and 12 bit A/D converters are expensive and substantially increase the cost of the camera. In an application requiring an inexpensive camera, the use of 10 bit or 12 bit AID converters is not practical.
It is therefore desirable to provide a low-Cost digital camera utilizing an inexpensive A/D converter, yet capable of generating a video signal containing image detail in bright areas and dark areas of the image.
The present invention provides a digital camera having an inexpensive A/D converter and including a processor for enhancing the dynamic range of the camera. The processor instructs a gain control amplifier to reduce the gain in bright areas of the image and increase the gain in dark areas of the image. These changes in gain settings for different portions of the image increase the image detail provided by the camera, thereby improving the overall image quality.
An embodiment of the present invention provides a sensor for capturing an image and generating a sensor output signal. A gain control amplifier is coupled to the sensor and receives the sensor output signal. The gain control amplifier has controls for applying various levels of gain to the sensor output signal. An analog-to-digital converter is coupled to the gain control amplifier and generates a digital output signal representing the captured image. A processor is coupled to the analog-to-digital converter and the gain control amplifier. The processor provides a control signal to the gain control amplifier for adjusting the level of gain applied by the amplifier.
Another feature of the present invention provides a gain map containing gain settings applied to the sensor output signal by the gain control amplifier. The gain map is continually updated by the processor to include changes in the captured image.
Other embodiments of the invention provide a register coupled to the processor and the gain control amplifier. The gain map containing gain settings is stored in the register and the gain control amplifier reads the gain settings from the register.
The present invention is illustrated by way of example in the following drawings in which like references indicate similar elements. The following drawings disclose various embodiments of the present invention for purposes of illustration only and are not intended to limit the scope of the invention.
The following detailed description sets forth numerous specific details to provide a thorough understanding of the invention. However, those skilled in the art will appreciate that the invention may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuits have not been described in detail so as not to obscure the invention.
The present invention provides a system for enhancing the dynamic range of a digital camera having an inexpensive A/D converter. An adaptive gain control mechanism is provided for adjusting gain settings applied to a captured image.
A first embodiment of the invention is illustrated in
As discussed above, A/D converter 108 may be a relatively inexpensive 6 bit or 8 bit A/D converter. To simplify the explanation of the invention, the specification describes camera 100 and its operation when using an 8 bit A/D converter 108.
A/D converter 108 converts the signal received from gain control amplifier 106 from an analog signal to a digital signal. The digital output of A/D converter 108 is provided on signal line 110 and represents an 8 bit digital video signal of the captured image. As noted above, the dynamic range of the signal on line 110 is limited by the 8 bit capacity of A/D converter 108.
The digital video signal on signal line 110 is also transmitted to a processor 114 using signal line 112. Processor 114 may be any type of processor capable of receiving a digital video signal and performing various operations or calculations related to the received signal. Processor 114 can be a dedicated image processing system or part of a system servicing other devices or performing other functions. In a preferred embodiment of the invention, processor 114 is a personal computer capable of performing a variety of operations and servicing numerous devices. In the preferred embodiment, the personal computer utilizes a Pentium® processor manufactured by Intel Corporation of Santa Clara, Calif.
The actual operations and processing steps performed by processor 114 may be implemented in software executed by processor 114. Those skilled in the art will appreciate that processor 114 may be a single processor device capable of processing digital data (e.g., a digital signal processor) or processor 114 may be a complete computer system containing a variety of devices and capable of performing multiple simultaneous operations. Thus, the invention is capable of being implemented on a wide variety of processing devices and computer platforms.
By way of example, the invention will be described in an implementation utilizing a personal computer as processor 114. Numerous type of computers may be used to practice the invention. The computer must be capable of receiving the digital video signal on signal line 112, performing the necessary operations and calculations, and transmitting a control signal to gain control amplifier 106 using a communication line 116. Line 116 functions as a control bus used to communicate control signals and data between processor 114 and gain control amplifier 106. In a specific embodiment of the invention, line 116 is a high-speed serial bus such as a Universal Serial Bus (USB). Although the invention will be described with reference to a high-speed serial bus, any communication line having sufficient bandwidth and low latency may be used to implement the invention.
In the embodiment of
At step 124 of
Referring to
Referring again to
Image 152 is divided into eight rows and eight columns, creating 64 image regions 154. Each image region 154 is associated with a particular gain setting contained in a corresponding cell of gain map 148 (FIG. 6A). Image region (2, 4) is a notation indicating the image region at the intersection of row 2 and column 4. The gain level applied to image region (2, 4) is stored in the gain map at cell (2, 4). Thus, each gain map cell provides a gain level to be applied to the corresponding image region. Although
At step 136 of
If image detail has been clipped from the image region at step 138 due to brightness, then the gain map setting for that region is reduced at step 140 to provide increased image detail; i.e., instead of producing all white regions, the gain setting is reduced to prevent clipping and provide enhanced image detail.
If significant bright portions are not identified at step 138, then step 142 determines whether the image region being analyzed contains significant dark portions; i.e., whether image detail has been lost or clipped by the 8 bit A/D converter. If image detail has been lost at step 142, then the gain map setting for that region is increased at step 140 to provide increased image detail. By increasing the gain setting, previously all black portions of the region may begin to show some image detail, thereby increasing the overall image quality.
If the image region being analyzed does not contain significant bright portions or significant dark portions, then-the routine continues to step 144 without updating the gain map setting for the particular image region. Step 144 determines whether the current row and column counters indicate the last row and column of the captured image. If all image regions have been analyzed, then the routine ends. Otherwise, the routine branches to step 146 where the row and/or column counters are incremented to select the next image region of the captured image. The routine then returns to step 136 to analyze the next image region. The image regions may be analyzed in any order. Row and column counters represent an example of a mechanism for analyzing each image region in a systematic manner.
When all regions of a captured image have been analyzed according to the procedure illustrated in
A particular example of the operations performed in
After capturing the first image, processor 114 divides the image into an array of image regions 154, as shown in FIG. 5. Each region 154 is analyzed by processor 114 to determine whether the gain setting for the region should be adjusted to provide increased image detail. Using
The gain settings illustrated in
The process of capturing an image, applying settings contained in a gain map to the image, and analyzing the digital video output signal to update gain settings is performed repeatedly by the system illustrated in FIG. 2. The gain settings contained in the gain map are updated in response to changes in the captured image; e.g., changes in light level, movement of objects in the image, and the like. Thus, the processor is repeatedly updating the gain map settings to enhance the overall image detail provided in the digital video output signal.
Those skilled in the art will appreciate that various methods may be used by processor 114 to analyze light intensity in each image region. An embodiment of the invention applies a histogram equalization algorithm to each image region. The histogram equalization algorithm generates a histogram of pixel luminance values in a particular region. The histogram is used to identify areas of high luminance and low luminance; i.e., areas requiring gain adjustments. Several exemplary histograms are illustrated in
In addition to the histogram equalization algorithm discussed above, various methods and algorithms for smoothing sharp edges and transitions between adjacent image regions will be known to those skilled in the art. These methods and algorithms may be utilized with the present invention to provide smoothing in transition areas and between adjacent image regions.
As noted above,
Referring to
The addition of register 160 to camera 156 increases the cost of the camera slightly, but eliminates the dependence of gain control amplifier 106 on communication line 116 (
Referring to
Referring to
From the above description and drawings, it will be understood by those skilled in the art that the particular embodiments shown and described are for purposes of illustration only and are not intended to limit the scope of the invention. Those skilled in the art will recognize that the invention may be embodied in other specific forms without departing from its spirit or essential characteristics. References to details of particular embodiments are not intended to limit the scope of the claims.
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
4962426, | Apr 07 1988 | Hitachi, Ltd. | Dynamic noise reduction circuit for image luminance signal |
5157493, | Sep 22 1989 | LABORATOIRE EUROPEEN DE RECHERCHES ELECTRONIQUES APPLIQUEES SOCIETE EN NOM COLLECTIF | Synch responsive agc utilizing A-D converter |
5282022, | Nov 17 1989 | Sanyo Electric Co., Ltd. | White balance adjusting apparatus for automatically adjusting white balance in response to luminance information signal and color information signal obtained from image sensing device |
5295001, | Nov 01 1990 | Canon Kabushiki Kaisha | Image sensing apparatus having tone control function |
5400074, | Mar 05 1992 | Sony Corporation | Video camera device having an exposure control circuit |
5402249, | Sep 24 1992 | Kabushiki Kaisha Toshiba | Integrated circuit with exterior gain control adjustment |
5420630, | Sep 11 1992 | Canon Kabushiki Kaisha | Image pickup apparatus performing white balance control based on data from regions of a frame |
5448306, | Jun 30 1992 | Canon Kabushiki Kaisha | Image processing apparatus with variable clamping |
5471241, | Mar 05 1993 | Canon Kabushiki Kaisha | Image sensing method and apparatus for producing a color video signal and luminance video signal |
5512948, | Aug 28 1990 | FUJIFILM Corporation | Negative-image signal processing apparatus |
6163342, | Jul 05 1994 | Canon Kabushiki Kaisha | Image sensing method and apparatus |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Jan 26 2000 | Intel Corporation | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Jul 22 2009 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Mar 10 2012 | 4 years fee payment window open |
Sep 10 2012 | 6 months grace period start (w surcharge) |
Mar 10 2013 | patent expiry (for year 4) |
Mar 10 2015 | 2 years to revive unintentionally abandoned end. (for year 4) |
Mar 10 2016 | 8 years fee payment window open |
Sep 10 2016 | 6 months grace period start (w surcharge) |
Mar 10 2017 | patent expiry (for year 8) |
Mar 10 2019 | 2 years to revive unintentionally abandoned end. (for year 8) |
Mar 10 2020 | 12 years fee payment window open |
Sep 10 2020 | 6 months grace period start (w surcharge) |
Mar 10 2021 | patent expiry (for year 12) |
Mar 10 2023 | 2 years to revive unintentionally abandoned end. (for year 12) |