An image capturing device and a method thereof are provided. The image capturing device includes an image sensor and a processor. The image sensor includes a color filter array and a sensor chip. The color filter array includes a first region permitting visible light and infrared light to pass and a second region permitting the infrared light to pass. The sensor chip obtains a target image through the color filter array, wherein the target image includes a composite image corresponding to the first region and a first group infrared image corresponding to the second region, and the composite image includes a second group infrared image, where the second group infrared image and the first group infrared image are linear correlated. The processor adjusts a proportion of the second group infrared image in the composite image by employing the first group infrared image, so as to generate a processed image.
|
1. An image capturing device, comprising:
an image sensor, configured to obtain a target image, wherein the image sensor comprises:
a color filter array, comprising a first region permitting visible light and infrared light to pass and a second region permitting the infrared light to pass; and
a sensor chip, obtaining the target image by the color filter array, wherein the target image comprises:
a composite image corresponding to the first region;
a first group infrared image corresponding to the second region, wherein the composite image is composed of a rgb image and the second group infrared image; and
a processor, generating a processed image by employing the first group infrared image,
wherein under a normal mode, the processor retrieves the rgb image from the composite image by employing the first group infrared image and sets the retrieved rgb image as the processed image, and
wherein under a glimmer mode, the processor adjusts the weight of the second group infrared image in the composite image by employing the first group infrared image to generate the processed image.
8. An image capturing method, applicable to an image capturing device, the image capturing method comprising:
obtaining a target image by the image capturing device, wherein the step of obtaining the target image comprises:
receiving light by a color filter array, so that visible light and infrared light pass through a first region of the color filter array and the infrared light passes through a second region of the color filter array; and
generating a composite image corresponding to the first region and a first group infrared image corresponding to the second region by a sensor chip, wherein the composite image is composed of a rgb image and the second group infrared image; and
generating a processed image by employing the first group infrared image,
wherein under a normal mode, the processor retrieves the rgb image from the composite image by employing the first group infrared image and sets the retrieved rgb image as the processed image, and
wherein under a glimmer mode, the processor adjusts the weight of the second group infrared image in the composite image by employing the first group infrared image to generate the processed image.
2. The image capturing device according to
3. The image capturing device according to
4. The image capturing device according to
a pixel sensor array, wherein the pixel sensor array is correspondingly disposed at a first side of the color filter array,
wherein the color filter array comprises a plurality of filter blocks, and each of the filter blocks comprises a plurality of filtering units, the filtering units are arranged in a repeating pattern and respectively correspond to one of pixel sensors in the pixel sensor array in space, and each of the filtering units permits light with at least one frequency band to pass, so that the pixel sensor array senses the target image under a light intensity with the at least one frequency band.
5. The image capturing device according to
6. The image capturing device according to
a lens, correspondingly disposed at a second side of the color filter array; and
an auto focus module, adjusting a focus distance of the lens relative to the sensor chip,
wherein the processor calculates the focus distance through the first group infrared image.
7. The image capturing device according to
a dual-band filter, configured to filter out light, so that the light with wavelengths located at a first frequency band and/or a second frequency band pass through the color filter array of the image capturing device, wherein the first frequency band comprises a wavelength range permitting the visible light to pass, and the second frequency band comprises a wavelength range permitting the infrared light to pass.
9. The image capturing method according to
progressively increasing the weight of the second group infrared image in the composite image as an ambient brightness gradually decreases.
10. The image capturing method according to
determining whether an exposure time of the image capturing device reaches a first predetermined value, and determining whether a signal gain value of the image capturing device reaches a second predetermined value; and
linearly setting an infrared adjustment factor and thereby adjusting the weight of the second group infrared image in the composite image according to an ambient brightness by employing the first group infrared image to generate the processed image.
11. The image capturing method according to
calculating a focus distance by the first group infrared image; and
adjusting the focus distance of a lens relative to the sensor chip in the image capturing device, wherein the lens is correspondingly disposed at a second side of the color filter array.
12. The image capturing method according to
correspondingly disposing a pixel sensor array at a first side of the color filter array, wherein the color filter array comprises a plurality of filter blocks, and each of the filter blocks comprises a plurality of filtering units;
arranging the filtering units in a repeating pattern, and respectively corresponding each of the filtering units to one of pixel sensors in the pixel sensor array in space, wherein each of the filtering units permits the light with at least one frequency band to pass; and
sensing the target image by the pixel sensor array under a light intensity with the at least one frequency band.
13. The image capturing method according to
filtering the light by a dual-band filter, so that the light with wavelengths located at a first frequency band and/or a second frequency band pass through the color filter array of the image capturing device, wherein the first frequency band comprises a wavelength range permitting the visible light to pass, and the second frequency band comprises a wavelength range permitting the infrared light to pass.
|
This application claims the priority benefits of U.S. provisional application Ser. No. 61/588,667, filed on Jan. 20, 2012. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.
The invention relates to an image capturing device and a method thereof, and more particularly, to an image capturing device capable of adjusting a content of an infrared (IR) image signal and a method thereof.
In recent years, the development of digital cameras has been progressed rapidly, so that the digital cameras have become one of the most indispensable parts of our daily life. Therefore, enhancing the image quality has always been one of the essential topics in the industry fields. In brief, photography is utilized light to irradiate on an object, and after the light is reflected and passed though a lens of a digital camera, an image processing is subsequently performed inside the digital camera. However, the light can further include a non-visible light such as an infrared (IR) light or an ultraviolet light besides the visible light. The current techniques are mostly employed optical filters to filter out the infrared light or adjust the light entering the inside of the digital camera.
Generally speaking, the infrared cut filter 20 has to be used to filter out the infrared light when a sufficient light source is provided. If the image sensor 10 receives an infrared signal, the image processing such as performing color calculation and automatic white balance (AWB) can be easily interfered by the infrared signal and the errors are caused in the calculation, so that there is a great difference between the color appeared form the image and the color seen by human eyes. However, the image sensor 10 can only increase the brightness through a flash lamp when an insufficient light source is provided, but the effect from the flash lamp tends to cause an unnatural feeling to the user with regard to the image quality.
Accordingly, the invention is directed to an image capturing device, which is capable of adjusting a proportion of an infrared image in an outputting composite image by a single image sensor.
The invention is directed to an image capturing method, which is cooperated with an auto exposure control to set an infrared adjustment factor, and is capable of adjusting a content of an infrared signal in an output image.
The invention provides an image capturing device including an image sensor and a processor, in which the image sensor includes a color filter array and a sensor chip. The color filter array includes a first region which permits visible light and infrared light to pass and a second region which permits the infrared light to pass. The sensor chip obtains a target image through the color filter array, in which the target image includes a composite image corresponding to the first region and a first group infrared image corresponding to the second region, in which the composite image includes a second group infrared image, and the second group infrared image and the first group infrared image are linear correlated. The processor adjusts a proportion of the second group infrared image in the composite image by employing the first group infrared image, so as to generate a processed image.
The invention provides an image capturing method which is applicable to an image capturing device. The image capturing method includes the following steps. A target image is obtained by the image capturing device, and a proportion of a second group infrared image in a composite image is adjusted by employing a first group infrared image so as to generate a processed image, in which the step of obtaining the target image includes: receiving light by a color filter array, so that visible light and infrared light pass through a first region of the color filter array and the infrared light passes through a second region of the color filter array; and generating a composite image corresponding to the first region and a first group infrared image corresponding to the second region by a sensor chip, in which the composite image includes a second group infrared image, and the second group infrared image and the first group infrared image are linear correlated.
Based on the above, in the image capturing device and the method thereof provided by the invention, by removing the infrared cut filter and cooperating with the dyestuffs that can sense the infrared light in the color filter array, the infrared signal is obtained. Moreover, by cooperating with the auto exposure control, the infrared adjustment factor is set, and the proportion of the infrared image in the outputting composite image can be adjusted.
In order to make the aforementioned features and advantages of the invention more comprehensible, embodiments accompanying figures are described in detail below.
The accompanying drawings are included to provide a further understanding of the present invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention.
The invention is to prevent the need to consider whether to use an infrared cut filter when a user uses a digital camera to capture an image depending upon the brightness in an environment. The invention provides an image capturing device, which can adjust a proportion of an infrared image in an output image at any time according to an ambient brightness, under the conditions that no infrared cut filter is used and only a single image sensor is employed. Below, exemplary embodiments will be described in detail with reference to accompanying drawings so as to be easily realized by a person having ordinary knowledge in the art. The inventive concept may be embodied in various forms without being limited to the exemplary embodiments set forth herein.
More specifically, in the present embodiment, the image sensor 210 includes a lens 212, a color filter array 214 and a sensor chip 216, for instance. The sensor chip 216 further includes a pixel sensor array 218. In the present embodiment, the lens 212 and the pixel sensor array 218 are disposed at two sides of the color filter array 214, respectively. First, the light incident through the lens 212 is filtered by the color filter array 214, so that the specific visible light and infrared light can pass. Then, the sensor chip 216 senses a light intensity by the pixel sensor array 218, and obtains a target image after converting an analog image signal into a digital image signal. The processor 220 further performes a digital image processing on the target image so as to obtain a processed image, in which the color filter array 214 includes a plurality of filter blocks, and arrangements thereof will be further described as below.
The processor 220 can be a central processing unit (CPU), or other programmable microprocessors with general purpose or special purpose, a digital signal processor (DSP), a programmable controller, an application specific integrated circuit (ASIC), or other similar devices, or a combination thereof.
First, in step S310, a target image is obtained by an image capturing device, in which step S310 can be further divided into two sub-steps S312 and S314. Specifically, the method of obtaining the target image includes the following steps. First, light is received by a color filter array, so that the visible light and the infrared light pass through a first region of the color filter array 214, and the infrared light passes through a second region of the color filter array 214. In order to illustrate the difference between the first region and the second region of the color filter array 214, the operation of the image capturing device 200 will be described with reference to
The filter block SP1 includes a first filtering unit P1, a second filtering unit P2, a third filtering unit P3 and a fourth filtering unit P4. Each of the filtering units permits the light having various frequency bands to pass, respectively, and is arranged in a repeating pattern. Moreover, each of the filtering units respectively corresponds to one of the pixel sensors in the pixel sensor array in space.
Specifically, for instance, the first filtering unit P1 includes a first dyestuff capable of simultaneously sensing the blue light and the infrared light, so that the blue light and the infrared light enter to the corresponding pixel sensors, thereby sensing and obtaining a first image signal (B+IR). The second filtering unit P2 includes a second dyestuff capable of simultaneously sensing the green light and the infrared light, so that the green light and the infrared light enter to the corresponding pixel sensors, thereby sensing and obtaining a second image signal (G+IR). The third filtering unit P3 includes a third dyestuff capable of simultaneously sensing the red light and the infrared light, so that the red light and the infrared light enter to the corresponding pixel sensors, thereby sensing and obtaining a third image signal (R+IR). The fourth filtering unit P4 includes a fourth dyestuff configured specifically to sense the infrared light, so that the infrared light enters to the corresponding pixel sensor, thereby sensing and obtaining a fourth image signal IR.
Based on the above, in the exemplary embodiment, the first region of the color filter array 214 includes the regions corresponding to the first filtering unit P1, the second filtering unit P2 and the third filtering unit P3 in each of the filter blocks. The second region of the color filter array 214 includes the regions corresponding to the fourth filtering unit P4 in each of the filter blocks. Thus, the composite image includes the first image signal (B+IR), the second image signal (G+IR) and the third image signal (R+IR) in each of filter blocks, and the first group infrared image includes the fourth image signal IR in each of filter blocks. It should be noted that, the second group infrared image in the composite image and the first group infrared image appear to be linear correlated.
Back to
Next, in step S320, the processor 220 employs the first group infrared image to adjust the proportion of the second group infrared image in the composite image, so as to generate the processed image. In brief, the composite image is composed of a red green blue (RGB) image and the second group infrared image. The second group infrared image and the first group infrared image are linear correlated, and accordingly, the processor 220 can adjust the proportion of the second group infrared image in the composite image.
For example, under the normal mode with a sufficient ambient light source, the infrared image causes a great difference between the color appeared form the image and the color seen by human eyes. Therefore, the processor 220 can employ the first group infrared image to retrieve the RGB image from the composite image so as to serve as the processed image. By contrast, under the glimmer mode with an insufficient ambient light source, the image brightness is increased by the infrared image so as to enhance the image quality. Thus, the processor 220 employs the first group infrared image to adjust the proportion of the second group infrared image in the composite image so as to generate the processed image. More specifically, the processor 220 can progressively increase the proportion of the second group infrared image in the composite image as the ambient brightness gradually decreases.
Accordingly, in the invention, the light having various frequency bands can be controlled by the arrangement of each filtering unit in the color filter array 214, such that the image capturing device of the invention can adjust the proportion of the infrared image in the output image according to the variation of the ambient light source (brightness). Additionally, it should be noted that, in the exemplary embodiment, the arrangement for each of the filtering units depicted in
The description that the processor employs the first group infrared image to adjust the proportion of the second infrared image in the composite image so as to generate an adequate processed image, which is further explained below.
Here, the relationships of the exposure time and the signal gain value of the image capturing device are illustrated firstly.
Next, referring to
Moreover, for example, the invention can adjust the proportion of the second group infrared image in the composite image by setting the infrared adjustment factor k. Referring to
The processor can adjust to generate the processed image ImgR,G,B(k) according to the following formula:
ImgR,G,B(k)=R+IR(k)+G+IR(k)+B+IR(k), where 0≦k≦1.
With reference to
Based on the above, in the invention, the proportion of the second group infrared image in the composite image can be adjusted by only implementing the single image sensor.
The description that the second linear function IR2 is employed to adjust the infrared adjustment factor k in another embodiment, which is further explained below.
When the color filter array 214 depicted in
ImgB,G,R,IR(k)=R+G+B+IR(k), where 0≦k≦1.
Similarly, the processor can set the infrared adjustment factor k according to the exposure time and the signal gain value of the image sensor. In other words, the processor is applicable to the second linear function IR2 depicted in
The image capturing device of the invention can be implemented as follow.
Referring to
Referring to
Based on the above, in the invention, the RGB image and the infrared image can be simultaneously outputted by only implementing the single image sensor. In contrast thereto, conventionally, it is required an image sensor to capture the RGB image and then another infrared sensor to capture the infrared image. Accordingly, the invention is capable of saving cost.
It should be mentioned that, although the first image signal (B+IR), the second image signal (G+IR) and the third image signal (R+IR) in the filtering units of the color filter array 214 depicted in
In order to prevent noise to interfere with each of the image signals sensed by the color filter array 214, the dual-band filter 830 is utilized to restrict the light which have wavelengths located at a first frequency band and a second frequency band to pass, and to filter out the light which have wavelengths located outside the first frequency band and the second frequency band.
In view of the foregoing, in the invention, the infrared image signal can be obtained by removing the infrared cut filter and cooperating with the dyestuffs capable of sensing the infrared light in the color filter array, unlike the conventional technique has to determine whether to use the infrared cut filter due to the impact of ambient brightness. Moreover, by cooperating with the auto exposure control, the infrared adjustment factor can be set, and the proportion of the infrared image in the composite image can be adjusted. By this way, only a single image sensor is employed in the invention, which can be applied to the shooting scene, the auto focus operation or the smart sensory game etc. with the auxiliary infrared light source, thereby saving cost and enhancing the image quality.
It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the disclosed embodiments without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the disclosure cover modifications and variations of this disclosure provided they fall within the scope of the following claims and their equivalents.
Patent | Priority | Assignee | Title |
10692192, | Oct 21 2014 | CONNAUGHT ELECTRONICS LTD | Method for providing image data from a camera system, camera system and motor vehicle |
Patent | Priority | Assignee | Title |
7728316, | Sep 30 2005 | Apple Inc | Integrated proximity sensor and light sensor |
7812827, | Jan 03 2007 | Apple Inc | Simultaneous sensing arrangement |
7916941, | Aug 29 2003 | Apple Inc. | Methods and apparatuses for restoring color and enhancing electronic images |
8008613, | May 05 2009 | Apple Inc. | Light sensing device having a color sensor and a clear sensor for infrared rejection |
8217336, | May 05 2009 | Apple Inc. | Light sensing device having a color sensor and a clear sensor for infrared rejection |
20050285966, | |||
20070279514, | |||
20080297487, | |||
20090273679, | |||
20100103268, | |||
20100289885, | |||
20110074931, | |||
20110090351, | |||
20110164146, | |||
20110181955, | |||
20110235017, | |||
20120026368, | |||
CN102075683, | |||
CN1310915, | |||
CN1794783, | |||
WO2010141772, | |||
WO2011131758, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Nov 29 2012 | HTC Corporation | (assignment on the face of the patent) | / | |||
Dec 03 2012 | LIN, CHUN-TA | HTC Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029662 | /0126 |
Date | Maintenance Fee Events |
Mar 07 2019 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Mar 08 2023 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Date | Maintenance Schedule |
Sep 22 2018 | 4 years fee payment window open |
Mar 22 2019 | 6 months grace period start (w surcharge) |
Sep 22 2019 | patent expiry (for year 4) |
Sep 22 2021 | 2 years to revive unintentionally abandoned end. (for year 4) |
Sep 22 2022 | 8 years fee payment window open |
Mar 22 2023 | 6 months grace period start (w surcharge) |
Sep 22 2023 | patent expiry (for year 8) |
Sep 22 2025 | 2 years to revive unintentionally abandoned end. (for year 8) |
Sep 22 2026 | 12 years fee payment window open |
Mar 22 2027 | 6 months grace period start (w surcharge) |
Sep 22 2027 | patent expiry (for year 12) |
Sep 22 2029 | 2 years to revive unintentionally abandoned end. (for year 12) |