The method and apparatus for processing pixel output signals from column lines in an imager having an array of pixels in rows and columns uses a sigma-delta type analog-to-digital converter to convert the output signals on each column line to digital signals and feeding them to a digital signal processor. The converter is monitored to stop sampling of a pixel with the detection of pixel saturation, which is carried out by counting a predetermined number of consecutive zeros in the converted signal. In addition, the next pixel in a column may be controlled to be read with the saturation of the previous pixel, and the next row of pixels may be controlled to be read with the saturation of the pixels in the previous row. Further, sets of a predetermined number of converter output samples are condensed by a decimator into binary numbers of predetermined bit length. The outputs of the decimators may be fed directly to the digital signal processor or they may be multiplexed to provide one or more inputs to the digital signal processor.
|
0. 29. An apparatus comprising:
a sigma-delta type analog-to-digital converter configured to convert pixel output signals to digital signals comprising mainly logical ones for a high input signal level and mainly zeros for a low input signal level, and to stop sampling a pixel in response to detected pixel saturation; and
a saturation counter configured to count consecutive zeros in the digital signals and to detect pixel saturation in response to obtaining a predetermined count associated with pixel saturation.
0. 25. In an imager having an array of pixels in rows and columns with column lines for pixel output signals, an apparatus comprising:
a sigma-delta type analog-to-digital converter configured to convert the pixel output signals on each column line to digital signals, wherein the sigma-delta type converter outputs mainly logical ones for a high input signal level and outputs mainly zeros for a low input signal level;
means for detecting pixel saturation; and
means for stopping sampling of a pixel in response to detecting pixel saturation;
wherein the detecting means comprises a first counter provided for each column for counting consecutive zeros in the digital signals to a predetermined number to detect pixel saturation.
0. 21. In an imager having an array of pixels in rows and columns with column lines for pixel output signals, a method comprising:
processing the pixel output signals on each column line using a sigma-delta type analog-to-digital converter, the sigma-delta type converter outputting digital signals comprising mainly logical ones for a high input signal level and comprising mainly zeros for a low input signal level; and
monitoring the sigma-delta type converter to stop sampling of a pixel in response to detecting pixel saturation;
wherein the monitoring step comprises, for each column:
at a first counter, counting consecutive zeros in the digital signals to a predetermined number; and
detecting pixel saturation in response to counting to the predetermined number at the first counter.
9. In an imager having an array of pixels in rows and columns with column lines for pixel output signals, apparatus for processing the pixel output signals comprising:
a sigma-delta type analog-to-digital converter for converting the pixel output signals on each column line to digital signals, the sigma-delta type converter outputting mainly logical ones for a high input signal level and outputting mainly zeros for a low input signal level;
means for feeding the digital signals to a digital signal processor;
means for detecting pixel saturation; and
means for stopping sampling of a pixel when the pixel saturation is detected,
wherein the detecting means comprises:
a first counter provided for each column for counting consecutive zeros in the digital signals to a predetermined number to determine when the pixel saturation has occurred.
1. In an imager having an array of pixels in rows and columns with column lines for pixel output signals, a method of processing the pixel output signals comprising:
converting the pixel output signals on each column line to digital signals using a sigma-delta type analog-to-digital converter, the sigma-delta type converter outputting mainly logical ones for a high input signal level and outputting mainly zeros for a low input signal level;
feeding the digital signals to a digital signal processor; and
monitoring the sigma-delta type converter to stop sampling of a pixel when pixel saturation is detected,
wherein the monitoring step comprises, for each column:
at a first counter, counting consecutive zeros in the digital signals to a predetermined number, and
detecting the pixel saturation in response to counting to the predetermined number at the first counter.
0. 38. An apparatus comprising:
an array of pixels in rows and columns;
a plurality of column lines to carry pixel output signals;
one or more sigma-delta type analog-to-digital converters, each of the one more sigma-delta analog-to-digital converters configured to convert pixel output signals to digital signals comprising mainly logical ones for a high input signal level and mainly zeros for a low input signal level, and to stop converting pixel output signals of a pixel in response to detected saturation of the pixel;
column output circuitry configured to receive the digital signals and to feed output signals to a digital signal processor; and
a plurality of saturation counters configured to detect pixel saturation, each saturation counter configured to count consecutive zeros in the digital signals of a corresponding column line, and to detect saturation in response to reaching a predetermined count.
2. A method of processing the pixel output signals as claimed in
sending the sigma-delta type converter output to the digital signal processor when the pixel saturation is detected.
3. A method of processing the pixel output signals as claimed in
reading the next pixel in a column when the previous pixel is saturated.
4. A method of processing the pixel output signals as claimed in
reading the next row of pixels when the pixels in the previous row are all saturated.
5. A method of processing the pixel output signals as claimed in
condensing sets of a predetermined number of sigma-delta type converter output samples in a decimator to binary numbers of predetermined bit length.
6. A method of processing the pixel output signals as claimed in
determining the number of sigma-delta type converter output samples and outputting the determined number when the value of the first counter reaches the predetermined number; and
feeding the determined number from the first counter to the decimator.
7. A method of processing the pixel output signals as claimed in
determining the number of sigma-delta type converter output samples and outputting the determined number when the value of the first counter reaches the predetermined number; and
feeding the determined number from the first counter to the digital signal processor.
8. A method of processing the pixel output signals as claimed in
multiplexing the outputs of the decimators to provide one or more inputs to the digital signal processor.
10. Apparatus for processing the pixel output signals as claimed in
means for sending the sigma-delta type converter output to the digital signal processor when the pixel saturation is detected.
11. Apparatus for processing the pixel output signals as claimed in
a decimator provided for each column for condensing sets of a predetermined number of sigma-delta type converter output samples to binary numbers of predetermined bit length.
12. Apparatus for processing the pixel output signals as claimed in
means provided for each column for determining the number of sigma-delta type converter output samples and outputting the determined number when the value of the first counter reaches the predetermined number; and
means for feeding the determined number from the number determining means to the corresponding decimator.
13. Apparatus for processing the pixel output signals as claimed in
14. Apparatus for processing the pixel output signals as claimed in
means provided for each column for determining the number of sigma-delta type converter output samples and outputting the determined number when the value of the first counter reaches the predetermined number; and
means for feeding the determined number from the number determining means to the digital signal processor.
15. Apparatus for processing the pixel output signals as claimed in
16. Apparatus for processing the pixel output signals as claimed in
means for multiplexing the outputs of the decimators to provide one or more inputs to the digital signal processor.
17. Apparatus for processing the pixel output signals as claimed in
18. Apparatus for processing the pixel output signals as claimed in
19. Apparatus for processing the pixel output signals as claimed in
means for reading the next pixel in a column when the previous pixel is saturated.
20. Apparatus for processing the pixel output signals as claimed in
means for reading the next row of pixels when the value of the first counter reaches the predetermined number.
0. 22. The method of claim 21 further comprising reading a next pixel in a column in response to detecting that a previous pixel is saturated.
0. 23. The method of claims 21 further comprising reading a next row of pixels in response to detecting pixels of a previous row are saturated.
0. 24. The method of claim 21 further comprising, for each column,
condensing sets of a predetermined number of sigma-delta type converter output samples in a decimator to binary numbers of predetermined bit length;
determining the number of sigma-delta type converter output samples and outputting the determined number when the value of the first counter reaches the predetermined number; and
feeding the determined number from the first counter to the decimator.
0. 26. The apparatus of claim 25 further comprising a decimator provided for each column, wherein the decimator is configured to condense sets of a predetermined number of sigma-delta type converter output samples to binary numbers of predetermined bit length.
0. 27. The apparatus of claim 26 further comprising:
means provided for each column for determining the number of sigma-delta type converter output samples and outputting the determined number when the value of the first counter reaches the predetermined number; and
means for feeding the determined number from the number determining means to the corresponding decimator.
0. 28. The apparatus of claim 27 wherein the number determining means comprises a second counter, and wherein when the second counter outputs the determined number to the corresponding decimator, the corresponding decimator outputs the binary numbers and the second counter resets to zero.
0. 30. The apparatus of claim 29 further comprising:
a digital signal processor configured to determine pixel values based upon output signals; and
output circuitry configured to receive the digital signals and to provide output signals to the digital signal processor.
0. 31. The apparatus of claim 29 further comprising:
a digital signal processor configured to determine pixel values based upon binary numbers of a predetermined bit length; and
output circuitry configured to receive the digital signals and to provide output signals comprising binary numbers of the predetermined bit length to the digital signal processor, the output circuitry comprising one or more decimators configured to condense sets of a predetermined number of sigma-delta type converter output samples of the digital signals to binary numbers of the predetermined bit length.
0. 32. The apparatus of claim 31 wherein the output circuitry further comprises one or more decimator counters configured to feed the one or more decimators with the predetermined number of sigma-delta type converter output samples in response to the saturation counter reaching the predetermined count.
0. 33. The apparatus of claim 32, wherein the output circuitry further comprises one or more multiplexers configured to receive the binary numbers of the predetermined length from the one or more decimators and to provide to the digital signal processor with the binary numbers of the predetermined length.
0. 34. The apparatus of claim 29 further comprising
a digital signal processor configured to determine pixel values based upon a predetermined number of sigma-delta type output samples of the digital signals; and
output circuitry configured to provide the predetermined number of sigma-delta type output samples to the digital signal processor, the output circuitry comprising one or more decimator counters configured to feed the digital signal processor with the predetermined number of sigma-delta type output samples in response to the saturation counter reaching the predetermined count.
0. 35. The apparatus of claim 34, wherein the output circuitry further comprises one or more multiplexers configured to receive the predetermined number of sigma-delta type output samples and to provide the predetermined number of sigma-delta type output samples to the digital signal processor.
0. 36. The apparatus of claim 29, wherein in response to detecting pixel saturation, the saturation counter provides a feedback that results in the sigma-delta type analog-to-digital converter reading a next pixel in a column of pixels.
0. 37. The apparatus of claim 29, further comprising
a plurality of sigma-delta type analog-to-digital converters, each sigma-delta type analog-to-digital converter configured to convert pixel output signals for a column of pixels to digital signals comprising mainly logical ones for a high input signal level and mainly zeros for a low input signal level, and to stop sampling a pixel in response to detected pixel saturation;
a plurality of saturation counters, each saturation counter configured to count consecutive zeros in the digital signals and to detect pixel saturation in response to reaching the predetermined count associated with pixel saturation;
row enable circuitry to select a next row of pixels for reading by the plurality of sigma-delta type analog-to-digital converters in response to a result that indicates that each of the plurality of saturation counters detected pixel saturation; and
a comparator to receive outputs from each of the plurality of saturation counters and to provide the row enable circuitry with the result in response to determining based upon the received outputs that each of the plurality of saturation counters detected pixel saturation.
0. 39. The apparatus of claim 38 further comprising a digital signal processor configured to determine pixel values based upon the output signals of the column output circuitry.
0. 40. The apparatus of claim 39 further comprising a digital signal processor configured to determine pixel values based upon binary numbers of a predetermined bit length, wherein the column output circuitry is configured to provide output signals comprising binary numbers of the predetermined bit length to the digital signal processor, the column output circuitry comprising a plurality of decimators configured to condense sets of a predetermined number of sigma-delta type converter output samples of the digital signals to binary numbers of the predetermined bit length.
0. 41. The apparatus of claim 40 wherein the column output circuitry further comprises a plurality decimation counters configured to feed a corresponding decimator of the plurality of decimators with the predetermined number of sigma-delta type converter output samples in response to corresponding saturation counters reaching the predetermined number.
0. 42. The apparatus of claim 41, wherein the column output circuitry further comprises a multiplexer configured to receive outputs from the plurality of decimators and to provide the outputs of the plurality of decimators to the digital signal processor.
0. 43. The apparatus of claim 39, wherein in response to detecting pixel saturation, a saturation counter of the plurality of saturation counters provides a feedback that results in a corresponding sigma-delta type analog-to-digital converter reading a next pixel in a column of pixels.
0. 44. The apparatus of claim 39, further comprising
row enable circuitry configured to select a next row of pixels for reading by the plurality of sigma-delta type analog-to-digital converters in response to a result that indicates that each of the plurality of saturation counters detected pixel saturation; and
a comparator configured to receive outputs from each of the plurality of saturation counters and to provide the row enable circuitry with the result in response to determining based upon the received outputs that each of the plurality of saturation counters detected pixel saturation.
|
This application is a 371 application of PCT application PCT/CA01/01773 filed Dec. 13, 2001, which claims the benefit of U.S. provisional application 60/256,336 filed Dec. 19, 2000. This application was published in English on Jun. 27, 2002 as International Publication Number WO 02/51128 A2.
The invention relates generally to CMOS imagers, and more particularly to the processing of column level signals in an imager.
Currently, CMOS imagers are well known in the art and are implemented in many different applications. One of the primary areas that CMOS imagers can be found is in portable applications such as digital cameras that use battery power. It is therefore very desirable to have a CMOS imager that is powered with a low supply voltage, for example one volt (1V).
Having a voltage this low, however, leads to a number of problems. For one, the quantization values of an imager become very small. For example, a typical CMOS imager might use 8 bits of resolution and have a supply voltage of 3.3 volts. This leads to a quantization level of 3.3V/256=12.9 mV. If an imager is implemented with a one volt supply and 8 bits of resolution then the quantization level is 1V/256=3.9 mV. The proportion of noise relative to this low quantization value becomes very large and has a greater impact on the output.
Another problem involves the pixel structure that is used in most CMOS imagers, namely using 3 n-type MOS transistors with a photodiode. The pixel consists of a precharge transistor, a source follower amplifying transistor and an access transistor. When the pixel is precharged to the supply voltage, one threshold voltage, Vt, is lost through the precharge transistor. Once the pixel is exposed to light and the output is being read out to the column, another Vt is lost through the source follower amplifying transistor. A typical Vt for MOS transistors is 0.07 V, leaving a maximum voltage output to the column of 0.86V (1V-2*0.07V), further limiting the quantization levels.
Quantizing the output from the pixels into digital data for the Digital Signal Processor (DSP) may be carried out in a number of known ways. Correlated Double Sampling (CDS) quantizes the output of the column signal twice, once at the beginning of the integration period and once at the end. From these two samples the DSP calculates the slope of the pixel output to determine the brightness of the pixel. This method is highly susceptible to noise and as a result of the high noise to quantization level ratio can be very inaccurate in a low voltage CMOS imager. Also, by using only two points to calculate the slope of the pixel output there is no accounting for a saturated pixel. Any pixels that saturate at different times in the integration period are all calculated as having the same slope, and thus all those pixels are considered to have the same value.
Successive approximation is another method of converting the analog output from the pixel into a digital signal. In this system the output at the end of the integration period is successively compared to different threshold levels, each representing one bit of resolution of the final output. The thresholds are created through capacitors that must be sized in precise ratios (C, ½C, ¼C, etc.) which is both difficult to implement and also occupies a large amount of space. Also, the capacitors on every column across the array must be matched with one another for consistent results, which is not always attainable due to process impurities. The problem of noise is a large factor in successive approximation, as both the small quantization levels to noise ratio as well as capacitance noise, which is larger in small capacitors, will accumulate to yield inaccurate results. Successive approximation also does not directly account for the problem of threshold voltage loss through the pixel transistors. Finally, successive approximation takes a single sample at the end of the integration period and therefore, has no way of differentiating between pixels that saturate at different times throughout the integration period.
Therefore, there is a need for an efficient method and apparatus that is suited to low voltage imagers for processing the pixel output signals at the column level.
The invention is directed to a method and apparatus for processing pixel output signals from column lines in an imager having an array of pixels in rows and columns. The output signals on each column line are converted to digital signals using a sigma-delta type analog-to-digital converter and the digital signals are fed to a digital signal processor.
In accordance with another aspect of the invention, the sigma-delta type converter is monitored to stop sampling of a pixel with the detection of pixel saturation. With the sigma-delta type converter providing an output of mainly logical ones for a high input signal level and an output of mainly zeros for a low input signal level, detecting pixel saturation is carried out by counting a predetermined number of consecutive zeros.
In accordance with a further aspect of the invention, the sigma-delta type converter output is sent to the digital signal processor with the detection of the saturation of a pixel.
In accordance with another aspect of the invention, sets of a predetermined number of sigma-delta type converter output samples are condensed by a decimator into binary numbers of predetermined bit length. In addition, the number of sigma-delta type converter output samples in a set wherein the pixel has saturated is determined using a further counter and this number is fed to the decimator. Alternately, the determined number may be fed to the digital signal processor.
In accordance with yet another aspect of this invention, the next pixel in a column is controlled to be read with the saturation of the previous pixel, and the next row of pixels may be controlled to be read with the saturation of the pixels in the previous row.
In accordance with a further aspect of this invention, the outputs of the decimators may be multiplexed to provide one or more inputs to the digital signal processor. A single multiplexer may be used providing one input line to the digital signal processor, or a number of multiplexers may be used providing a number of input lines to the digital signal processor.
Other aspects and advantages of the invention, as well as the structure and operation of various embodiments of the invention, will become apparent to those ordinarily skilled in the art upon review of the following description of the invention in conjunction with the accompanying drawings.
The invention will be described with reference to the accompanying drawings, wherein:
The column level sigma-delta analog-to-digital converter 208 is able to solve the various problems associated with having a low voltage CMOS imager 200. The sigma-delta converter 208 overcomes the problems of noise through oversampling the output of the pixels 202. More samples yields more accurate results through line-of-best-fit approximations on the output of the converter 208. The line-of-best-fit approximation also allows the sigma-delta converter 208 to indirectly account for the threshold voltage Vt drops across the transistors in the pixel.
In accordance with the present invention the sigma-delta converter 208 is capable of accounting for saturating pixels 202. The integration time tint of the imager 200 corresponds to a set number of outputs from the sigma-delta converter 208 depending on the sampling frequency. If a pixel 202 does not saturate, then the sigma-delta converter 208 will output the set number of samples and the digital signal processor (DSP) will calculate the correct output. In order to account for a pixel 202 that saturates, the output of the sigma-delta converter 208 is monitored and will stop sampling when pixel 202 saturation has been detected. The DSP will then calculate the line-of-best-fit knowing that it has received less values and correspondingly calculates the result.
The invention may be implemented using column level sigma-delta converters 308 having inputs connected to the analog column output lines 304 for pixels 302 with each column level sigma-delta converter 308 having a simple counter 309 associated with it, as shown in
There are a number of benefits that are created as a result of implementing this type of column level sigma-delta converter 308. The first and most important of which is that it compensates for the various problems associated with having a low supply voltage Vsupply. The method used to compensate for pixels 302 saturating also leads to additional benefits. With dark pixels 302, the sigma-delta converter 308 will simply read out the pixel 302 with the predetermined number of samples during the integration period resulting in excellent resolution. On the other hand, if the pixel 302 is exposed to bright light, the pixel 302 saturates and the sampling period is less, allowing the DSP to process and output the data faster. Also by accommodating for pixels 302 to saturate, this invention acts as an integrated auto-exposure feature.
Implementing a sigma-delta analog-to-digital converter 208 at the column level in this manner also leads to a number of further embodiments. The first embodiment is illustrated in
In a further embodiment as illustrated in
In
Similarly, as shown in
While the invention has been described according to what is presently considered to be the most practical and preferred embodiments, it must be understood that the invention is not limited to the disclosed embodiments. Those ordinarily skilled in the art will understand that various modifications and equivalent structures and functions may be made without departing from the spirit and scope of the invention as defined in the claims. Therefore, the invention as defined in the claims must be accorded the broadest possible interpretation so as to encompass all such modifications and equivalent structures and functions.
Patent | Priority | Assignee | Title |
11818479, | May 14 2020 | BOARD OF TRUSTEES OF MICHIGAN STATE UNIVERSITY | Sigma Delta quantization for images |
8477056, | Jun 01 2010 | Infinera Corporation | Method, system, and apparatus for interpolating an output of an analog-to-digital converter |
9140795, | Sep 20 2011 | AMS SENSORS SINGAPORE PTE LTD | Time of flight sensor with subframe compression and method |
9175957, | Sep 24 2012 | MANTISVISION LTD | Grayscale patterns from binary spatial light modulators |
9316826, | Sep 24 2012 | MANTISVISION LTD | Grayscale patterns from binary spatial light modulators |
Patent | Priority | Assignee | Title |
5134487, | Nov 05 1990 | Canon Kabushiki Kaisha | Using common circuitry for different signals |
5248971, | May 19 1992 | Method and apparatus for multiplexed oversampled analog to digital modulation | |
5264940, | Oct 08 1990 | Olympus Optical Co., Ltd. | Image sensing apparatus having exposure level and dynamic range control circuit |
5461425, | Feb 15 1994 | Stanford University | CMOS image sensor with pixel level A/D conversion |
5886659, | Aug 21 1997 | California Institute of Technology | On-focal-plane analog-to-digital conversion for current-mode imaging devices |
5955899, | Jan 27 1997 | Intel Corporation | Compact comparator |
6104844, | Apr 11 1997 | Siemens Aktiengesellschaft | Image sensor having a multiplicity of pixel sensor regions |
6353324, | Nov 06 1998 | Bridge Semiconductor Corporation | Electronic circuit |
6473122, | Dec 06 1999 | Method and apparatus to capture high resolution images using low resolution sensors and optical spatial image sampling | |
6606120, | Apr 24 1998 | FOVEON, INC | Multiple storage node full color active pixel sensors |
6665013, | Jan 28 1994 | California Institute of Technology | Active pixel sensor having intra-pixel charge transfer with analog-to-digital converter |
6678039, | May 23 2001 | Microsoft Technology Licensing, LLC | Method and system to enhance dynamic range conversion useable with CMOS three-dimensional imaging |
6765619, | Apr 04 2000 | PIXIM, INC | Method and apparatus for optimizing exposure time in image acquisitions |
6831684, | May 09 2000 | PIXIM INCORPORATED | Circuit and method for pixel rearrangement in a digital pixel sensor readout |
7339621, | Dec 13 2001 | Harusaki Technologies, LLC | Imager output signal processing |
EP912043, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Dec 13 2001 | Harusaki Technologies, LLC | (assignment on the face of the patent) | / | |||
Sep 17 2007 | PSION TEKLOGIX SYSTEMS INC | Harusaki Technologies, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025654 | /0826 | |
Sep 17 2007 | PSION TEKLOGIX SYSTEMS INC | Harusaki Technologies, LLC | CORRECTIVE ASSIGNMENT TO CORRECT THE ADDRESS OF THE ASSIGNEE, PREVIOUSLY RECORDED ON REEL 025654 FRAME 0826 ASSIGNOR S HEREBY CONFIRMS THE ASSIGNMENT | 026074 | /0126 |
Date | Maintenance Fee Events |
Aug 25 2015 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Aug 14 2019 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Sep 27 2014 | 4 years fee payment window open |
Mar 27 2015 | 6 months grace period start (w surcharge) |
Sep 27 2015 | patent expiry (for year 4) |
Sep 27 2017 | 2 years to revive unintentionally abandoned end. (for year 4) |
Sep 27 2018 | 8 years fee payment window open |
Mar 27 2019 | 6 months grace period start (w surcharge) |
Sep 27 2019 | patent expiry (for year 8) |
Sep 27 2021 | 2 years to revive unintentionally abandoned end. (for year 8) |
Sep 27 2022 | 12 years fee payment window open |
Mar 27 2023 | 6 months grace period start (w surcharge) |
Sep 27 2023 | patent expiry (for year 12) |
Sep 27 2025 | 2 years to revive unintentionally abandoned end. (for year 12) |