The present invention provides an image pick up apparatus having means for cumulatively adding image signals and storing the image signals in memory, which is capable of preventing cumulative addition of image signals for which camera-shake correction is insufficient and improving picture quality of image signals stored in memory.
|
4. A method for correcting an image signal picked up by an image pickup device, said method comprising:
detecting a camera-shake amount of said image pickup section and forming a camera-shake signal based on the detected camera-shake amount;
calculating a camera-shake correction amount based on said camera-shake signal and calculating a maximum camera-shake correction amount;
correcting said image signal picked up by said image pickup section based on the calculated camera-shake correction amount; and
comparing the calculated camera-shake correction amount and the calculated maximum camera-shake correction amount, and cumulatively adding the corrected image signal to a previously corrected image signal or signals only if the calculated camera-shake correction amount used to correct the image signal is less than the calculated maximum camera-shake correction amount; and
multiplying a coefficient value calculated according to a number of times cumulative addition is performed and the cumulatively added corrected image signal so as to form a multiplied signal, and outputting the multiplied signal.
3. An image pickup apparatus comprising:
an image pickup section for picking up an image signal;
means for detecting a camera-shake amount of said image pickup section and for forming a camera-shake signal based on the detected camera-shake amount;
means for calculating a camera-shake correction amount based on said camera-shake signal and for calculating a maximum camera-shake correction amount;
means for correcting said image signal picked up by said image pickup section based on the calculated camera-shake correction amount;
means for comparing the calculated camera-shake correction amount and the calculated maximum camera-shake correction amount, and for cumulatively adding the corrected image signal to a previously corrected image signal or signals only if the calculated camera-shake correction amount used to correct the image signal is less than the calculated maximum camera-shake correction amount; and
means for multiplying a coefficient value calculated according to a number of times cumulative addition is performed and the cumulatively added corrected image signal so as to form a multiplied signal, and for outputting the multiplied signal.
2. A_method for correcting an image signal picked up by an image pickup device, said method comprising:
detecting a camera-shake amount of said image pickup device and outputting a camera-shake signal according to the detected camera-shake amount;
correcting said image signal picked up by said image pickup section based on a camera-shake correction amount calculated according to said camera-shake signal;
setting a recording image size as an actual image area which is positioned in a possible image area showing a range for which imaging is possible at an image pickup element of said image pickup section;
calculating a maximum value for the camera-shake correction amount based on the set recording image size and a size of said possible image area;
cumulatively adding the corrected image signal and storing the same in a memory;
comparing the calculated camera-shake correction amount and the calculated maximum value for said camera-shake correction amount and subjecting to cumulative addition and storing in the memory the corrected image signal only if the image signal is corrected by less than the maximum correction amount; and
multiplying a coefficient value calculated according to a number of times cumulative addition is performed and the corrected image signal cumulatively added and stored in said memory so as to form a multiplied signal, and outputting the multiplied signal.
1. An image pickup apparatus comprising:
an image pickup section having an image pickup element for picking up an image signal;
camera-shake detection means for detecting a camera-shake amount of said image pickup section and outputting a camera-shake signal according to the detected camera-shake amount;
camera-shake correction means for correcting said image signal picked up by said image pickup section based on a camera-shake correction amount calculated according to said camera-shake signal outputted by said camera-shake detection means;
user interface for setting a recording image size as an actual image area which is positioned in a possible image area showing a range for which imaging is possible at said image pickup element of said image pickup section;
maximum correction amount calculating means for calculating a maximum value for the camera-shake correction amount based on the recording image size set by said user interface and a size of said possible image area;
means for cumulatively adding the corrected image signal and for storing the same in a memory;
cumulative addition control means for comparing said camera-shake correction amount calculated by said camera-shake correction means and the maximum value for said camera-shake correction amount calculated by said maximum correction amount calculating means and for subjecting to cumulative addition and storing in the memory the corrected image signal only if the image signal is corrected by less than the maximum correction amount; and
means for multiplying a coefficient value calculated according to a number of times cumulative addition is performed and the corrected image signal cumulatively added and stored in said memory so as to form a multiplied signal, and for outputting the multiplied signal.
|
|||||||||||||||||||||||||||||
1. Field of the Invention
This invention relates to an image pickup apparatus and method, and more particularly, an image pickup apparatus and method for preventing cumulative addition of image signals for which camera-shake correction is insufficient, and for improving picture quality of image signals stored in memory.
2. Description of the Related Art
Various forms of camera-shake prevention technologies for preventing blurring of images resulting from vibration due to movement of a camera operator and preventing degrading (blurring, dislocating) of recording images have been proposed for image pickup apparatus such as digital still cameras and video cameras capable of taking and recording images in the related art.
For example, an image pickup apparatus that detects an amount of dislocation in the event of taking a picture from an image (still image) captured over a prescribed period or prescribed number of times and then records images (still images) for which the dislocation amount is the smallest has been proposed, and described in, for example, Japanese Patent Application Publication No. Hei. 11-317904 (at page 11-12, FIG. 1, FIG. 26).
However, as shown in
Further, as the recording size of image signal (hereafter referred to as recording image size) is set in advance, when recording image size is set to be large against the range for which pictures can be taken by the image pickup element (hereafter referred to as possible image size), the range in which camera-shake correction is possible (hereafter referred to as correction range) becomes small, and the ends of the correction range are reached even for comparatively small camera-shake vibrations.
As an image signal corrected at the ends (hereinafter, correction ends) of the correction range is an image signal corresponding to the straight line parts of
Therefore, there is the problem to be resolved for image pickup apparatus having means for cumulatively adding image signals and storing the image signals in memory, with respect to preventing cumulative addition of image signals for which camera-shake correction is insufficient, and improving picture quality of image signals stored in memory.
In order to resolve the aforementioned problems, the image pickup apparatus of the present invention is configured in the following manner.
(1) An image pickup apparatus having means for cumulatively adding image signals picked up by an image pickup section for storage in memory, characterized by: camera-shake detection means for detecting an amount of vibration due to hand-shaking of the image pickup section and outputting a camera-shake signal according to the detected camera-shake vibration amount, camera-shake correction means for correcting the image signal photographed by the image pickup section based on a camera-shake correction amount calculated according to the camera-shake signal outputted by the camera-shake detection means, recording image size setting means for setting recording image size in the event of recording the image signal, maximum correction amount calculating means for calculating a maximum value for a camera-shake correction amount for correcting camera-shake of the image signal based on recording image size set using the recording image size setting means and a possible image size constituting a range for which imaging is possible at the image pickup element of the image pickup section, image discrimination means for comparing the camera-shake correction amount calculated by the camera-shake correction means and the maximum value for the camera-shake correction amount calculated by the maximum value correction amount calculating means and performing discrimination processing on an image signal cumulatively added to the memory, and cumulative addition control means for controlling cumulative addition processing of the image signal corrected by the camera-shake correction means in accordance with the image discrimination means.
(2) An image pickup apparatus having means for cumulatively adding image signals picked up by an image pickup section for storage in memory, characterized by: camera-shake detection means for detecting an amount of vibration due to hand-shaking of the image pickup section and outputting a camera-shake signal according to the detected camera-shake vibration amount, camera-shake correction means for correcting the image signal picked up by the image pickup section based on a camera-shake correction amount calculated according to the camera-shake signal outputted by the camera-shake detection means, recording image size setting means for setting recording image size in the event of recording the image signal, maximum correction amount calculating means for calculating a maximum value for a camera-shake correction amount for correcting camera-shake of the image signal based on recording image size set using the recording image size setting means and a possible image size constituting a range for which imaging is possible at the image pickup element of the image pickup section, image discrimination means for comparing the camera-shake correction amount calculated by the camera-shake correction means and the maximum value for the camera-shake correction amount calculated by the maximum value correction amount calculating means and performing discrimination processing on an image signal cumulatively added to the memory, cumulative addition control means for controlling cumulative addition processing of the image signal corrected by the camera-shake correction means in accordance with the image discrimination means, and image output means for multiplying a coefficient value calculated according to the number of times of cumulative addition processing by the cumulative addition control means and an image signal cumulatively added and then stored and outputted.
With an image pickup apparatus of this configuration, a maximum value for a camera-shake correction amount calculated based on recording image size set by the recording image size setting means and actual image size constituting a range where imaging is possible of the image pickup element of the image pickup section and a camera-shake correction amount calculated by the camera-shake correction means are compared, and cumulative addition processing of the image signal corrected by the camera-shake correction means is controlled. It is therefore possible to only cumulatively add and store to memory image signals appropriately corrected within the range where camera-shake correction is possible.
By comparing a maximum value (hereinafter, maximum correction value) for a camera-shake correction amount calculated based on the predetermined recording image size and possible image size constituted by the range of the image pickup element of the image pickup section for which imaging is possible and a camera-shake correction amount calculated based on camera-shake vibration amount of the image pickup section and then cumulatively adding and storing in memory only image signals corrected using camera-shake correction amounts smaller than the maximum correction amount, image signals for which correction processing is insufficient (those reaching the correction ends, image signals correction-processed using the maximum correction amounts) are not subjected to cumulative addition, and only image signals that have been appropriately corrected are cumulatively added. This means that superior results can be obtained where the picture quality (S/N ratio) of image signals stored in memory is improved.
Further, the maximum correction amount is calculated based on the predetermined recording image size. Therefore, even when with respect to the possible image size the recording image size is large, i.e. the camera-shake correction range is small, since only appropriately corrected image signals are cumulatively added, superior results that the picture quality (S/N ratio) of the image signals stored in memory is improved, can be obtained.
Moreover, the aforementioned image signal stored in memory is multiplied by a coefficient value corresponding to the number of times of cumulative addition and outputted. It is therefore possible to record image signals (still images) of superior picture quality (S/N ratio) to recording media and recording devices.
The following is a description with reference to the drawings of a preferred embodiment of an image pickup apparatus of the present invention. However, the drawings are provided for the purpose of description, and in no way limit the technological scope of the present invention.
The image pickup section 110 is equipped with an image pickup element such as a lens and CCD (Charge Coupled Device) etc. and a signal processing circuit etc., and converts light from a subject inputted via the lens to an image signal using an image pickup element. This image signal is then subjected to prescribed signal processing at a signal processing circuit and is then sent to the correction circuit 120.
The correction circuit 120 corrects image signals transmitted from the image pickup section 110 based on correction vector data specified by the microcomputer 160 and transmits these image signals to the cumulative addition circuit 130.
As shown in
The adder 131 of the cumulative addition circuit 130 adds and outputs the image signal outputted from the correction circuit 120 and the image signal stored in the memory 132. When the switch 133is on, the memory 132 stores image signals outputted from the adder 131 and sends image signals to the adder 131 when image signals are inputted from the correction circuit 120. Further, the switch 133 is turned on and off according to the camera-shake signal of the camera-shake detector 170 in accordance with the control of the microcomputer 160, and the output (image signal) of the adder 131 is sent to the memory 132 when the switch 133 is on.
The multiplier 140 subjects image signals outputted from the cumulative addition circuit 130 to multiplication processing under the control of the microcomputer 160 based on coefficient values specified by the microcomputer 160 and outputs the image signals to the recording section 150.
The recording section 150 records image signals outputted from the multiplier 140 under the control of the microcomputer 160. For example, image signals are recorded on a recording medium such as a hard disc device, memory card, magnetic tape, or optical disc etc.
The microcomputer 160 controls each section within the image pickup apparatus and each circuit based on pre-stored programs and data etc.
For example, camera-shake correction amounts and correction vector data for correcting camera-shake are calculated based on a camera-shake signal detected by the camera-shake detector 170 and are sent to the correction circuit 120.
Further, a maximum value for the camera-shake correction amount (hereinafter, maximum correction amount) is calculated based on the size of the range for which photographing is possible using the image pickup element of the image pickup section 110 and the recording image size set by the UI section 180, and the cumulative addition circuit 130 is controlled based on the calculated maximum correction amount and the camera-shake correction amount.
Further, calculation of coefficient values for the multiplier 140 and transmission of these coefficient values to the multiplier 140, control of timing in the event of recording image signals to the recording section 150, etc. are carried out.
The camera-shake detector 170 detects the amount of vibration due to camera-shake of the image pickup section 110 and sends the detected vibration amount to the microcomputer 160 as a camera-shake signal. Detection of the camera-shake signal is carried out by utilizing a gyro sensor and image recognition technology etc., a value that is the integral of the amount of vibration due to the camera-shake, being detected as a camera-shake signal and sent to the microcomputer 160 every time an image signal is captured.
For example, with the integrator expressing the complete integral shown in
Y(z)=1/((1−z−1)/X(z))
The UI (user interface) section 180 transmits control signals so that prescribed setting/operations are executed at the microcomputer 160 according to user operations.
A description is now given with reference to
When the length of the possible image area is taken to be bh in the vertical direction and bw in the horizontal direction, and the length of the actual image area is taken to be ah in the vertical direction and aw in the horizontal direction, for example, in a state where camera-shake correction is not carried out, the actual image area is positioned at the center of the possible image area. The maximum correction amount in this state can then be calculated using the following equation.
Correction amount in vertical direction:
Sw=(bh−ah)/2
Correction amount in horizontal direction:
Sw=(bw−aw)/2 where bh≧ah, bw≧aw.
A description is given of the operation flow occurring at the image pickup apparatus of this configuration with reference to the flowchart of
When imaging commences, first, the microcomputer 160 determines whether or not the value of a capture counter (hereinafter referred to as CC) counting the number of image signals captured (hereinafter referred to as capture number) is “0” (ST110).
When CC is “0”, the microcomputer 160 clears the correction amount for the correction circuit 120 to “0” (ST111).
Switch 133 of the cumulative addition circuit 130 is then turned on, and image signals taken by the image pickup section 110 are captured, cumulatively added, and stored in the memory 132 (ST112, ST113).
As a result, the first image signal taken as a reference is stored in memory 132 of the cumulative addition circuit 130. This is to say that the correction amount for the image signal is first cleared to “0”, and the maximum correction amount is not exceeded, so that an image signal is prevented from not being captured at all in cases where for image signals captured thereafter the camera-shake correction amount exceeds the maximum correction amount.
The microcomputer 160 then increases the value of CC by 1 and increases the value of the time counter (hereinafter referred to as TC) by 1, so that processing is complete (ST114,ST115→END)
On the other hand, when CC is not “0”, then a captured signal is being cumulatively added at the cumulative addition circuit 130, and the microcomputer 160 then determines whether or not the value of CC is greater than or equal to the maximum value (hereinafter referred to as CCmax) of the capture number set in advance (ST110→ST120).
When the value of CC is greater than or equal to the maximum value CCmax (CC≧CCmax), processing for recording the image signal to the recording section 150 is advanced to (ST120→ST130→ . . . ).
On the other hand, in the event that the value of CC is smaller than the maximum value CCmax (CC<CCmax), the microcomputer 160 determines whether or not the value of TC is greater than or equal to a prescribed set time (hereinafter referred to as TCmax) (ST120→ST121).
When the value of TC is greater than or equal to the set time TCmax (TC≧TCmax), processing for recording the image signal to the recording section 150 is advanced to (ST121→ST130→ . . . ).
On the other hand, in the event that the value of TC is smaller than the set time TCmax (TC<TCmax), the microcomputer 160 calculates the camera-shake correction amount based on the camera-shake signal from the camera-shake detector 170 and calculates correction vector data for transmission to the correction circuit 120 based on this camera-shake correction amount. At the correction circuit 120, the image signal sent from the image pickup section 110 is corrected based on the correction vector data (ST121→ST122).
Continuing on, the microcomputer 160 compares the absolute value of the calculated camera-shake correction amount and the maximum correction amount calculated based on the image pickup element size and the recording image size set by the UI section 180 (ST122→ST123)
When the absolute value of the camera-shake correction amount is less than the maximum correction amount, the switch 133 of the cumulative addition circuit 130 is turned on, and the image signal from the correction circuit 120 is cumulatively added and stored in the memory 132 (ST123→ST112, ST113)
Next, the microcomputer 160 increases the value of CC by 1 and increases the value of the time counter (hereinafter referred to as TC) by 1, so that processing is complete (ST114, ST115→END)
On the other hand, when the absolute value of the camera-shake correction amount exceeds the maximum correction value, the value of TC is increased by 1 and processing ends (ST123→ST115→END).
Further, in step S120 above, in the event that the value of CC is greater than or equal to CCmax (CC≧CCmax), or in step S121 above, in the event that the value of TC is greater than or equal to TCmax (TC≧TCmax), processing commences to record the image signal stored in the cumulative addition circuit 130 to the recording section 150.
First, the microcomputer 160 provides the reciprocal of the value of CC to the multiplier 140 as a prescribed coefficient (ST120→ST130, or ST121→ST130)
Next, the multiplier 140 multiplies the prescribed coefficient provided by the microcomputer 160 and the image signal cumulatively added at the cumulative addition circuit 130 and outputs this to the recording section 150, and the recording section 150 records the image signal outputted from the multiplier 140 on a prescribed recording medium (recording device) (ST131).
When the image signal is recorded in the recording section 150, the microcomputer 160 clears the value of CC, clears the value of TC, and ends processing (ST132→ST133→END).
In the graph of
Nakamura, Makibi, Nomura, Hirofumi
| Patent | Priority | Assignee | Title |
| 10535123, | Nov 27 2006 | GOOGLE LLC | Method for sliced inpainting |
| 7961233, | Feb 23 2006 | OM DIGITAL SOLUTIONS CORPORATION | Electronic blurring compensation device and electronic blurring compensation method |
| 9053530, | Nov 27 2006 | GOOGLE LLC | Method for sliced inpainting |
| 9870604, | Nov 27 2006 | GOOGLE LLC | Method for sliced inpainting |
| Patent | Priority | Assignee | Title |
| 4490744, | Jun 30 1982 | RCA Corporation | Smear reduction technique for CCD field-transfer imager system |
| 4907287, | Oct 16 1985 | Hitachi, Ltd. | Image correction system for scanning electron microscope |
| 5210559, | Mar 19 1991 | Sony Corporation | Camera shake correcting device |
| 5296925, | Apr 27 1990 | Canon Kabushiki Kaisha | Movement vector detection device |
| 5416557, | Dec 28 1989 | Olympus Optical Co., Ltd. | Camera apparatus having drift detecting unit |
| 5712474, | Sep 29 1993 | Canon Kabushiki Kaisha | Image processing apparatus for correcting blurring of an image photographed by a video camera |
| 6906746, | Jul 11 2000 | FUJIFILM Corporation | Image sensing system and method of controlling operation of same |
| JP11317904, | |||
| JP2000341577, | |||
| JP2001285698, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Jun 29 2004 | Sony Corporation | (assignment on the face of the patent) | / | |||
| Oct 01 2004 | NOMURA, HIROFUMI | Sony Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 015911 | /0981 | |
| Oct 04 2004 | NAKAMURA, MAKIBI | Sony Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 015911 | /0981 |
| Date | Maintenance Fee Events |
| Jan 21 2013 | REM: Maintenance Fee Reminder Mailed. |
| Jun 09 2013 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
| Date | Maintenance Schedule |
| Jun 09 2012 | 4 years fee payment window open |
| Dec 09 2012 | 6 months grace period start (w surcharge) |
| Jun 09 2013 | patent expiry (for year 4) |
| Jun 09 2015 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Jun 09 2016 | 8 years fee payment window open |
| Dec 09 2016 | 6 months grace period start (w surcharge) |
| Jun 09 2017 | patent expiry (for year 8) |
| Jun 09 2019 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Jun 09 2020 | 12 years fee payment window open |
| Dec 09 2020 | 6 months grace period start (w surcharge) |
| Jun 09 2021 | patent expiry (for year 12) |
| Jun 09 2023 | 2 years to revive unintentionally abandoned end. (for year 12) |