An image processing apparatus is provided which offers higher versatility than conventional image processing apparatuses. When an input signal to a spatial filtering block is a monochrome signal that contains Y component only, a selector selects its input terminal and a selector selects its input terminal. Then, a low-pass filter output signal of a programmable spatial filter is inputted to a spatial filter, and a low-pass filter output signal of the spatial filer is inputted to a spatial filter. That is, the programmable spatial filter and the spatial filters are connected in series (in cascade), and the cascade-connected three spatial filters perform filtering operation. In this example, low-pass filters with 5×5 taps are connected in cascade in three stages, which enables low-pass filtering with 13×13 taps.
|
1. An image processing apparatus comprising:
a plurality of image processing blocks configured to process in real-time an image signal inputted from outside, said plurality of image processing blocks including a pixel interpolation block that applies a pixel interpolation to an input signal, said pixel interpolation block including a pixel register group,
wherein,
when said input signal to said pixel interpolation block is a signal that requires said pixel interpolation, the pixel interpolation is performed, and
when said input signal to said pixel interpolation block is a signal that does not require said pixel interpolation, a spatial filtering is performed by a first spatial filter using said pixel register group,
the plurality of image processing blocks further including a spatial filtering block that performs a spatial filtering by a second spatial filter, wherein, when said input signal to said pixel interpolation block is a signal that requires said pixel interpolation, the spatial filtering by said second spatial filter is applied to the signal that has been subjected to said pixel interpolation in said pixel interpolation block, and when said input signal to said pixel interpolation block is a signal that does not require said pixel interpolation, a spatial filtering is performed by continuously applying said first and second spatial filters.
8. An image processing method for an image processing apparatus, comprising:
processing in real-time, at a plurality of image processing blocks, an image signal inputted from outside;
applying, at a pixel interpolation block included in the plurality of image processing blocks and having a pixel register group, a pixel interpolation to an input signal,
performing the pixel interpolation and applying a spatial filtering to the interpolated input signal in response to said input signal to said pixel interpolation block being a signal requiring said pixel interpolation;
performing a spatial filtering by a first spatial filter using the pixel register group in response to said input signal to said pixel interpolation block being a signal that does not require said pixel interpolation; and
performing, at a spatial filtering block included in the plurality of image processing blocks, a spatial filtering by a second spatial filter, wherein, when said input signal to said pixel interpolation block is a signal that requires said pixel interpolation, the spatial filtering by said second spatial filter is applied to the signal that has been subjected to said pixel interpolation in said pixel interpolation block, and when said input signal to said pixel interpolation block is a signal that does not require said pixel interpolation, a spatial filtering is performed by continuously applying said first and second spatial filters.
2. The image processing apparatus according to
3. The image processing apparatus according to
4. The image processing apparatus according to
5. The image processing apparatus according to
6. The image processing apparatus according to
7. The image processing apparatus according to
|
1. Field of the Invention
The present invention relates to image processing apparatuses, and particularly to an image processing apparatus having a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside.
2. Description of the Background Art
For an image input apparatus such as a digital still camera, Japanese Patent Application Laid-Open No. 2000-236473 discloses a conventional image processing apparatus including an imaging device, an analog signal processing circuit connected to the imaging device, and a Real-time Processing Unit (RPU) connected to the analog signal processing circuit, where the RPU is capable of processing in a real-time manner an image signal outputted from the analog signal processing circuit.
However, this conventional image processing apparatus does not offer sufficient versatility in the following points:
the number of taps of spatial filter in the spatial filtering block is fixed;
it is not possible to obtain a high-frequency component when the spatial filter is set not as a high-pass filter;
it is not possible to obtain a mixture signal, such as a signal containing center pixel signal and low-frequency component at a desired ratio, as an output signal from the spatial filter;
image quality is considerably deteriorated when a divergence occurs between a pixel as a source of generation of feature data for modulation and a pixel as a target of modulation of luminance signal;
image quality is considerably deteriorated when a divergence occurs between a pixel as a source of generation of feature data for modulation and a pixel as a target of modulation of color signal;
no means is provided which is capable of applying high-speed color transformation to 2-channel color signals (Cb, Cr) outputted from the color-space transformation block;
signals are processed as YCbCr color-space signals after the output of the color-space transformation block, and only a signal of the same color space can be obtained as the final output signal; and
signals are processed as 8-bit-system signals after the output of the gamma correction block, and only a signal of the same number of bits can be obtained as the final output signal.
An object of the present invention is to obtain an image processing apparatus having higher versatility than conventional image processing apparatuses.
According to a first aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a spatial filtering block that applies a spatial filtering to an input signal. The spatial filtering block includes a plurality of spatial filters. When the input signal is a multicolor signal, the plurality of spatial filters perform the spatial filtering in parallel for individual channels corresponding to individual colors, and when the input signal is a monochrome signal, the plurality of spatial filters are connected in series to perform the spatial filtering.
According to this configuration, when the input signal is a monochrome signal, the plurality of spatial filters, connected in series, perform spatial filtering operation. Thus, an extra spatial filter that is originally not related to the processing of monochrome signals can be utilized to achieve filtering operation using spatial filter with a larger number of taps. Also, the total processing time can be shorter than when a filtering operation using the same spatial filter is repeated multiple times.
According to a second aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a pixel interpolation block that applies a pixel interpolation to an input signal. The pixel interpolation block includes a pixel register group. When the input signal to the pixel interpolation block is a signal that requires the pixel interpolation, the interpolation is performed, and when the input signal to the pixel interpolation block is a signal that does not require the pixel interpolation, a spatial filtering is performed by a first spatial filter using the pixel register group.
According to this configuration, when the input signal to the pixel interpolation block is a signal that does not require pixel interpolation, a spatial filtering using the pixel resister group is performed. It is thus possible to realize filtering operation using spatial filter with a larger number of taps. Also, the total processing time can be shorter. Furthermore, since the first spatial filter performs spatial filtering using the pixel register group of the pixel interpolation block, the circuit scale can be smaller than when a similar pixel register group is newly provided in the spatial filtering block.
According to a third aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a spatial filtering block that applies a spatial filtering to an input signal. The spatial filtering block includes a first low-pass filter, a programmable spatial filter, and a first operator. The first low-pass filter performs a first low-pass filtering. The first operator subtracts a first low-pass filter output signal that is an output signal of the first low-pass filter from a center pixel signal that is an original pixel signal, so as to generate a high-frequency component. The spatial filtering block is capable of simultaneously outputting at least two of the first low-pass filter output signal, an output signal of the programmable spatial filter, and the high-frequency component.
According to this configuration, the first operator subtracts the first low-pass filter output signal from the center pixel signal to generate a high-frequency component. It is thus possible to obtain a high-frequency component similar to a high-pass filter output signal, without using a high-pass filter.
According to a fourth aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a spatial filtering block that applies a spatial filtering to an input signal. The spatial filtering block includes a first low-pass filter and a first operator. The first low-pass filter performs a first low-pass filtering. The first operator multiplies together a center pixel signal that is an original pixel signal and a first coefficient, multiplies together a first low-pass filter output signal that is an output signal of the first low-pass filter and a second coefficient, and adds the results of these multiplications together. The sum of the first coefficient and the second coefficient is constant.
According to this configuration, the first operator multiplies together the center pixel signal and a first coefficient, multiplies together the first low-pass filter output signal and a second coefficient, and adds the results of the multiplications together. It is thus possible, by setting the first and second coefficients to desired values, to obtain an output signal in which the center pixel signal and the first low-pass filter output signal are mixed at a desired ratio. The output signal thus obtained can be arbitrarily utilized according to the purpose. It is also possible to keep the direct-current component gain at a constant value because the sum of the first coefficient and the second coefficient is constant.
According to a fifth aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a spatial filtering block and a signal modulation block. The spatial filtering block applies a spatial filtering to an input signal. The signal modulation block applies a modulation to an output signal from the spatial filtering block as a signal to be modulated. The spatial filtering block includes a spatial filter that outputs a luminance signal that has been subjected to the spatial filtering. The signal modulation block includes a low-pass filter and an operator. The low-pass filter applies a low-pass filtering to a pixel unit feature signal corresponding to a characteristic of each pixel. The operator modulates the luminance signal by using the pixel unit feature signal that has been subjected to the low-pass filtering.
According to this configuration, the luminance signal is modulated with a pixel unit feature signal that has been low-pass-filtered. Applying the low-pass filtering to the pixel unit feature signal suppresses variations among pixels. This avoids significant variations in the degree of modulation of the luminance signal among pixels. As a result, even if a divergence occurs between a pixel as a source of generation of the pixel unit feature signal and a pixel as a target of the luminance signal modulation, it is possible to lower the degree of image quality deterioration due to divergence in luminance from proper value, than when the pixel unit feature signal is not low-pass-filtered.
According to a sixth aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a spatial filtering block and a signal modulation block. The spatial filtering block applies a spatial filtering to an input signal. The signal modulation block applies a modulation to an output signal from the spatial filtering block as a signal to be modulated. The spatial filtering block includes a spatial filter that outputs a color signal that has been subjected to the spatial filtering. The signal modulation block includes a low-pass filter and an operator. The low-pass filter applies a low-pass filtering to a pixel unit feature signal corresponding to a characteristic of each pixel. The operator modulates the color signal by using the pixel unit feature signal that has been subjected to the low-pass filtering.
According to this configuration, the color signal is modulated with a pixel unit feature signal that has been low-pass-filtered. Applying low-pass filtering to the pixel unit feature signal suppresses variations among pixels. This avoids significant variations in the degree of modulation of the color signal among pixels. As a result, even if a divergence occurs between a pixel as a source of generation of the pixel unit feature signal and a pixel as a target of color signal modulation, it is possible to lower the degree of image quality deterioration due to divergence in color from proper value, than when the pixel unit feature signal is not low-pass-filtered.
According to a seventh aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a color-space transformation block that applies a color-space transformation to an input signal. The color-space transformation block includes a color-space transformation circuit and a two-dimensional lookup table. The color-space transformation circuit performs the transformation to output 2-channel color signals. The two-dimensional lookup table follows the color-space transformation circuit and transforms the 2-channel color signals on the basis of a plurality of transform values previously defined which represent a correspondence between input data value pairs and output data value pairs.
According to this configuration, the two-dimensional lookup table is connected to follow the color-space transformation circuit that outputs two-channel color signals. The color-space transformation block performs color-space transformation by real-time processing. When the color-space transformation is performed using a three-dimensional lookup table that involves large amounts of calculations, the processing in the three-dimensional lookup table forms a bottleneck to delay the entire processing. Also, a three-dimensional lookup table involves a very large circuit scale. In contrast, using a two-dimensional lookup table avoids increased circuit scale and achieves high-speed color-space transformation without delaying the entire processing.
According to an eighth aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a first color-space transformation block, a processing block, and a second color-space transformation block. The first color-space transformation block transforms a signal of a first color space to a signal of a second color space including a luminance component and a plurality of color components. The processing block applies predetermined image processing to the signal of the second color space outputted from the first color-space transformation block. The second color-space transformation block transforms the signal of the second color space outputted from the processing block into a signal of a color space that is different from the second color space.
According to this configuration, the second color-space transformation block transforms a signal of the second color space outputted from the processing block into a signal of a color space that is different from the second color space. Thus, the processing block performs various operations using signals of the second color space, and then the second color-space transformation block transforms the color space. It is thus possible to finally obtain an output signal of the first color space, for example.
According to a ninth aspect of the present invention, an image processing apparatus includes a plurality of image processing blocks that are capable of real-time processing of an image signal inputted from outside. The plurality of image processing blocks include a resizing block that applies an image resizing to an input signal. The resizing block includes a first resizing circuit and a second resizing circuit. The first resizing circuit is capable of processing a signal with a first number of bits. The second resizing circuit is capable of processing a signal with a second number of bits that is smaller than the first number of bits. When the input signal is of the first number of bits, the resizing block performs the resizing by using the first resizing circuit, and when the input signal is of the second number of bits, the resizing block performs the resizing by using the second resizing circuit.
According to this configuration, 12-bit-system signals for main images and 8-bit-system signals for display or for thumbnail images, for example, are inputted as input signals to the resizing block. Then, it is possible to select which resizing circuit should be used according to the number of bits of the input signal, from among a plurality of resizing circuits configured to process signals of different numbers of bits. Furthermore, the circuit scale can be smaller than when the plurality of resizing circuits are all provided as resizing circuits capable of processing signals of the first number of bits.
These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.
A preferred embodiment of the present invention will now be described in detail referring to the drawings. In the drawings, the same reference characters indicate the same or corresponding components.
The CCD 1 has a three-color-system color filter for, e.g., R (Red), G (Green) and B (Blue), or a four-color-system color filter for, e.g., Y (yellow), M (Magenta), C (Cyan), and W (White), and it pictures an image with an optical lens and outputs the image signal thus obtained. Other type of imaging device, such as a CMOS image sensor, may be used in place of the CCD 1.
The analog signal processing circuit 2 receives the analog image signal from the CCD 1, and applies signal processings to the analog image signal, such as noise reduction, signal amplification, A/D conversion, etc., so as to output a digital image signal.
The SPU 3 receives the image signal from the analog signal processing circuit 2, and applies thereto various signal processings related to problems due to image sensor characteristics, such as white balance control.
The RPU 4 receives the image signal from the SPU 3, or an image signal from the SDRAM 11 through the main bus 5, and applies various signal processings thereto, such as pixel interpolation, color-space transformation, false color suppression, etc. (which will be fully described later).
The CPU 6 generally controls the operations of the components shown in
The LCD 7 functions as a finder of the digital still camera. The LCD 7 displays a low-resolution image represented by, e.g., an 8-bit-system image signal. The term “8-bit system” means eight bits with no sign or nine bits with a sign. Hereinafter, “n-bit system” means n bits with no sign or n+1 bits with a sign. The display of images in the LCD 7 is controlled by the LCD driver 8.
When a user presses an image-taking button (not shown), a high-resolution image (main image) represented by, e.g., a 12-bit-system image signal, is recorded in the memory card 13. The resolution of the main image can be selected by the user. The recording of images in the memory card 13 is controlled by the card controller 12.
The RPU 4 is capable of processing an image signal directly inputted from the SPU 3 (real-time processing) and an image signal once stored in the SDRAM 11 from the SPU 3 and then transferred from the SDRAM 11 through the main bus 5 (post processing).
In real-time processing, all processing blocks 20 to 29 operate on the basis of a given pixel clock synchronized with a read clock for reading pixel signals from the CCD 1. That is, the processing blocks 20 to 29 receive input image signals from the preceding processing blocks according to the pixel clock, and process the input image signals according to the pixel clock. All processing blocks 20 to 29 are capable of operating in parallel according to the pixel clock, and so the RPU 4 is constructed as a pipeline image processing device. In post processing, a pixel clock for the RPU 4 may be set independently of the read clock for the CCD 1.
An image signal inputted to the single pixel processing block 20 from the SPU 3 or the SDRAM 11 is sequentially processed in the processing blocks 20 to 29, and then transferred from the resizing block 29 to the SDRAM 11 through the main bus 5. It is also possible to bypass arbitrary one or ones of the processing blocks 20 to 29.
The single pixel processing block 20, the pixel interpolation block 21, and the first gamma-correction block 22 have inputs and outputs connected to the main bus 5. Accordingly, it is possible to input an image signal to each of the processing blocks 20 to 22 from the SDRAM 11 through the main bus 5, and to transfer the image signal, processed in the processing blocks 20 to 22, to the SDRAM 11 through the main bus 5. For example, it is possible to input an image signal from the SDRAM 11 to the first gamma-correction block 22 and transfer the image signal, processed in the first gamma-correction block 22, to the SDRAM 11 from the first gamma-correction block 22.
The RPU 4 has 4-channel input DMA channels IN-CH0 to IN-CH3 (see
In the notation of the numbers of bits of image signals, “S” and “U” preceding the numbers of bits represent “with a sign” and “with no sign”, respectively. For example, “S17” means 17 bits with a sign, and “U8” means 8 bits with no sign. Also, in the diagrams, a reduction of the number of bits indicates “clipping” of omitting low-order bits after a multiplier, or omitting high-order bits after an adder or a shifter, unless otherwise stated.
Single Pixel Processing Block 20
The configuration of the single pixel processing block 20 is illustrated in
The single pixel processing block 20 has a function of processing, pixel by pixel, an image signal (Input Data) 120 inputted from the SPU 3, and an image signal inputted from the SDRAM 11 through the main bus 5, and it is capable of selectively performing temporal averaging, shading correction, etc.
The temporal averaging is a process of averaging an image signal inputted to the RPU 4 over a plurality of frames or a plurality of fields, while reading pixel signals from the CCD 1 according to the read clock. The scheme of addition of pixel signals for averaging can be selected from cumulative addition and circulating addition, according to the settings of the selectors 107 and 108 and the coefficients B1 and B2. Cumulative addition is performed when the selector 107 selects its input terminal 1072, the selector 108 selects its input terminal 1082, and the coefficients B1 and B2 are both set to “1”. On the other hand, circulating addition is performed when the selector 107 selects its input terminal 1071, the selector 108 selects its input terminal 1082, and the coefficient B2 is set to “α”.
Also, the shading correction can be performed when the selector 107 selects its input terminal 1072, the selector 108 selects its input terminal 1081, and the coefficient B1 is set to “0”. In this case, a given shading correction parameter is inputted to the input terminal 1081 of the selector 108 from the SDRAM 11 through the main bus 5 and the selector 106.
Pixel Interpolation Block 21
The configuration of the pixel interpolation block 21 is illustrated in
The interpolation operation unit 208 receives the 49 pixel signals stored in the pixel registers of the pixel register group 201. On the basis of the input pixel signals, the interpolation operation unit 208 interpolates a color component absent for the target pixel, by referring to the pixel signals around the target pixel. For example, when the target pixel is a pixel having R component in the RGB color space, the absent G and B components are generated by interpolation. Then, the color channel C0 output terminal of the interpolation operation unit 208 outputs R-component pixel signal, the color channel C1 output terminal outputs G-component pixel signal, and the color channel C2 output terminal outputs B-component pixel signal. The color channel C3 output terminal outputs, as needed, a KEY signal for particularly featuring each pixel. When the input signal to the RPU 4 is a 4-color-system signal of Y, M, C, W, then the color channel C0 output terminal outputs Y-component pixel signal, the color channel C1 output terminal outputs M-component pixel signal, the color channel C2 output terminal outputs C-component pixel signal, and the color channel C3 output terminal outputs W-component pixel signal.
The interpolation operation unit 208 also has a function of determining the degree of correlation between target and surrounding pixels using an arbitrary correlation determining scheme (e.g., vertical/horizontal correlation determination) on the basis of the pixel signals inputted from the pixel register group 201, and the resultant correlation signal is outputted from the channel K0 (channel C4) or channel K1 (channel C5) output terminal.
The interpolation operation unit 208 also has a function of detecting color level (chroma, saturation) of a target pixel on the basis of the pixel signals inputted from the pixel register group 201, and the resultant color level signal is outputted from the channel K0 or channel K1 output terminal.
Also, the interpolation operation unit 208 has a function of performing Sobel filtering for edge detection on the basis of the pixel signals inputted from the pixel register group 201, and the resultant Sobel filter output signal is outputted from the channel K0 or channel K1 output terminal.
These correlation signal, color level signal, and Sobel filter output signal represent features of each pixel, and they can be regarded as “pixel unit feature signals” corresponding to characteristics of each pixel. It is possible to arbitrarily select which of the correlation signal, color level signal, and Sobel filter output signal should be outputted from the channels K0 and K1. It is also possible to output one of the pixel unit feature signals as the above-mentioned KEY signal from the color channel C3 output terminal.
First Gamma-Correction Block 22
The configuration of the first gamma-correction block 22 is illustrating in
The gamma correction unit 301 is preceded by a linear matrix converter 302. The linear matrix converter 302 performs color corrections in linear region, such as correction of difference from ideal characteristics of the color filter of the CCD 1.
When selectors 209 to 212 respectively select their input terminals 2092 to 2122 and selectors 303 to 306 respectively select their input terminals 3032 to 3062, the output signals from the interpolation operation unit 208 can be inputted to the gamma correction unit 301. When the selectors 303 to 306 respectively select their input terminals 3031 to 3061 and a selector 308 selects its input terminal 3081, an output signal from the SPU 3 can be inputted to the gamma correction unit 301. When the selectors 303 to 306 respectively select their input terminals 3031 to 3061 and the selector 308 selects its input terminal 3082, an output signal from the shifter/limiter 118 of the single pixel processing block 20 can be inputted to the gamma correction unit 301. Also, when the selectors 303 to 306 respectively select their input terminals 3031 to 3061 and the selector 308 selects its input terminal 3083, an image signal read from the SDRAM 11 can be inputted to the gamma correction unit 301 through a color sampling module 307.
The output signals from the gamma correction unit 301 are inputted to a clipping circuit 310 through a selector 309, clipped in the clipping circuit 310 to 12-bit-system image signals, and then inputted to the following first color-space transformation block 23. The output signals from the gamma correction unit 301 can also be inputted to the SDRAM 11 through the selector 309 and the main bus 5. Pixel unit feature signals outputted from the channels K0, K1 of the interpolation operation unit 208 are inputted to the following first color-space transformation block 23 through the selector 309 and the clipping circuit 310.
First Color-Space Transformation Block 23
The configuration of the first color-space transformation block 23 is illustrated in
The color-space transformation circuit 401 is capable of performing matrix operation with 4-channel inputs and 4-channel outputs, which, for example, transforms an RGB color-space or YMCW color-space image signal into the YCbCr color space (YUV color space) and outputs the image signal. The color channel C0 output terminal of the color-space transformation circuit 401 outputs Y-component pixel signal, the color channel C1 output terminal outputs Cb-component pixel signal, and the color channel C2 output terminal outputs Cr-component pixel signal. The color channel C3 output terminal outputs a KEY signal as needed.
The two-dimensional lookup table 402 is connected to the output terminals of the color channels C1 and C2 of the color-space transformation circuit 401, and it transforms the values of the Cb-component and Cr-component pixel signals to desired values, on the basis of a plurality of transform values representing a correspondence between pairs of input data values (Cb, Cr) and pairs of output data values (Cb, Cr). The two-dimensional lookup table 402 will be described in detail later.
An exposure determining evaluator 403 is connected to the color channel C0 output terminal of the color-space transformation circuit 401. For preconditions to determine shutter speed and diaphragm stop, the exposure determining evaluator 403 determines exposure level on the basis of proper luminance of the actual image signal. It divides one frame of image into a plurality of blocks and performs a luminance evaluation to level the luminances of the blocks.
Spatial Filtering Block 24
The configuration of the spatial filtering block 24 is illustrated in
Referring to
Referring to
The spatial filters 505 and 506 are each capable of outputting a center pixel signal (center), a low-pass filter output signal (LPF-L) from 3×3-tap fixed low-pass filter, and a low-pass filter output signal (LPF-LL) from 5×5-tap fixed low-pass filter.
When the input signal to the spatial filtering block 24 is a color image signal (in this example, a YCbCr color-space signal), a selector 508 selects its input terminal 5081 and a selector 509 selects its input terminal 5091. In this case, the Y-component image signal is inputted to the programmable spatial filter 504, the Cb-component image signal is inputted to the spatial filter 505, and the Cr-component image signal is inputted to the spatial filter 506. Then, the programmable spatial filter 504 and the spatial filters 505 and 506 perform spatial filtering operations in parallel.
On the other hand, when the input signal to the spatial filtering block 24 is a monochrome image signal including Y component only, the selector 508 selects its input terminal 5082 and the selector 509 selects its input terminal 5092. Then, the low-pass filter output signal (LPF-LL) of the programmable spatial filter 504 is inputted to the spatial filter 505, and the low-pass filter output signal (LPF-LL) of the spatial filter 505 is inputted to the spatial filter 506. That is, the programmable spatial filter 504 and the spatial filters 505 and 506 are connected in series (in cascade), and the cascade-connected, three spatial filters perform filtering operation. In this example, 5×5-tap low-pass filters are cascade-connected in three stages, enabling low-pass filtering with 13×13 taps.
In this way, when the input signal is a monochrome signal, the filtering operation is performed by the cascade-connected multiple spatial filters 504 to 506. The spatial filters 505 and 506 that are originally not related to the processing of monochrome signals can thus be utilized to enable filtering operation using spatial filter with a larger number of taps. Also, the processing can be done through a single pass, and so the total processing time can be shorter than when the filtering operation using the spatial filter 504 is repeated through a plurality of passes. Furthermore, because the cascade-connected spatial filters 504 to 506 are all low-pass filters, the filter coefficient distribution of the low-pass filters can be maintained even though a plurality of spatial filters are cascade-connected, and the function as low-pass filter is not damaged.
Also, with a monochrome image signal which does not need the pixel interpolation in the pixel interpolation block 21, the selectors 209 to 212 shown in
Thus, when the input signal to the pixel interpolation block 21 does not require pixel interpolation, a spatial filtering operation with an increased number of taps can be realized by using the fixed low-pass filter 502 and the spatial filters 504 to 506. Also, the processing can be done through a single pass, and so the total processing time can be shorter than when the filtering operation using the spatial filter 504 is repeated through a plurality of passes. Furthermore, since the fixed low-pass filters 501 to 503 perform filtering by using the pixel register group 201 in the pixel interpolation block 21, the circuit scale can be smaller than when a similar pixel register group is newly provided in the spatial filtering block 24. Moreover, because the cascade-connected spatial filters 502, 504 to 506 are all low-pass filters, the filter coefficient distribution of the low-pass filters can be maintained even though a plurality of spatial filters are cascade-connected, and the function as low-pass filter is not damaged.
Referring to
Next, referring to
The subtracter 510 subtracts the 3×3-tap low-pass filter output signal from the center pixel signal of the programmable spatial filter 504, so as to generate a relatively narrow-band, high-frequency component (HF-N). The subtracter 511 subtracts the 5×5-tap low-pass filter output signal from the center pixel signal of the programmable spatial filter 504, so as to generate a relatively wide-band, high-frequency component (HF-W). Accordingly, high-frequency components similar to a high-pass filter output signal can be obtained even when the programmable spatial filter 504 is set not as a high-pass filter.
The subtracter 512 subtracts the 5×5-tap low-pass filter output signal from the 3×3-tap low-pass filter output signal of the programmable spatial filter 504, so as to generate a medium-frequency component (MF). It is thus possible to generate a medium-frequency component similar to a band-pass filter output signal without using a band-pass filter that passes medium-frequency component. The medium-frequency component thus obtained can be arbitrarily utilized according to the purpose of the user, whereby the versatility can be enhanced.
The selector 513 selects and outputs one of the high-pass filter output signal of the programmable spatial filter 504, the output signal from the subtracter 510, and the output signal from the subtracter 511. The selector 514 selects and outputs one of an output signal from the programmable spatial filter 504 (particularly, the output signal provided when the programmable spatial filter 504 is set as a low-pass filter), the 3×3-tap low-pass filter output signal, and the 5×5-tap low-pass filter output signal.
As shown in
Also, the selector 515 selects and outputs one of the center pixel signal of the programmable spatial filter 504, the output signal from the selector 513, the output signal from the subtracter 512, the output signal from the selector 514, the 3×3-tap Sobel filter output signal, and the 5×5-tap Sobel filter output signal.
The multiplier 516 multiplies together the output signal (a high-frequency component) from the selector 513 and an arbitrary coefficient (RATHF) to provide an output, and the multiplier 517 multiplies together the output signal (medium-frequency component) from the subtracter 512 and an arbitrary coefficient (RATMF) to provide an output. The adder 518 adds together the output signal from the multiplier 516 and the output signal from the multiplier 517 to provide an output. The high-frequency component and medium-frequency component can thus be mixed at a desired ratio, by setting the coefficients (RATHF, RATMF) at desired values. The output signal from the adder 518 can be arbitrarily utilized according to the purpose of the user, offering enhanced versatility.
The selector 530 selects and outputs one of a coefficient (RATCNT) and a coefficient (64-Alpha). The multiplier 531 multiplies together the center pixel signal of the programmable spatial filter 504 and the output signal from the selector 530 to provide an output. The multiplier 532 multiplies together the output signal from the selector 514 and a given coefficient to provide an output. This given coefficient is set to make a constant sum (desirably, “1”) with the coefficient (64-Alpha) inputted to the selector 530. The direct-current component gain can be kept at a constant value because the sum of the coefficients is a constant value. The adder 533 adds together the output signal from the multiplier 531 and the output signal from the multiplier 532 to provide an output. It is possible, by setting the coefficient (64-Alpha) at a desired value, to obtain an output signal in which the center pixel signal and low-frequency component are mixed at a desired ratio. This output signal can be arbitrarily utilized according to the purpose, offering enhanced versatility.
The adder 535 adds together the output signal from the adder 533 and the output signal from the limiter 534 to provide an output. The output signal from the adder 533 contains direct-current component and low-frequency component, and the output signal from the limiter 534 contains high-frequency component and medium-frequency component. Accordingly, an output signal that contains direct-current component, low-frequency component, medium-frequency component, and high-frequency component is obtained by the adder 535 adding the output signal from the adder 533 and the output signal from the limiter 534. This output signal can be arbitrarily utilized according to the purpose, offering enhanced versatility.
The selector 536 selects and outputs one of the output signal from the adder 535 and the output signal from the selector 515.
The subtracter 519 subtracts the 3×3-tap low-pass filter output signal from the center pixel signal of the spatial filter 505 shown in
The selector 521 selects and outputs one of the 3×3-tap low-pass filter output signal of the spatial filter 505 and the 5×5-tap low-pass filter output signal. The selector 522 selects and outputs one of the center pixel signal of the spatial filter 505, the output signal from the subtracter 519, the output signal from the subtracter 520, and the output signal of the selector 521.
The subtracter 523 subtracts the 3×3-tap low-pass filter output signal from the center pixel signal of the spatial filter 506 shown in
The selector 525 selects and outputs one of the 3×3-tap low-pass filter output signal of the spatial filter 506 and the 5×5-tap low-pass filter output signal. The selector 526 selects and outputs one of the center pixel signal of the spatial filter 506, the output signal from the subtracter 523, the output signal from the subtracter 524, and the output signal of the selector 525.
Coring Block 25
The configuration of the coring block 25 is illustrated in
Color Suppression Block 26
The configuration of the color suppression block (signal modulation block) 26 is illustrated in
The spatial filters 739, 753 and 762 are all 5×5-tap spatial filters, and they are each capable of outputting a center pixel signal (center), a low-pass filter output signal (LPF-L) from 3×3-tap fixed low-pass filter, and a low-pass filter output signal (LPF-LL) from 5×5-tap fixed low-pass filter.
A selector 736 receives as its input a KEY signal for modulation corresponding to the color channel C3 (or a fourth-color pixel signal) from the interpolation operation unit 208 shown in
A subtracter 740 subtracts the 3×3-tap low-pass filter output signal from the center pixel signal of the spatial filter 739, so as to generate a relatively narrow-band, high-frequency component (HF-N). A subtracter 741 subtracts the 5×5-tap low-pass filter output signal from the center pixel signal of the spatial filter 739, so as to generate a relatively wide-band, high-frequency component (HF-W).
A selector 742 selects and outputs one of the center pixel signal and the high-pass filter output signal of the programmable spatial filter 504 shown in
The output signal from the selector 742 is absolutized in an absolutizing circuit 743 and inputted to one input terminal of a selector 744. The other input terminal of the selector 744 directly receives the output signal from the selector 742. The selector 744 selects and outputs one of the input signals.
A selector 750 receives as its input a pixel unit feature signal for modulation corresponding to the channel K0 (channel C4), from the interpolation operation unit 208 shown in
In this way, the spatial filter 753 applies low-pass filtering to the pixel unit feature signal corresponding to the channel K0, whereby variations among individual pixels can be suppressed. This makes it possible to avoid considerable variations in the degree of modulation of individual pixels, when modulating the Y component (color channel C0), Cb component (color channel C1), and Cr component (color channel C2) on the basis of the pixel unit feature signal corresponding to the channel K0. As a result, even when a divergence occurs between a pixel as the source of generation of the pixel unit feature signal and a pixel as the target of modulation, it is possible to lower the degree of deterioration of image quality due to luminance or color divergence from the proper value, than when the pixel unit feature signal is not low-pass-filtered.
A selector 754 selects and outputs one of the center pixel signal and the high-pass filter output signal of the programmable spatial filter 504 shown in
The output signal from the selector 754 is absolutized in an absolutizing circuit 755 and inputted to one input terminal of a selector 756. The other input terminal of the selector 756 directly receives the output signal from the selector 754. The selector 756 selects and outputs one of the input signals.
A selector 759 receives as its input a pixel unit feature signal for modulation corresponding to the channel K1 (channel C5) from the interpolation operation unit 208 shown in
In this way, the spatial filter 762 applies low-pass filtering to the pixel unit feature signal corresponding to the channel K1, whereby variations among individual pixels can be suppressed. This makes it possible to avoid considerable variations in the degree of modulation of individual pixels, when modulating the Y component, Cb component, and Cr component on the basis of the pixel unit feature signal corresponding to the channel K1.
A selector 763 selects and outputs one of the center pixel signal and the high-pass filter output signal of the programmable spatial filter 504 shown in
The output signal from the selector 763 is absolutized in an absolutizing circuit 764 and inputted to one input terminal of a selector 765. The other input terminal of the selector 765 directly receives the output signal from the selector 763. The selector 765 selects and outputs one of the input signals.
Referring to
Referring to
A selector 745 selects and outputs one of the center pixel signal of the spatial filter 739, the 3×3-tap low-pass filter output signal, and the 5×5-tap low-pass filter output signal. The output signal from the selector 745 is modulated by a lookup table 748.
Referring to
Referring to
An absolutizing circuit 708 absolutizes the output signal from the coring circuit 601 shown in
A selector 710 selects and outputs, the output signal from the lookup table 709 and the output signal from the selector 715. A selector 711 selects and outputs a given coefficient and a modulation parameter sent from the SDRAM 11 through the input DMA channel IN-CH2 or IN-CH3. A selector 712 selects and outputs the output signal from the selector 716 and a modulation parameter sent from the SDRAM 11 through the input DMA channel IN-CH2 or IN-CH3.
A selector 723 selects and outputs the output signal from the lookup table 717 and the output signal from the selector 726. A selector 730 selects and outputs the output signal from the lookup table 717 and the output signal from the selector 733.
Referring to
Referring to
A multiplier 727 multiplies together the output signal from the selector 731 and the output signal from the coring circuit 603 corresponding to Cr component, shown in
In this way, the color suppression block 26 is capable of arbitrarily modulating the pixel unit feature signals that are modulating signals for modulating signals to be modulated (luminance signal high-frequency component and medium-frequency component, or color signals), by using the absolutizing circuits 751, 755, 760, 764, the luminance modulation units 757, 766, and the lookup tables 758, 767 shown in
Second Color-Space Transformation Block 27
The configuration of the second color-space transformation block 27 is illustrated in
The color-space transformation circuit 801 is capable of performing matrix operation of 4-channel inputs and 4-channel outputs. For example, it transforms a YCbCr color-space (YUV color-space) image signal into an RGB color-space or YMCW color-space image signal and outputs it. It is also capable of outputting a YCbCr color-space image signal without transforming it, when a coefficient for the color-space transformation circuit 801 is set to such a value as not to effect color-space transformation.
In this way, the spatial filtering block 24, the coring block 25, and the color suppression block 26 perform various operations using YCbCr color-space image signals, and then the second color-space transformation block 27 transforms them into, e.g., the RGB color space, whereby an output signal of a desired color space is finally obtained and the versatility is thus enhanced.
Second Gamma-Correction Block 28
The configuration of the second gamma-correction block 28 is illustrated in
In this way, the first color-space transformation block 23, the spatial filtering block 24, the coring block 25, the color suppression block 26, and the second color-space transformation block 27 perform various operations using 12-bit-system image signals, and then the second gamma-correction block 28 performs gamma correction to convert them to 8-bit-system image signals. Thus, an output signal of a desired number of bits is finally obtained and the versatility is thus enhanced.
Resizing Block 29
The configuration of the resizing block 29 is illustrated in
The image signals that have been gamma-corrected in the gamma correction unit 901 are inputted to the resizers 1001 to 1004 through selection by a selector 1005. Then, the resizers 1001 to 1004 perform resolution conversion, and the signals are transferred through the output DMA channels OUT-CH0 to OUT-CH3 for storing in the SDRAM 11 or display in the LCD 7. In this way, when the input signal to the resizing block 29 is of 8-bit system, not only the resizers 1002 to 1004 but also the resizer 1001 are used to perform resolution conversion. Accordingly, the resizer 1001 is not wasted when the input signal is of 8-bit system.
An image signal that is not subjected to gamma correction in the gamma correction unit 901 is inputted from the color-space transformation circuit 801 to the resizer 1001 through the selector 1005. Then, the resolution is converted in the resizer 1001 and it is stored in the SDRAM 11 or recorded in the memory card 13 through the output DMA channel OUT-CH0.
In this way, while a 12-bit-system signal for main image and an 8-bit system signal for display or for thumbnail image are inputted as input signals to the resizing block 29, it is possible to select which resizer or resizers are to be used according to the number of bits of the input signal, among the resizer 1001 and the resizers 1002 to 1004 configured to process signals of different numbers of bits. Furthermore, the circuit scale can be smaller than when all resizers 1001 to 1004 are provided as resizers capable of processing 12-bit-system input signals.
Two-Dimensional Lookup Table 402
Now, the two-dimensional lookup table 402 shown in
In the two-dimensional lookup table, mesh intersections V(p, q) are defined at intervals of 256 digits both in vertical and horizontal directions. The values p and q are integers of not less than −8 nor more than 8. Transform values (x, y), representing output values for data X and Y, are set at each intersection V(p, q). For example, when the input values to the two-dimensional lookup table are (X, Y)=(256, 512), the data is inputted to the intersection V(1, 2) and the transform values (x, y)=(256, 512) set at that intersection V(1, 2) are outputted as the output values from the two-dimensional lookup table. The input values (X, Y) and the output values (x, y) are equal in this example because
In the two-dimensional lookup table, the transform values (x, y) are discretely defined only at the mesh intersections V(p, q), and so the circuit scale can be smaller than when transform values are defined in correspondence with all input values.
The first color-space transformation block 23 performs color-space transformation by real-time processing. Accordingly, when the color-space transformation is performed using a three-dimensional lookup table that involves large amounts of calculations, then the processing in the three-dimensional lookup table will form a bottleneck to delay the entire processing. Also, a three-dimensional lookup table involves a very large circuit scale. In contrast, using a two-dimensional lookup table avoids increased circuit scale and achieves high-speed color-space transformation without delaying the entire processing.
When the input data has values not defined as an intersection V(p, q), the transform values corresponding to the input values can be calculated by interpolation using the transform values at the four intersections surrounding the input values.
When input values (Xin, Yin) are surrounded by four intersections V0(X0, Yin0), V1(X0, Yin1), V2(X1, Yin0), and V3(X1, Yin1), then vector data about the intersection V0, (DT0X(X0, Yin0), DT0Y(X0, Yin0)), vector data about the intersection V1, (DT1X(X0, Yin1), DT1Y(X0, Yin1)), vector data about the intersection V2, (DT2X(X1, Yin0), DT2Y(X1, Yin0)), and vector data about the intersection V3, (DT3X(X1, Yin1), DT3Y(X1, Yin1)), are obtained. Where X0≦Xin<X1 or X0<Xin≦X1, Yin0≦Yin<Yin1 or Yin0<Yin≦Yin1.
Then, the transform values (Xout, Yout) corresponding to the input values (Xin, Yin) are calculated by the interpolation shown below.
In the two-dimensional lookup table 402 shown in
Since the YCbCr color space only involves two-channels of color signals, it is possible to transform all two-channel color signals outputted from the color-space transformation circuit 401 by using the two-dimensional lookup table 402, without a need to use a three-dimensional lookup table.
While the invention has been described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is understood that numerous other modifications and variations can be devised without departing from the scope of the invention.
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
6226397, | Jul 22 1996 | Canon Kabushiki Kaisha | Image processing using different image sizes depending on whether the processing is color or monochrome |
7218418, | Jul 01 2002 | Xerox Corporation | Digital de-screening of documents |
7554583, | Nov 04 2004 | Sprint Spectrum LLC | Pixel signal processor and pixel signal processing method |
7719575, | Dec 22 2003 | Mitsubishi Denki Kabushiki Kaisha | Pixel signal processing apparatus and pixel signal processing method |
20010045923, | |||
20020126209, | |||
20040105016, | |||
20080043131, | |||
JP1042150, | |||
JP2000236473, | |||
JP2003198836, | |||
JP2003283815, | |||
JP200440790, | |||
JP983823, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Apr 02 2007 | SASAKI, GEN | MEGACHIPS CORPORATION | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 019146 | /0405 | |
Apr 11 2007 | MEGACHIPS CORPORATION | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Date | Maintenance Schedule |
Jun 21 2014 | 4 years fee payment window open |
Dec 21 2014 | 6 months grace period start (w surcharge) |
Jun 21 2015 | patent expiry (for year 4) |
Jun 21 2017 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jun 21 2018 | 8 years fee payment window open |
Dec 21 2018 | 6 months grace period start (w surcharge) |
Jun 21 2019 | patent expiry (for year 8) |
Jun 21 2021 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jun 21 2022 | 12 years fee payment window open |
Dec 21 2022 | 6 months grace period start (w surcharge) |
Jun 21 2023 | patent expiry (for year 12) |
Jun 21 2025 | 2 years to revive unintentionally abandoned end. (for year 12) |