An image processing method for increasing the stereoscopic effect of an image is provided. difference mask data including an object area and a background area is created from image data, and the center coordinate of the object area is calculated. Then, a gradation pattern is selected in accordance with the average brightness value of the object area and applied to the background area, whereby a gradation mask data is created. After that, the image data of the background area is converted into image data based on the gradation mask data, so that the stereoscopic effect of the image is increased.
|
9. A method for processing image data comprising the steps of:
creating binary difference mask data including a first area corresponding to an object existing in image data and a second area corresponding to the other area in the image data, wherein the first area is represented by white and the second area is represented by black;
calculating center coordinate data of the first area;
calculating an average brightness value of the first area;
selecting a gradation pattern in accordance with the average brightness value of the first area;
creating gradation mask data by applying the gradation pattern to the second area with a center coordinate of the first area as its center; and
converting the image data using the gradation mask data.
1. A method for processing image data comprising the steps of:
calculating a difference between continuous frames of image data;
creating binary difference mask data including a first area corresponding to an object existing in n-th frame image data and a second area corresponding to the other area in the n-th frame image data, wherein the first area is represented by white and the second area is represented by black;
calculating center coordinate data of the first area;
calculating an average brightness value of the first area;
selecting a gradation pattern based on the average brightness value;
creating gradation mask data by applying the gradation pattern to the second area with a center coordinate of the first area as its center; and
converting the image data using the gradation mask data.
5. A method for processing image data comprising the steps of:
creating difference image data between a (n−1)-th frame and a n-th frame, and between the n-th frame and a (n+1)-th frame of image data, the difference image data being black-and-white binary data;
creating difference mask data including a first area corresponding to an object existing in n-th frame image data and a second area corresponding to the other area in the n-th frame image data, wherein the first area is represented by white and the second area is represented by black;
calculating center coordinate data of the first area;
calculating an average brightness value of the first area;
selecting a gradation pattern based on the average brightness value;
creating gradation mask data by applying the gradation pattern to the second area with a center coordinate of the first area as its center; and
converting the image data using the gradation mask data.
2. The method according to
3. The method according to
4. The method according to
6. The method according to
7. The method according to
8. The method according to
10. The method according to
11. The method according to
12. The method according to
|
|||||||||||||||||||||||||||||||
1. Field of the Invention
The present invention relates to an image processing method. The present invention also relates to an image processing program.
2. Description of the Related Art
A variety of display devices have come onto the market, ranging from large-size display devices such as television receivers to small-size display devices such as cellular phones. From now on, the display devices will be expected to have higher added values, and development has been advanced. In recent years, display devices capable of displaying stereoscopic images have been actively developed to provide more realistic images.
Many of the display devices capable of displaying stereoscopic images utilize binocular parallax. The method utilizing binocular parallax mostly uses, for example, special glasses for delivering different images to the left and the right eyes of a viewer.
Some of the methods utilizing binocular parallax do not require special glasses: a display portion is additionally provided with a mechanism (e.g., a parallax barrier, a lenticular lens, or a microlens array) for delivering different images to the left and the right eyes of a viewer, so that stereoscopic images can be seen by a naked eye (see Patent Document 1, for example).
The aforementioned display devices capable of displaying stereoscopic images by utilizing binocular parallax, needs an additional mechanism of a display portion or special glasses. Accordingly, an increase in manufacturing cost and power consumption becomes serious.
Other than the binocular parallax, physiological factors allow the human eye to perceive stereoscopic images psychologically or empirically. The physiological factors include the size of an object, the arrangement of an object, contrast, shade, clarity, and the like. These sensuous factors for perceiving stereoscopic images are advantageous in that a display portion does not require an additional mechanism; however, they are already provided to a two-dimensional image. Therefore, a method for further increasing a stereoscopic effect is needed.
Thus, an object of one embodiment of the present invention is to provide an image processing method which is a simple method for further increasing the stereoscopic effect of a two-dimensional image.
One embodiment of the present invention is an image processing method for further increasing the stereoscopic effect of two-dimensional moving image data having a plurality of frames. First, difference mask data is created by dividing image data into an object area and a background area. Then, the center coordinate data of each object area is calculated. Next, a gradation pattern is selected in accordance with the average brightness value of the object area, and gradation mask data is created by applying the gradation pattern to the background area. After that, original image data of the background area is converted based on the gradation mask data, whereby the stereoscopic effect of the image can be increased.
One embodiment of the present invention is an image processing method in which moving image data having a plurality of frames is processed with an arithmetic unit. A difference of image data between frames is calculated to create binary difference mask data including an object area and a background area. The center coordinate data of the object area is calculated and the average brightness value of the object area is calculated. Gradation mask data is created by applying a gradation pattern based on the average brightness value to the background area. The image data is converted into image data based on the gradation mask data.
In the image processing method of one embodiment of the present invention, the difference mask data is preferably created based on difference image data that is obtained from the difference in the brightness value of image data between the (n−1)-th frame and the n-th frame, and between the n-th frame and the (n+1)-th frame.
In the image processing method of one embodiment of the present invention, the center coordinate data is preferably obtained by labeling processing.
In the image processing method of one embodiment of the present invention, the average brightness value is preferably obtained by dividing the sum of the brightness values of pixels in the object area by the number of pixels in the object area.
In the image processing method of one embodiment of the present invention, the gradation pattern is preferably selected from the following in accordance with the brightness value: a pattern in which a brightness value decreases outward from the center coordinate; and a pattern in which a brightness value increases outward from the center coordinate.
In the image processing method of one embodiment of the present invention, image data is converted in accordance with gradation mask data, and the brightness value of the image data is preferably used in the object area while the average of the brightness value of the image data and the brightness value of the gradation mask data is preferably used in the background area.
According to one embodiment of the present invention, it is possible to provide an image processing method which is a simple method for further increasing the stereoscopic effect of a two-dimensional image.
In the accompanying drawings:
FIGS. 8A1 to 8D are views showing Embodiment 1;
FIGS. 10A1 to 10C2 are views showing Embodiment 1;
The present invention will be described below with reference to drawings. Note that the present invention can be implemented in a lot of different ways, and it is apparent to those skilled in the art that modes and details can be modified in a wide variety of ways without departing from the spirit and scope of the present invention. Therefore, the present invention is not interpreted as being limited to the description of the embodiment.
A display device of one embodiment of the present invention will be described with reference to
As shown in
The image data processing portion 11 shown in
The input portion 13 is an interface for converting the format of the image data input to the display device 10 into the format for the display device 10.
In the description, the image data input to the input portion 13 is moving image data having a plurality of frames; however, it may be still image data. Further, the image data may correspond to either a full-color image or a monochrome image. In the case of corresponding to a full-color image, the image data includes pieces of image data for the respective hues.
Note that in this specification, a full-color image refers to an image displayed by using the gray scale of a plurality of colors with different hues. Meanwhile, a monochrome image refers to an image displayed by using the gray scale of colors with a hue.
The arithmetic unit 14 has a function of collectively controlling the operation of the input portion 13, the memory unit 15, and the display controller 16 included in the image data processing portion 11. Furthermore, in the arithmetic unit 14, image data is subjected to image processing by various kinds of arithmetic operations such as logic operations and four arithmetic operations, whereby image data after conversion can be obtained.
The memory unit 15 has a function of temporarily storing image data whose format has been converted in the input portion 13. Specifically, moving image data having a plurality of frames is temporarily stored in the memory unit 15 and used for the arithmetic operation in the arithmetic unit 14. In addition, the memory unit 15 can temporarily store data used for the arithmetic operation in the arithmetic unit 14. The memory unit 15 can also temporarily store image data converted in the arithmetic unit 14. Furthermore, the memory unit 15 can store instruction (program) executed in the arithmetic unit 14. The instruction (program) to be executed in the arithmetic unit 14, which is stored in the memory unit 15, can be executed when read out in the arithmetic unit 14.
With the use of the image data that has been subjected to image processing, the display controller 16 generates an image signal which meets the specifications of the display portion 12. The generated image signal is supplied to the display portion 12. The display controller 16 also has a function of supplying the display portion 12 with a power supply potential or a driving signal for controlling driving of the display portion 12, such as a clock signal or a start pulse signal.
The display portion 12 includes a pixel portion 17 for displaying an image with the use of an image signal, and driver circuits 18 for controlling operation of the pixel portion 17. For the pixel portion 17, the following devices for displaying an image by controlling the gray scale of each pixel can be used: a liquid crystal display device, a light-emitting device including a light-emitting element such as an organic light-emitting element (OLED), electronic paper, digital micromirror device (DMD), plasma display panel (PDP), field emission display (FED), and the like.
Next, description is made on the image processing method of one embodiment of the present invention, which is performed in the display device 10 shown in
First, a difference of image data between frames is calculated to create binary difference mask data including an object area and a background area (step S11: difference mask data creation).
Note that the object area refers to a part of one frame image, which is occupied by an object to which a viewer pays attention, here, a moving object. The background area refers to the other part of one frame image, namely, the area other than the object area.
Note that the difference mask data is obtained in such a manner that each pixel in one frame image is represented by binary data. Specifically, the difference mask data corresponds to data of one frame in which pixels in the object area are each represented by ‘1’ and pixels in the background area are each represented by ‘0’. In the case where the difference mask data in one frame is made visible, for example, the object area can be represented by white and the background area can be represented by black.
Then, the center coordinate data of the object area is calculated (step S12: center coordinate data creation)
Next, the average brightness value of the object area is calculated to create gradation mask data, which is obtained by replacing the difference mask data corresponding to the background area with data having a gradation pattern based on the average brightness value (step S13: gradation mask data creation).
Note that the average brightness value is obtained by dividing the sum of the brightness values of pixels in the object area by the number of pixels in the object area. The “brightness” here refers to the brightness of a pixel, and the brightness value may be referred to as a grayscale level. In the case where three pixels having hues of red (R), green (G), and blue (B) are used to display one color, each color of RGB is weighted and the brightness values of the three pixels are added to obtain a brightness value. Weighted brightness value of RGB is obtained by multiplying weighted brightness values of the respective colors and adding them. Specifically, in NTSC system, the brightness value S of a pixel is represented by the equation S=0.30RG+0.59GG+0.11BG where RG denotes the brightness value of R, GG denotes the brightness value of G, and BG denotes the brightness value of B. Note that hues other than RGB may also be used.
Instead of the average brightness value, the intermediate value between the maximum brightness value and the minimum brightness value can also be used.
Note that the gradation pattern means a pattern in which a continuous change from white to black or from black to white occurs in one direction or plural directions.
FIG. 10A1 shows an example of a concentric gradation pattern in which a continuous change from black to white occurs from the center coordinate (x in FIG. 10A1). The brightness value of the gradation pattern may be controlled as appropriate as shown in FIG. 10A2 in which dotted line XY denotes the horizontal direction: the brightness value sinks downwards at the center coordinate and the maximum brightness value is obtained in the periphery. Note that a continuous change of the brightness value from black to white in the gradation pattern may be a linear change. Alternatively, a continuous change from black to white in the gradation pattern may expand from the center coordinate in a rectangular shape with a width as shown in
In the case where the center coordinate to which the gradation pattern is applied is at the end of a screen, the brightness value may be controlled as appropriate as shown in FIGS. 10C1 and 10C2 in a manner similar to that in FIGS. 10A1 and 10A2: the brightness value sinks downwards at the center coordinate and the maximum brightness value is obtained in the periphery.
In the case where a plurality of center coordinates exist, the gradation pattern may be formed so that continuous changes from the center coordinates do not overlap each other as shown in
Note that the gradation mask data is data having a gradation pattern with the object area as its center. The gradation mask data is obtained in such a manner that the difference mask data of the background area represented by black is replaced with data based on the aforementioned gradation pattern.
Then, the image data is converted into image data based on the gradation mask data (step S14: image data conversion).
Each of the aforementioned steps S11 to S14 will be specifically described with reference to the flow charts of
In the flow charts showing the steps S11 to S14, description is made on an example of converting image data in the n-th frame (n is a natural number of two or more) among the continuous frames: (n−1)-th frame, n-th frame, and (n+1)-th frame. Note that the image data in the (n−1)-th frame, the n-th frame, and the (n+1)-th frame can be made visible as shown in FIGS. 8A1 to 8A3, respectively, for example. In FIGS. 8A1 to 8A3, an object moves in the successive frames. In FIGS. 8A1 to 8A3, the object is denoted by a solid line 200, and the objects in the (n−1)-th frame and the (n+1)-th frame are denoted by dotted lines 201 and 202, respectively.
First, the difference in the brightness value of a pixel (x, y) in image data is calculated between the (n−1)-th frame and the n-th frame, and between the n-th frame and the (n+1)-th frame (step S21). A difference in brightness value may be calculated in the following manner: in the case where, for example, the brightness value of RGB in the (n−1)-th frame is (255, 255, 255) and the brightness value of RGB in the n-th frame is (125, 125, 125), (130, 130, 130) is obtained as a difference in brightness value and then, the brightness value of each color is weighted and calculated.
Next, it is determined whether the difference in the brightness value calculated in the step S21 is equal to or higher than a predetermined threshold brightness value (step S22). The threshold brightness value may be set as appropriate, and for example, may be set to an intermediate value between the maximum brightness value and the minimum brightness value.
When the difference in the brightness value calculated in the step S21 is determined in the step S22 to be equal to or higher than the predetermined threshold brightness value, difference image data corresponding to the pixel (x, y) is set to ‘0’, namely, black (step S23). Note that the difference image data is temporary data for creating binary difference mask data including an object area and a background area.
When the difference in the brightness value calculated in the step S21 is determined in the step S22 to be lower than the predetermined threshold brightness value, difference image data corresponding to the pixel (x, y) is set to ‘1’, namely, white (step S24).
Note that in the case where the difference image data is determined to be black in the step S23, among the image data for one frame, the brightness value of the pixel (x, y) in which the difference image data is determined to be black is added (step S25). Further, a counter is incremented to count the number of pixels (x, y) in which the difference image data is set to black in the image data for one frame (step S26). Note that the accumulated data of the brightness values added in the step S25, and the value counted in the step S26 are used for selecting a gradation pattern in a later step. It is to be noted that the steps S23 to S26 and the step S24 are performed alternately.
After the steps S24 and S26, it is determined whether the calculation of the difference in brightness value is completed for all the pixels for one frame (step S27). If the calculation of the difference in brightness value is not completed for all the pixels for one frame, the step S21 is restarted for another pixel.
When the calculation of the difference in brightness value is completed for all the pixels for one frame in the step 27, finished is creation of the difference image data between the (n−1)-th frame and the n-th frame, and between the n-th frame and the (n+1)-th frame, which is black-and-white binary data (step S28). Note that the difference image data between the (n−1)-th frame and the n-th frame of the image data shown in FIGS. 8A1 and 8A2 can be made visible as shown in
Following A after the step S28, comparison is performed in a pixel (x, y) between the difference image data between the (n−1)-th frame and the n-th frame, and the difference image data between the n-th frame and the (n+1)-th frame (step S29).
Next, it is determined whether both of the black-and-white binary difference image data compared in the step S29 are black (step S30).
If the difference image data compared in the step S29 are determined to be black in the step S30, difference mask data corresponding to the pixel (x, y) is set to ‘1’, namely, white (step S31).
Meanwhile, when neither of the difference image data compared in the step S29 are determined to be black in the step S30, difference mask data corresponding to the pixel (x, y) is set to ‘0’, namely, black (step S32).
After the steps S31 and S32, it is determined whether the comparison of the difference image data is completed for all the pixels for one frame (step S33). If the comparison of the difference image data is not completed for all the pixels for one frame, the step S29 is restarted for another pixel.
When the comparison of the difference image data is completed for all the pixels for one frame in the step S33, creation of the difference mask data which is binary data is finished (step S34). In the difference mask data, an object area is represented by white and a background area is represented by black. Note that the binary difference mask data corresponding to the difference image data shown in
That is the flow chart showing an example of the difference mask data creation in the step S11. Note that one embodiment of the present invention is not limited to the processing shown in the flow charts of
First, labeling of difference mask data is performed for an object area, whereby center coordinate data is specified (step S41).
Then, it is determined whether another object area to be processed in the step S41 exists (step S42).
If the labeling of the difference mask data is completed for all the object areas in the step S42, detection of the center coordinate data of the object areas is finished (step S43). When the labeling of the difference mask data is not completed for all the object areas, the step S41 is restarted for another object area. Note that the center coordinate data of the object area, which is obtained by labeling of the difference mask data shown in
That is the flow chart showing an example of the center coordinate data creation in the step S12. Note that one embodiment of the present invention is not limited to the processing shown in the flow chart of
First, the sum of the brightness values of pixels corresponding to an object area is divided by the count number of a counter, whereby the average brightness value in the object area is calculated (step S51). Note that the values obtained in the aforementioned steps S25 and S26 are used as the sum of the brightness values of the pixels corresponding to the object area and the count number.
In the case where the intermediate brightness value is used instead of the average brightness value, the aforementioned step S51 for calculating the average brightness value is not performed, and the intermediate brightness value may be calculated. In the case of using the intermediate brightness value, the aforementioned steps S25 and S26 can be omitted.
Next, it is determined whether the average brightness value obtained in the step S51 is equal to or lower than the threshold value (step S52). A threshold average brightness value may be set as appropriate, and for example, may be set to an intermediate value between the maximum brightness value and the minimum brightness value.
When the average brightness value calculated in the step S51 is determined in the step S52 to be equal to or higher than a predetermined threshold average brightness value, a gradation pattern, in which the brightness value increases outward from the center coordinate, is selected (step S53).
When the average brightness value calculated in the step S51 is determined in the step S52 to be lower than the predetermined threshold average brightness value, a gradation pattern, in which the brightness value decreases outward from the center coordinate, is selected (step S54).
When the selection of a gradation pattern is completed in the steps S53 and S54, the selected gradation pattern is applied to the background area with the center coordinate of the object area as its center, whereby creation of the gradation mask data is finished (step S55). Application of the gradation pattern in which the brightness value increases outward from the center coordinate to the background area of the difference mask data shown in
That is the flow chart showing an example of the gradation mask data creation in the step S13. Note that one embodiment of the present invention is not limited to the processing shown in the flow chart of
First, it is determined whether a pixel (x, y) in the n-th frame image data corresponds to a background area in the gradation mask data (step S61).
When the pixel (x, y) in the n-th frame image data is determined in the step S61 to correspond to the background area in the gradation mask data, the average of the brightness value of the image data for the pixel (x, y) and the brightness value of the gradation mask data corresponding to the pixel (x, y), is calculated and used as a brightness value of image data after image processing (step S62).
When the pixel (x, y) in the n-th frame image data is determined in the step S61 to not correspond to the background area in the gradation mask data, that is, correspond to an object area, the brightness value of the image data for the pixel (x, y) before image processing is used without change as a brightness value of image data after image processing (step S63).
Then, it is determined whether the image processing in the step S61 is completed for all the pixels (step S64).
When the image processing is determined in the step S64 to be completed for all the pixels, the image processing of the n-th frame image data is completed (step S65). If the image processing is not completed for all the pixels in the step S64, the step S61 is restarted for another pixel. Note that the image processing using the gradation mask data, which is applied to the n-th frame image data shown in FIG. 8A2, can be made visible as shown in
That is the flow chart showing an example of the image data conversion in the step S14. Note that one embodiment of the present invention is not limited to the processing shown in the flow chart of
As described above, a gradation pattern is applied to a background area in the structure of this embodiment, whereby a higher stereoscopic effect can be given to data of a two-dimensional moving image with a plurality of frames. Thus, unlike a display device capable of displaying stereoscopic images by utilizing binocular parallax, an additional mechanism of a display portion or special glasses are not needed, resulting in a reduction in manufacturing cost and power consumption.
In this example, an image obtained by using the image processing method of one embodiment of the present invention will be described with reference to
The image data of
This application is based on Japanese Patent Application Ser. No. 2011-189538 filed with Japan Patent Office on Aug. 31, 2011, the entire contents of which are hereby incorporated by reference.
| Patent | Priority | Assignee | Title |
| Patent | Priority | Assignee | Title |
| 5732204, | Feb 01 1994 | Ricoh Company, Ltd. | Method and device for 3D image processing |
| 6064391, | Nov 28 1990 | Hitachi, Ltd. | Method for displaying region extracting processing in an image processing system |
| 6157733, | Apr 18 1997 | AT&T Corp | Integration of monocular cues to improve depth perception |
| 7348978, | Jun 22 2004 | Kabushiki Kaisha Sega | Image processing |
| 7471849, | Aug 28 2001 | Adobe Inc | Methods and apparatus for shifting perspective in a composite image |
| 7664338, | Sep 28 2004 | Qualcomm Incorporated | Perspective transformation of two-dimensional images |
| 8687918, | Mar 05 2008 | Semiconductor Energy Laboratory Co., Ltd. | Image processing method, image processing system, and computer program |
| 20040105493, | |||
| 20040220466, | |||
| 20070086675, | |||
| 20080143716, | |||
| 20080247655, | |||
| 20090128478, | |||
| 20090136158, | |||
| 20090225183, | |||
| 20090324074, | |||
| 20100097447, | |||
| 20110221756, | |||
| JP2000512833, | |||
| JP2003259395, | |||
| JP2005258013, | |||
| JP2009147922, | |||
| JP2009239903, | |||
| JP8036145, | |||
| WO2009066590, | |||
| WO2009066783, | |||
| WO9848381, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Oct 23 2014 | Semiconductor Energy Laboratory Co., Ltd. | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Jun 07 2017 | ASPN: Payor Number Assigned. |
| Sep 10 2020 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
| Date | Maintenance Schedule |
| Mar 21 2020 | 4 years fee payment window open |
| Sep 21 2020 | 6 months grace period start (w surcharge) |
| Mar 21 2021 | patent expiry (for year 4) |
| Mar 21 2023 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Mar 21 2024 | 8 years fee payment window open |
| Sep 21 2024 | 6 months grace period start (w surcharge) |
| Mar 21 2025 | patent expiry (for year 8) |
| Mar 21 2027 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Mar 21 2028 | 12 years fee payment window open |
| Sep 21 2028 | 6 months grace period start (w surcharge) |
| Mar 21 2029 | patent expiry (for year 12) |
| Mar 21 2031 | 2 years to revive unintentionally abandoned end. (for year 12) |