A gesture recognition module, for recognizing a gesture of a user, includes a detecting unit, for capturing at least one hand image of a hand of the user, so as to sequentially acquire a first coordinate and a second coordinate; a computing unit, coupled to the detecting unit for defining a first zone and a second zone according to the first coordinate and the second coordinate, respectively, and calculating a first area and a second area according to the first zone and the second zone; and a determining unit, coupled to the detecting unit and the computing unit for recognizing the gesture according to the first coordinate, the second coordinate, the first area and the second area.
|
12. A gesture recognition method for a gesture recognition module, the gesture recognition method comprising:
capturing at least one hand image of a hand of a user, to sequentially acquire a first coordinate and a second coordinate;
defining a first zone and a second zone according to the first coordinate and the second coordinate;
calculating a first area and a second area according to the first zone and the second zone; and
recognizing the gesture according to the first coordinate, the second coordinate, the first area and the second area;
wherein the step of calculating the first area and the second area according to the first zone and the second zone comprises:
acquiring a minimum depth coordinate among a plurality of pixel areas in the first zone as a hand start depth coordinate,
adding the hand start depth coordinate and a depth for acquiring a hand end depth coordinate; and
accumulating a plurality of pixel areas between the hand start depth coordinate and the hand end depth coordinate for generating the first area.
1. A gesture recognition module, for recognizing a gesture of a user, the gesture recognition module comprising:
a detecting unit, for capturing at least one hand image of a hand of the user, so as to sequentially acquire a first coordinate and a second coordinate;
a computing unit coupled to the detecting unit, the computing unit for defining a first zone and a second zone according to the first coordinate and the second coordinate, respectively, and calculating a first area and a second area according to the first zone and the second zone; and
a determining unit coupled to the detecting unit and the computing unit, the determining unit for recognizing the gesture according to the first coordinate, the second coordinate, the first area and the second area
wherein after the computing unit acquires a minimum depth coordinate among a plurality of pixel areas in the first zone as a hand start depth coordinate, the computing unit adds the hand start depth coordinate and a depth for acquiring a hand end depth coordinate and accumulates a plurality of pixel areas between the hand start depth coordinate and the hand end depth coordinate for generating the first area.
2. The gesture recognition module of
3. The gesture recognition module of
4. The gesture recognition module of
5. The gesture recognition module of
6. The gesture recognition module of
when a difference between the length of a horizontal line of the plurality of horizontal lines corresponding to a first vertical coordinate and the length of a horizontal line corresponding of the plurality of horizontal lines to a second vertical coordinate is greater than a length threshold and the length of the horizontal line of the plurality of horizontal lines corresponding to the second vertical coordinate is smaller than or equal to a length of the horizontal line of the plurality of horizontal lines corresponding to a third vertical coordinate, the detecting unit defines the second vertical coordinate as a turning vertical coordinate and defines a pixel with the highest vertical coordinate in the first hand image as a start pixel; and the detecting unit generates the first coordinate according to the start pixel and the turning vertical coordinate; wherein the first vertical coordinate, the second vertical coordinate and the third vertical coordinate are contiguous vertical coordinates.
7. The gesture recognition module of
8. The gesture recognition module of
9. The gesture recognition module of
10. The gesture recognition module of
11. The gesture recognition module of
13. The gesture recognition method of
capturing at least one 3D image and at least one 2D image of the user's face, for detecting a face coordinate;
acquiring a hand image start depth coordinate via subtracting a first depth from a face depth coordinate of the face coordinate and acquiring a hand image end depth coordinate via subtracting a second depth from the face depth coordinate; and
detecting a plurality of pixel areas corresponding to a plurality of depth coordinates between the hand image start depth coordinate and the hand image end depth coordinate for generating the hand images.
14. The gesture recognition method of
selecting a pixel area with the greatest area among the plurality of pixel areas as the hand image, wherein the area of the pixel area is greater than an area threshold.
15. The gesture recognition method of
acquiring a first characteristic vector and a second characteristic vector of a first hand image according to the first hand image, wherein the first characteristic vector is orthogonal to the second characteristic vector and a length of the first characteristic vector is greater than a length of the second vector; and
rotating the first hand image according to the first vector for restoring the first vector to be perpendicular to a horizontal area of the first hand image.
16. The gesture recognition method of
calculating lengths of a plurality of horizontal line corresponding to a plurality of vertical coordinates in a first hand image of the hand images according to the first hand image; and
acquiring a first vertical coordinate as a hand vertical coordinate of the first coordinate, acquiring a horizontal coordinate of a middle point of a horizontal line corresponding to the first vertical coordinate as a hand horizontal coordinate of the first coordinate and acquiring a depth coordinate of the first hand image as a hand depth coordinate of the first coordinate when a difference between the length of a horizontal line of the plurality of horizontal lines corresponding to the first vertical coordinate and the length of a horizontal line of the plurality of horizontal lines corresponding to a second vertical coordinate is greater than a length threshold;
wherein the first vertical coordinate and the second vertical coordinate are contiguous vertical coordinates.
17. The gesture recognition method of
calculating lengths of a plurality of horizontal line corresponding to a plurality of vertical coordinates in a first hand image according to the first hand image;
when a difference between the length of a horizontal line of the plurality of horizontal lines corresponding to a first vertical coordinate and the length of a horizontal line of the plurality of horizontal lines corresponding to a second vertical coordinate is greater than a length threshold and the length of the horizontal line of the plurality of horizontal lines corresponding to the second vertical coordinate is smaller than or equal to the length of a horizontal line of the plurality of horizontal lines corresponding to a third vertical coordinate, defining the second vertical coordinate as a turning vertical coordinate and defining a pixel with the highest vertical coordinate in the first hand image as a start pixel; and
generating the first coordinate according to the start pixel and the turning vertical coordinate;
wherein the first vertical coordinate, the second vertical coordinate and the third vertical coordinate are contiguous vertical coordinates.
18. The gesture recognition method of
calculating a middle point of a virtual vertical line between the start pixel and the horizontal line corresponding to the turning vertical coordinate, for acquiring a vertical coordinate and a horizontal coordinate of the middle point as a hand vertical coordinate and a hand horizontal coordinate of the first coordinate, respectively; and
acquiring a depth coordinate of the first hand image as a hand depth coordinate of the first coordinate.
19. The gesture recognition method of
calculating a barycentre of a pixel area located between the vertical coordinate of the start pixel and the turning vertical coordinate in the first hand image, for acquiring a vertical coordinate and a horizontal coordinate of the barycentre as a hand vertical coordinate and a hand horizontal coordinate of the first coordinate, respectively; and
acquiring a depth coordinate of the first hand image as a hand depth coordinate of the first coordinate.
20. The gesture recognition method of
21. The gesture recognition method of
determining the gesture is a grabbing gesture when a distance between the first coordinate and the second coordinate is smaller than a moving threshold and the difference between the first area and the second area is greater than a grabbing threshold.
22. The gesture recognition method of
acquiring a first dragging ratio according to the second area and the hand depth coordinate of the second coordinate when determining the gesture is the grabbing gesture;
acquiring a third coordinate;
acquiring a third area according to the third coordinate;
acquiring a second dragging ratio according to the third area and the hand depth coordinate of the third coordinate; and
determining the user performs a dragging gesture when a difference between the first dragging ratio and the second dragging ratio is smaller than a dragging threshold.
|
1. Field of the Invention
The present application relates to a gesture recognition module and a gesture recognition method, and more particularly, to a gesture recognition module and a gesture recognition method capable of accurately recognizing whether a hand of a user performs a grabbing gesture.
2. Description of the Prior Art
Electronic devices may utilize three dimensional (3D) images or two dimensional (2D) images to recognize gestures of a user. In comparison with the 2D images, the 3D images comprise depth information. When the electronic devices utilize the 3D images for recognizing gestures, the electronic devices can use the depth information to exclude unnecessary image information, which increases the accuracy of recognizing the gestures. Since the details of 3D images are fuzzier than those of 2D images, however, it is difficult for an electronic device to use 3D images as references when recognizing gestures. For example, when an electronic device captures images of a user's fist, the greater clarity of the 2D images can show details such as the fingers of the user whereas the 3D images can only show the shape of the fist. Thus, if the ability of the electronic device to recognize details in a 3D image can be improved, the accuracy of using 3D images to recognize the gestures of a user can be increased.
The present application discloses a gesture recognition module and a gesture recognition method capable of recognizing gestures of a user via calculating area variations of the user's hand in 3D images.
The present invention discloses a gesture recognition module, for recognizing a gesture of a user, comprising: a detecting unit, for capturing at least one image of a hand of the user, so as to sequentially acquire a first coordinate and a second coordinate; a computing unit, coupled to the detecting unit for defining a first zone and a second zone according to the first coordinate and the second coordinate, respectively, and calculating a first area and a second area according to the first zone and the second zone; and a determining unit, coupled to the detecting unit and the computing unit for recognizing the gesture according to the first coordinate, the second coordinate, the first area and the second area.
The present invention further discloses a gesture recognition method for a gesture recognition module, comprising: capturing at least one image of a hand of the user, to sequentially acquire a first coordinate and a second coordinate; defining a first zone and a second zone according to the first coordinate and the second coordinate; calculating a first area and a second area according to the first zone and the second zone; and recognizing the gesture according to the first coordinate, the second coordinate, the first area and the second area.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
Please refer to
Please note that, the gesture recognition module 10 shown in
Please refer to
According to the above procedures, the detecting unit 100 acquires the hand images HI_1, HI_2, sequentially. The detecting unit 100 acquires the coordinates HC_1, HC_2 according to the hand images HI_1, HI_2. According to different applications, there can be various methods of the detecting unit 100 acquiring the coordinates HC_1, HC_2 according to the hand images HI_1, HI_2. Please refer to
Next, since the hand of the user may be slightly curved, the computing unit 102 defines zones having volumes slightly greater than the hand of the user according to the coordinates HC_1, HC_2 as the zones HZ_1, HZ_2, for acquiring accurate hand areas via accumulating the areas of all pixel areas in the zones HZ_1, HZ_2. Please refer to
According to the zone HZ_1, the computing unit 102 acquires a hand start coordinate Z_HS1 by acquiring the minimum Z-axis coordinate among the plurality of pixel areas in the zone HZ_1. The computing unit 102 then adds the hand start coordinate Z_HS_1 and a depth Z_D3, to acquire a hand end coordinate Z_HE1, wherein the depth Z_D3 is a predetermined value. Each Z-axis coordinate between the hand start coordinate Z_HS1 and the hand end coordinate Z_HE1 may correspond to at least one pixel area. For example, the pixel area corresponding to the hand start coordinate Z_HS1 may be a rectangular area in the middle of the palm, and the pixel areas corresponding to the hand end coordinate Z_HE1 may be the two rectangular areas on the top side and the bottom side of the hand. Via accumulating the pixel areas corresponding to all the Z-axis coordinates located between the hand start coordinate Z_HS1 and the hand end coordinate Z_HE1, the computing unit 102 acquires the area HA_1 corresponding to the coordinate HC_1. Please refer to
After obtaining the coordinates HC_1, HC_2 and areas HA_1, HA_2, the determining unit 104 recognizes the gesture of the user according to the variation between the coordinates HC_1, HC_2 and the variation between the areas HA_1, HA_2. In order to avoid slight shaking of the hand affecting the accuracy of the determining unit 104, the determining unit 104 compares the area HA_1 and the area HA_2 when a distance between the coordinate HC_1 and the coordinate HC_2 is smaller than a moving threshold TH_M (i.e. |HC1−HC2|<TH_M), for determining whether the user performs a grabbing gesture. As can be seen from
Please note that, the above embodiments determines whether the gesture of the user is the grabbing gesture according to the hand coordinates and the hand area sequentially acquired, so as to control the electronic device coupled to the gesture recognition module. According to different applications, those skilled in the art may observe appropriate alternations and modifications. For example, when the hand of the user is askew with respect to the horizontal plane, the detecting unit 100 can rotate the hand images according to the characteristic vectors while acquiring the hand images. Please refer to
When the gesture recognition module 10 determines the gesture of the user is the grabbing gesture according to the coordinates HC_1, HC_2 and the areas HA_1, HA_2, the computing 102 may calculate a dragging ratio TR1 according to a ratio between the area HA_2 and the Z-axis coordinate Z_HC_2 of the coordinate HC_2. The detecting unit 100 further acquires a coordinate HC_3 and the computing unit 102 acquires an area HA_3 according to the coordinate HC_3. Via calculating a ratio between the area HA_3 and a Z-axis coordinate Z_HC_3 of the coordinate HC_3, the computing unit 102 acquires a dragging ratio TR2. After acquiring the dragging ratios TR1, TR2, the determining unit 104 determines whether the user continues to make the grabbing gesture according to the dragging ratios TR1, TR2. When a difference between the dragging ratios TR1, TR2 is smaller than a dragging ratio TH_T, the determining unit 104 determines the user continues performing the grabbing gesture. When the difference between the dragging ratios TR1, TR2 is smaller than the dragging ratio TH_T, the determining unit 104 determines the user performs a dragging gesture. The grabbing gesture and the dragging gesture may correspond to specific functions of the electronic device coupled to the gesture recognition module 10. For example, the gesture recognition module 10 may couples to an electronic device with a display device. The electronic device 10 controls a cursor displayed on the display device according to the coordinates (e.g. coordinates HC_1-HC_3) generated by the gesture recognition module 10. When the gesture module 10 determines the user makes the grabbing gesture according to the coordinates HC_1, HC_2 and the areas HA_1, HA_2, the gesture recognition module 10 controls the electronic device to perform a click operation at the position pointed to by the cursor. Next, when the gesture recognition module 10 determines the user performs the dragging gesture according to the relation between the dragging ratios TR1, TR2, the gesture recognition module 10 controls the electronic device to maintain the click operation and to move the cursor according to the relation between the coordinates HC_2, HC_3.
According to different applications, the method of the detecting unit 100 generating the coordinates of the hand can be appropriately modified. Please refer to
The method of the gesture recognition module 10 determining the gesture of the user can be summarized by a gesture recognition method 70, as shown in
Step 700: Start.
Step 702: Capture at least one hand image for acquiring the coordinates HC_1, HC_2, sequentially.
Step 704: Acquire the zones HZ_1, HZ_2 according to the coordinate HC_1, HC_2.
Step 706: Calculate the areas HA_1, HA_2 according to the zones HZ_1, HZ_2.
Step 708: Determine whether the distance between the coordinates HC_1, HC_2 is smaller than the moving threshold TH_M. If yes, perform step 710; otherwise, perform step 714.
Step 710: Determine whether the difference between the areas HA_1, HA_2 is greater than the grabbing threshold TH_G. If yes, perform step 712; otherwise, perform step 714.
Step 712: Determine the gesture of the user is the grabbing gesture.
Step 714: End.
According to the gesture recognition method 70, the area variation of the hand in the captured 3D image can be utilized as the references of determining the gesture of the user, so as to determine whether the user makes the grabbing gesture. The detailed operations of the gesture recognition method 70 can be known by referring to the above, and are therefore not narrated herein for brevity.
To sum up, the gesture recognition module and the gesture recognition method of the above embodiments determines whether the gesture of the user is the grabbing gesture according to the area variation of the hand in the captured 3D images. The gesture recognition module and the gesture recognition method of the above embodiments therefore utilize the depth information of the 3D images to further increase the accuracy of recognizing the gesture of the user.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.
Chou, Chia-Te, Wei, Shou-Te, Lee, Chih-Hsuan, Lee, Chung-Wei
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
8339359, | Dec 10 2009 | Tatung Company; TATUNG UNIVERSITY | Method and system for operating electric apparatus |
20020126090, | |||
20110175802, | |||
20130120250, | |||
20140104168, | |||
TW201120681, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
May 09 2013 | LEE, CHUNG-WEI | Wistron Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030399 | /0369 | |
May 09 2013 | LEE, CHIH-HSUAN | Wistron Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030399 | /0369 | |
May 09 2013 | CHOU, CHIA-TE | Wistron Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030399 | /0369 | |
May 09 2013 | WEI, SHOU-TE | Wistron Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030399 | /0369 | |
May 13 2013 | Wistron Corporation | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Jul 11 2019 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Jul 12 2023 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Date | Maintenance Schedule |
Mar 15 2019 | 4 years fee payment window open |
Sep 15 2019 | 6 months grace period start (w surcharge) |
Mar 15 2020 | patent expiry (for year 4) |
Mar 15 2022 | 2 years to revive unintentionally abandoned end. (for year 4) |
Mar 15 2023 | 8 years fee payment window open |
Sep 15 2023 | 6 months grace period start (w surcharge) |
Mar 15 2024 | patent expiry (for year 8) |
Mar 15 2026 | 2 years to revive unintentionally abandoned end. (for year 8) |
Mar 15 2027 | 12 years fee payment window open |
Sep 15 2027 | 6 months grace period start (w surcharge) |
Mar 15 2028 | patent expiry (for year 12) |
Mar 15 2030 | 2 years to revive unintentionally abandoned end. (for year 12) |