An apparatus, a method, and a computer program product are provided. The apparatus generates at least one signal in response to a contactless gesture performed proximate to an infrared proximity sensor situated on the optical device, the contactless gesture corresponding to a command, identifies the command using the at least one signal, and executes the command.
|
1. A method of controlling an optical device configured as an augmented reality pair of glasses and configured to display content, the method comprising:
generating one of a plurality of gesture signals in response to a corresponding plurality of different contactless gestures performed proximate to a receiver situated on the pair of glasses between at least first and second infrared light emitting diodes (IR LEDs), the first IR LED transmitting a first signal directly to the receiver, the second IR LED transmitting a second signal directly to the receiver, the receiver configured between the first IR LED and the second IR LED such that the receiver receives the first signal directly from the first IR LED on a first portion of the receiver and receives the second signal directly from the second IR LED on a second portion of the receiver, the gesture signals generated in response to the receiver detecting a sequence of obstructions of the first signal and the second signal transmitted directly to the receiver, each contactless gesture corresponding to a command, wherein each contactless gesture is mapped to a plurality of commands using a mapping, and wherein the plurality of commands comprise actions or menu selections that are executed by an augmented reality application;
identifying a command from the plurality of commands mapped to the contactless gesture using the generated one of a plurality of different gesture signals and the mapping; and
executing the command,
wherein a first gesture signal corresponds to an obstruction of the first signal followed by an obstruction of the second signal, and a second gesture signal corresponds to an obstruction of the second signal followed by an obstruction of the first signal.
6. An apparatus for controlling an optical device configured as an augmented reality pair of glasses and configured to display content, the apparatus comprising:
means for generating one of a plurality of gesture signals in response to a corresponding plurality of different contactless gestures performed proximate to a receiver situated on the pair of glasses between at least first and second infrared light emitting diodes (IR LEDs), the first IR LED transmitting a first signal directly to the receiver, the second IR LED transmitting a second signal directly to the receiver, the receiver configured between the first IR LED and the second IR LED such that the receiver receives the first signal directly from the first IR LED on a first portion of the receiver and receives the second signal directly from the second IR LED on a second portion of the receiver, the gesture signals generated in response to the receiver detecting a sequence of obstructions of the first signal and the second signal transmitted directly to the receiver, each contactless gesture corresponding to a command, wherein each contactless gesture is mapped to a plurality of commands using a mapping, and wherein the plurality of commands comprise actions or menu selections that are executed by an augmented reality application;
means for identifying a command from the plurality of commands mapped to the contactless gesture using the generated one of a plurality of different gesture signals and the mapping; and
means for executing the command,
wherein a first gesture signal corresponds to an obstruction of the first signal followed by an obstruction of the second signal, and a second gesture signal corresponds to an obstruction of the second signal followed by an obstruction of the first signal.
11. An apparatus for controlling an optical device configured as an augmented reality pair of glasses and configured to display content, the apparatus comprising:
a processing system configured to:
generate one of a plurality of gesture signals in response to a corresponding plurality of different contactless gestures performed proximate to a receiver situated on the pair of glasses between at least first and second infrared light emitting diodes (IR LEDs), the first IR LED transmitting a first signal directly to the receiver, the second IR LED transmitting a second signal directly to the receiver, the receiver configured between the first IR LED and the second IR LED such that the receiver receives the first signal directly from the first IR LED on a first portion of the receiver and receives the second signal directly from the second IR LED on a second portion of the receiver, the gesture signals generated in response to the receiver detecting a sequence of obstructions of the first signal and the second signal transmitted directly to the receiver, each contactless gesture corresponding to a command, wherein each contactless gesture is mapped to a plurality of commands using a mapping, and wherein the plurality of commands comprise actions or menu selections that are executed by an augmented reality application;
identifying a command from the plurality of commands mapped to the contactless gesture using the generated one of a plurality of different gesture signals and the mapping; and
execute the command,
wherein a first gesture signal corresponds to an obstruction of the first signal followed by an obstruction of the second signal, and a second gesture signal corresponds to an obstruction of the second signal followed by an obstruction of the first signal.
16. A non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of an optical device configured as an augmented reality pair of glasses, the optical device configured to display content, to perform operations comprising:
generating one of a plurality of gesture signals in response to a corresponding plurality of different contactless gestures performed proximate to a receiver situated on the pair of glasses between at least first and second infrared light emitting diodes (IR LEDs), the first IR LED transmitting a first signal directly to the receiver, the second IR LED transmitting a second signal directly to the receiver, the receiver configured between the first IR LED and the second IR LED such that the receiver receives the first signal directly from the first IR LED on a first portion of the receiver and receives the second signal directly from the second IR LED on a second portion of the receiver, each gesture signal generated in response to the receiver detecting a sequence of obstructions of the first signal and the second signal transmitted directly to the receiver, each contactless gesture corresponding to a command, wherein the contactless gesture is mapped to a plurality of commands using a mapping, and wherein the plurality of commands comprise actions or menu selections that are executed by an augmented reality application;
identifying a command from the plurality of commands mapped to the contactless gesture using the generated one of a plurality of different gesture signals and the mapping; and
executing the command,
wherein a first gesture signal corresponds to an obstruction of the first signal followed by an obstruction of the second signal, and a second gesture signal corresponds to an obstruction of the second signal followed by an obstruction of the first signal.
2. The method of
3. The method of
4. The method of
5. The method of
7. The apparatus of
8. The apparatus of
9. The apparatus of
10. The apparatus of
12. The apparatus of
13. The apparatus of
14. The apparatus of
15. The apparatus of
17. The non-transitory processor-readable storage medium of
18. The non-transitory processor-readable storage medium of
executing the command comprises performing a function for controlling at least one of the content and a graphical user interface (GUI).
19. The non-transitory processor-readable storage medium of
20. The non-transitory processor-readable storage medium of
21. The method of
22. The method of
|
Field
The present disclosure relates generally to an apparatus and method for an infrared contactless gesture system.
Background
Systems and applications for implementing Augmented Reality (AR) have become very popular and widespread, since AR allows users to interact with their surroundings in a new way. For example, AR may display data about a user's surroundings in real time and present such data to a user in a natural way. Although many AR applications may be run on smartphones and tablets, the most natural form factor for implementing AR systems are optical devices, such as glasses.
The conventional methods for interacting with such AR glasses include voice commands or button presses on the AR glasses. However, such methods suffer from significant drawbacks. For example, a user may not be able to control the AR glasses in an area requiring silence (e.g., in a meeting or theater). User's that are not capable of speaking cannot control the AR glasses and voice command accuracy is significantly affected by voice quality and ambient noises. Moreover, voice recognition must be configured to support different languages. Finally, pressing buttons on glasses may be uncomfortable and inconvenient to users.
In an aspect of the disclosure, an apparatus, a method, and a computer program product are provided. The apparatus generates at least one signal in response to a contactless gesture performed proximate to an infrared proximity sensor situated on the optical device, the contactless gesture corresponding to a command, identifies the command using the at least one signal, and executes the command.
The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In some instances, well known structures and components are shown in block diagram form in order to avoid obscuring such concepts.
Several aspects of an infrared contactless gesture system will now be presented with reference to various apparatus and methods. These apparatus and methods will be described in the following detailed description and illustrated in the accompanying drawings by various blocks, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system.
By way of example, an element, or any portion of an element, or any combination of elements may be implemented with a “processing system” that includes one or more processors. Examples of processors include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. One or more processors in the processing system may execute software. Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
Accordingly, in one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
In one aspect, the optical device 100 may be configured as an augmented reality (AR) display (also referred to as “AR glasses”). In such an aspect, the optical device 100 may be configured to project content, such as AR imagery, through the lens 108 and/or lens 110 using methods known in the art. In another aspect, the optical device 100 may be configured to project content through the lens 108 and/or lens 110, such as a movie or an electronic book (also referred to as an “e-book”). As described infra, the optical device 100 may be configured with a user interface (UI) that allows interaction with the optical device 100 using contactless gestures. A contactless gesture refers to a gesture that is performed without making physical contact with the optical device 100.
In one aspect, with reference to
Each contactless gesture, that is, each signal generated by the receiver 116 in response to a contactless gesture, may correspond to one or more commands. In one configuration, each signal generated by the receiver 116 in response to a contactless gesture may be mapped to one or more commands. Accordingly, when a user performs a contactless gesture to input a command, the optical device 100 may use the signal generated by the receiver 116 in response to the contactless gesture to identify the command. For example, the command may be an action or menu selection that may be executed by an application, such as an AR application, a movie application, or an e-book application.
For example, a right swipe contactless gesture may correspond to a command for zooming out in a camera application or a map application, a command for selecting an item on a list from top to bottom in a media player application, a command for scrolling down a page in an e-book application, and/or a command for setting focus on application icons one by one from left to right in an application menu. A left swipe contactless gesture may correspond to a command for zooming in on a camera application or a map application, a command for selecting an item on a list from bottom to top in a media player application, a command for scrolling up a page in an e-book application, and/or a command for setting focus on application icons one by one from right to left in an application menu. A center tap contactless gesture may correspond to a command for exiting a current application. A front tap contactless gesture may correspond to a command for taking a picture in a camera application, a command accepting a conversation request, a command for going to a next page in an e-book application, and/or a command for increasing the volume in a media player application or during a conversation. A rear tap contactless gesture may correspond to a command for saving a picture in a camera application, a command for rejecting a conversation request, a command for going to a previous page in an e-book application, and/or a command for decreasing the volume in a media player application or during a conversation.
In one aspect, the transmitters 112 and 114 and the receiver 116 may be situated on temple portion 106 instead of temple portion 104. In another aspect, additional transmitters and a receiver may be situated on temple portion 106 using a same or different configuration as transmitters 112 and 114 and the receiver 116. The additional transmitters and the receiver may be included on optical device 100 to supplement the number of available commands.
In one aspect, the receiver 116 may generate a different signal for each contactless gesture performed by a user, such as a gesture from transmitter 114 to transmitter 112 (left swipe), a gesture from transmitter 112 to transmitter 114 (right swipe), a push/pull gesture on or proximate to the receiver 116 (center tap), a push/pull gesture on or proximate to the transmitter 114 (front tap), and a push/pull gesture on or proximate to the transmitter 112 (rear tap). In another aspect, with reference to
Each contactless gesture may correspond to one or more commands. In one configuration, each signal generated by the receiver 116 in response to a contactless gesture may be mapped to one or more commands. For example, the command may be an action or menu selection that may be executed by an application, such as an AR application, a movie application, or an e-book application.
For example, a right swipe contactless gesture may correspond to a command for zooming out in a camera application or a map application, a command for selecting an item on a list from top to bottom in a media player application, a command for scrolling down a page in an e-book application, and/or a command for setting focus on application icons one by one from left to right in an application menu. A left swipe contactless gesture may correspond to a command for zooming in on a camera application or a map application, a command for selecting an item on a list from bottom to top in a media player application, a command for scrolling up a page in an e-book application, and/or a command for setting focus on application icons one by one from right to left in an application menu. A center tap contactless gesture may correspond to a command for exiting a current application. A front tap contactless gesture may correspond to a command for taking a picture in a camera application, a command accepting a conversation request, a command for going to a next page in an e-book application, and/or a command for increasing the volume in a media player application or during a conversation. A rear tap contactless gesture may correspond to a command for saving a picture in a camera application, a command for rejecting a conversation request, a command for going to a previous page in an e-book application, and/or a command for decreasing the volume in a media player application or during a conversation.
At step 404, the optical device 100 may identify the command using the at least one signal. As previously described, each signal generated by the receiver 116 in response to a contactless gesture may be mapped to one or more commands. Therefore, when a user performs a contactless gesture to input a command, the optical device 100 may use the mapping and the signal generated by the receiver 116 in response to the contactless gesture to identify the command corresponding to generated signal.
Finally, at step 406, the optical device 100 may execute the command. In one aspect, the optical device 100 may execute the command by performing a function for controlling at least one of the content and a UI. For example, the UI may be a graphical user interface (GUI).
The processing system 614 includes a processor 604 coupled to a computer-readable medium 606. The processor 604 is responsible for general processing, including the execution of software stored on the computer-readable medium 606. The software, when executed by the processor 604, causes the processing system 614 to perform the various functions described supra for any particular apparatus. The computer-readable medium 606 may also be used for storing data that is manipulated by the processor 604 when executing software. The processing system further includes at least one of the modules 504, 506, and 508. The modules may be software modules running in the processor 604, resident/stored in the computer readable medium 606, one or more hardware modules coupled to the processor 604, or some combination thereof.
In one configuration, the apparatus 502/502′ includes means for generating at least one signal in response to a contactless gesture performed proximate to an infrared proximity sensor situated on the optical device, the contactless gesture corresponding to a command, means for identifying the command using the at least one signal, and means for executing the command.
The aforementioned means may be one or more of the aforementioned modules of the apparatus 502 and/or the processing system 614 of the apparatus 502′ configured to perform the functions recited by the aforementioned means.
It is understood that the specific order or hierarchy of steps in the processes disclosed is an illustration of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged. Further, some steps may be combined or omitted. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.
The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims. No claim element is to be construed as a means plus function unless the element is expressly recited using the phrase “means for.”
Lee, Jangwon, Chen, An M., Cai, Ning
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
6495832, | Mar 15 2000 | Azonix Corporation | Photoelectric sensing array apparatus and method of using same |
8199126, | Jul 18 2011 | GOOGLE LLC | Use of potential-touch detection to improve responsiveness of devices |
8217856, | Jul 27 2011 | GOOGLE LLC | Head-mounted display that displays a visual representation of physical interaction with an input interface located outside of the field of view |
8319745, | Feb 24 2000 | Silverbrook Research Pty LTD | Method of estimating nib position using images captured at different pen rotations |
8319746, | Jul 22 2011 | GOOGLE LLC | Systems and methods for removing electrical noise from a touchpad signal |
8823643, | Jul 09 2012 | Optical computer input controller device | |
20060086896, | |||
20060114237, | |||
20070220108, | |||
20090139778, | |||
20100045530, | |||
20100079356, | |||
20100295773, | |||
20100299642, | |||
20110102378, | |||
20110194029, | |||
20110310005, | |||
20120021806, | |||
20120056847, | |||
20120062445, | |||
20120120103, | |||
20120188206, | |||
20120194478, | |||
20120312956, | |||
20130106686, | |||
CN101890719, | |||
CN102622132, | |||
DE102006020570, | |||
WO2011082004, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Dec 03 2012 | Qualcomm Incorporated | (assignment on the face of the patent) | / | |||
Jan 10 2013 | CAI, NING | Qualcomm Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029780 | /0885 | |
Jan 10 2013 | LEE, JANGWON | Qualcomm Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029780 | /0885 | |
Jan 10 2013 | CHEN, AN M | Qualcomm Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029780 | /0885 |
Date | Maintenance Fee Events |
Oct 14 2021 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Date | Maintenance Schedule |
May 22 2021 | 4 years fee payment window open |
Nov 22 2021 | 6 months grace period start (w surcharge) |
May 22 2022 | patent expiry (for year 4) |
May 22 2024 | 2 years to revive unintentionally abandoned end. (for year 4) |
May 22 2025 | 8 years fee payment window open |
Nov 22 2025 | 6 months grace period start (w surcharge) |
May 22 2026 | patent expiry (for year 8) |
May 22 2028 | 2 years to revive unintentionally abandoned end. (for year 8) |
May 22 2029 | 12 years fee payment window open |
Nov 22 2029 | 6 months grace period start (w surcharge) |
May 22 2030 | patent expiry (for year 12) |
May 22 2032 | 2 years to revive unintentionally abandoned end. (for year 12) |