A virtual data input device to optically interface with an operator to detect the position of objects within an input zone, i.e., an area defined as a “virtual keyboard” in which the operator may interact to enter character or other data into computing equipment. The input device includes a source of optical sensor light illuminating the input zone that reflects off the objects in a direction generally toward the input device. The device also includes an optical detector arranged to receive reflected sensor light as a light pattern, representing the position of the objects within the input zone, and converts the reflected light pattern to an electrical signal. A microprocessor then receives the electrical signal and correlates the electrical signal to character or other data. An image generator may be used to project an optical image that represents, for example, an image of a keyboard.
|
0. 59. A method, comprising:
receiving light reflected from one or more objects in an input zone associated with a computing device, wherein a pattern of said reflected light is usable to determine position and/or motion of said one or more objects within said input zone;
operating said computing device in a first input mode for receiving virtual keyboard input data; and
operating said computing device in a second input mode for receiving data usable to control a location of a pointer on a display of said computing device.
0. 23. A method, comprising:
receiving a first light pattern reflected from one or more objects in an input zone associated with a computing device, wherein position and/or motion of said one or more objects correspond to desired input data from a user of said computing device;
generating, from said first light pattern, actual input data;
conveying said actual input data to said user via said computing device;
receiving feedback from said user regarding correspondence between said desired input data and said actual input data; and
using said feedback to configure said computing device to associate said first light pattern with said desired input data.
0. 4. A device, comprising:
an optical detector configured to receive a light pattern reflected from one or more objects in an associated input zone, wherein positions of said one or more objects within said input zone correspond to desired input data from a user of said device, wherein said optical detector is configured to convert said reflected light pattern into an electrical signal;
a microprocessor configured to receive said electrical signal and generate actual input data to said device therefrom;
wherein said device is configured to receive information from said user that is usable to associate said positions of said one or more objects with said desired input data.
0. 44. A method, comprising:
receiving a light pattern reflected from one or more objects in an input zone associated with a computing device, wherein said one or more objects are arranged in a first set of positions within said input zone, and wherein said first set of positions correspond to desired input data from a user of said computing device;
receiving information from said user assigning said first set of positions to a sequence of two or more desired input characters;
upon subsequently receiving a reflected light pattern corresponding to said first set of positions of said one or more objects, generating actual input data corresponding to said sequence of two or more desired input characters.
0. 33. A device, comprising:
an optical detector configured to receive light patterns reflected from one or more objects in an input zone, wherein positions of said one or more objects within said input zone correspond to desired character input data from a user of said device, and wherein said optical detector is configured to convert said reflected light patterns into electrical signals;
a microprocessor configured to receive said electrical signals from said optical detector and generate actual input data therefrom;
wherein said device is configured to receive input from said user assigning a first set of positions of said one or more objects in said input zone to represent two or more characters of desired actual input data, such that said device, upon subsequently receiving a first reflected light pattern corresponding to said first set of positions of said one or more objects, is configured to generate actual input data that corresponds to said two or more characters of desired actual input data.
0. 48. A device, comprising:
an optical detector configured to receive light patterns reflected from one or more objects in an input zone, wherein position and/or motion of said one or more objects in said input zone correspond to desired input data from a user of said device, wherein said optical detector is configured to convert said reflected light patterns into electrical signals;
a microprocessor configured to receive said electrical signals from said optical detector and generate actual input data therefrom;
a display;
wherein said device is configured to operate in a plurality of input modes including a first input mode and a second input mode;
wherein, in said first input mode, positions of said one or more objects within said input zone correspond to virtual keyboard input data, and wherein said actual input data generated by said microprocessor in said first input mode corresponds to said virtual keyboard input data;
wherein, in said second input mode, positions and/or motion of said one or more objects within said input zone correspond to virtual pointing device input data, and wherein said actual input data generated by said microprocessor in said second input mode is usable to control a location of a pointer on said display.
1. A data input device optically interfacing with an operator to detect the position of one or more objects within an input zone to allow the operator to enter character data into associated electronic equipment, wherein each position of the objects within the input zone corresponds to a unique selection of data to be entered into the electronic equipment, the input device comprising:
a source of optical sensor light illuminating the input zone with sensor light, the sensor light reflecting off the objects in a direction generally toward the input device;
an optical detector arranged to receive the reflected sensor light as a reflected light pattern representing the position of the objects within the input zone, the optical detector converting the reflected right pattern to an electrical signal;
a light generator positioned to emit visible light and a deflective optical element that projects an optical image of an input device having discrete sub-images at unique positions within the optical image that represent different character data that may be selected for input into the electronic equipment; wherein the deflective optical element comprises a stencil designed to allow light to pass through portions of the stencil and wherein the light generator is positioned to emit visible fight through the portions of the stencil such that the light passed through the stencil forms the optical image; and
a microprocessor that receives the electrical signal from the optical detector and correlates the electrical signal to the unique selection of data.
3. A data light device optically interfacing with an operator to detect the position of an object within an input zone to allow the operator to control the location of a pointer displayed on a viewing screen of associated electronic equipment, the input device comprising:
a source of optical sensor light illuminating the input zone with sensor light, the sensor light reflecting off the object in a direction generally toward the input device;
an optical detector arranged to receive the reflected sensor light as a reflected light pattern that represents the position of the object within the input zone, the optical detector converting the reflected light pattern to an electrical signal representing the position of the object within the input zone;
a microprocessor that receives the electrical signal from the optical detector and is programmed to correlate the electrical signal to the location of the pointer displayed on said viewing screen; and
further comprising an image generator that projects an optical image within the input zone, wherein the optical image has discrete sub-images within the optical image; and
wherein the image generator comprises a light generator positioned to emit visible light and a deflective optical element designed to form the optical image; and
wherein the deflective optical element comprises a stencil designed to allow light to pass through portions of the stencil and wherein the light generator is positioned to emit visible light through the portions of the stencil such that the light passed through the stencil forms the optical image.
2. The optical data input device of
0. 5. The device of
0. 6. The device of
0. 7. The device of
0. 8. The device of
0. 9. The device of
0. 10. The device of
0. 11. The device of
0. 12. The device of
0. 13. The device of
0. 14. The device of
0. 15. The device of
0. 16. The device of
0. 17. The device of
0. 18. The device of
0. 19. The device of
0. 20. The device of
0. 21. The device of
0. 22. The device of
0. 24. The method of
0. 25. The method of
0. 26. The method of
0. 27. The method of
0. 28. The method of
0. 29. The method of
0. 30. The method of
0. 31. The method of
0. 32. The method of
0. 34. The device of
0. 35. The device of
0. 36. The device of
0. 37. The device of
0. 38. The device of
0. 39. The device of
0. 40. The device of
0. 41. The device of
0. 42. The device of
0. 43. The device of
0. 45. The method of
0. 46. The method of
0. 47. The method of
0. 49. The device of
0. 50. The device of
0. 51. The device of
0. 52. The device of
0. 53. The device of
0. 54. The device of
0. 55. The device of
0. 56. The device of
0. 57. The device of
0. 58. The device of
0. 60. The method of
0. 61. The method of
0. 62. The method of
0. 63. The method of
0. 64. The method of
|
This is a continuation of application Ser. No. 09/572,349, filed on May 17, 2000 now U.S. Pat. No. 6,611,252.
The present invention relates generally to data input devices and, more particularly, to data input devices adapted for use with portable communications and computing equipment.
Demand for compact communications and computing equipment has increased dramatically over the last decade. Computers that can be held in the palm of your hand and wireless phones that fit in a shirt pocket are two examples of popular miniaturized machines. More recently, demand for wireless e-mail and Internet access has begun to soar, with experts projecting future demand to rise at unprecedented rates.
One problem associated with miniaturized communications and computing equipment is having a convenient way to input data, such as character and pointing device data, into such equipment. Early miniaturized computing equipment, typical of the 1990s, included a miniaturized keyboard that was scaled to fit the desired sized of the computing equipment. Typing a few words using such systems is quite laborious because the operator's fingers are typically too large to use the device as a traditional keyboard. Portable communication equipment, on the other hand, typically include a conventional 12-button keypad to input data. It is extremely difficult to use this set-up to enter non-numerical data. For example, to enter the word CALL, an operator would hit the button marked “2-A-B-C” three times for C, the button marked “2-A-B-C” once for A, the button marked “5-J-K-L” three times for L, and finally the button marked “5-J-K-L” three times again for the final L.
To ease the problem of character input, some manufacturers of both communications and computing equipment have recently developed pen-type portable devices in which a real keyboard or keypad is not provided and data input operations can be carried out by utilizing a miniaturized virtual keyboard displayed on a touch sensitive screen. The pen can also be used as a pointing device to select items on the screen. An additional feature of many touch sensitive screen systems is the ability to write characters with a pen or stylus that is recognized as individual characters by the device. Another recent development is the collapsible keyboard, such as those currently being marketed by Palm Computing. This keyboard may be folded and carried in a briefcase or even a pocket, and is opened and plugged into the miniaturized equipment before it is ready to use. Yet another development is voice recognition. However, this technology is not currently highly reliable and, as a result, input errors are common. Furthermore, numerous circumstances arise when voice input is not practical or appropriate. Moreover, voice recognition is not suitable for entering pointing device information.
While each of these methods represents a form of improvement over previous technologies, the need remains for a data input device for use with miniaturized communications and computing equipment that allows an operator to easily input characters and data into such equipment. Preferably, such an input device would incorporate wireless techniques to sense the position and motion of the operator's fingers to allow the user to enter data without the use of a physical keyboard or a pointing device.
A data input device having these desired features has now been developed. Broadly speaking, the data input device of the present invention optically interfaces with an operator to detect the position of objects within a particular input zone, e.g., an area defined as a “virtual keyboard” in which the operator may interact to enter character data into associated computing equipment. Preferably, the objects are the operator's fingers placed within the input zone. As a character input device, each character data corresponds to a unique arrangement and position of the objects within the input zone. As a pointing device, the relative motion of the operator's fingers defines the input area. The input device includes a source of optical sensor light illuminating the input zone with sensor light. The source of optical sensor light may be ambient light surrounding the operator or a light emitting device adapted to emit light in a direction toward the operator's fingers. Preferably, the source of optical light covers the input zone. The sensor light reflects off the objects in a direction generally toward the light device. The data input device also includes an optical detector arranged to receive the reflected sensor light as a reflected light pattern representing the relative position of the operator's fingers within the input zone. The optical detector converts the reflected light pattern to an electrical signal representing the particular character data desired to be entered by the operator. A microprocessor then receives the electrical signal and correlates the electrical signal to character or position/motion data.
In one embodiment of the virtual data input device, an image generator is used to project an optical image that represents character data, e.g., an image of a real keyboard. The image generator may be formed by an optical element, such as a stencil, mask, holographic element, mirror array, or other suitable device known in the art of image projection, designed to allow light to pass through portions of the optical element and a light generator positioned to emit visible light through the portions of the optical element, whereby the light passed through the optical element forms the optical image.
The present invention also includes a novel method to enter character data into communications or computing equipment based on the position of objects within an input zone, e.g., an operator's fingers on a virtual keyboard. The position of the objects within the input zone uniquely corresponds to particular character data. A reference position is established to associate an initial position of the objects within the input zone and to associate a plurality of positions of the objects within the input zone to unique character data. A source of optical sensor light is provided to illuminate the input zone such that the sensor light reflects off the objects. The reflected sensor light is then received as a reflected light pattern representing the position of the objects within the input zone. The reflected light pattern is converted to an electrical signal that is then correlated to the unique selection of character data, which may then be input into the electronic equipment.
These and other features, aspects, and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings wherein:
These drawings are provided for illustrative purposes only and should not be used to unduly limit the scope of the present invention.
The light generator 22 delivers a visible light beam 24 to a collimating lens 26. The collimating lens 26 diverges the light beam 24 and emits collimated rays 28 of light having an increased cross-sectional area. Rays 28 are then projected through a deflective optical element 30 and through lens 32. Lens 32 expands the rays 28 to project an image, such as of a full size keyboard, at a suitable distance from the input device 10 and preferably adjacent an operator. Alternatively, the image generator 20 may be designed to project light through the deflective optical element 30 at a particular distance without the use of lens 32. A diverging image beam 34 is produced that projects an image, such as an image 80 of a conventional keyboard, onto any surface. The image 80 is typically formed from a plurality of discrete sub-images 82, that may represent, for example, particular keys on a conventional keyboard. The image 80 generally defines an input zone, i.e., the two-dimensional area within which the virtual data input device 10 will detect the position of the operator's fingers and correlate such position to unique character data. Each of the discrete sub-images 82 may then be selected by the operator for input into the equipment 70 based upon a unique position of objects within the input zone, e.g., a particular arrangement of fingers on the “virtual keyboard.”
The light generator 22 may include features to control the brightness, contrast and focal length of image 80. In addition, collimating lens 26 and/or lens 32 may be adjustable to allow the operator to control the position and focus of the projected image 80.
As shown in
To provide accurate input data and to reduce the possibility of input errors, the optical sensor receiving unit 50 may incorporate features to limit the wavelength of light detected by the light sensing device 52. For example, the light sensing device 52 may be designed to detect only the wavelength of light emitted by the light emitting device 42. Thus, the receiving unit 50 may include a light filter 56 to eliminate light of other wavelengths. Alternatively, the light sensing device 52 may be tuned to be wavelength sensitive or selective and designed for the particular wavelength of light emitting by the light emitting device 42.
As those skilled in the art will appreciate, the optical sensor receiving unit 50 requires a source of optical sensor light, which is described above as the light source 40. In an alternative embodiment, the virtual data input device 10 may rely upon ambient light existing in the operator's surroundings as the source of optical sensor light to be reflected off of the operator's fingers and detected by the light sensing device 52. This embodiment, which eliminates the need for the components illustrated in
The optical sensor receiving unit 50 transmits the electric signal that it generates to a microprocessor 60, which may be included within the virtual data input device 10. Alternatively, the optical sensor receiving unit 50 may send the electrical signal to a microprocessor 60 of the equipment 70 such that the signals may be interpreted and appropriate data input into the equipment 70. Thus, the microprocessor 60 interprets the signal from the optical sensor receiving unit 50 that is then delivered to and recognized by communications and computing equipment 50. The microprocessor 60 preferably includes an algorithm to determine which of the character data is selected. As an example, the microprocessor 60 may include a stored database of character data signals and then compare the received electrical signals to the signals stored in the database to correlate the received signals to character data. It may be preferred for the operator to first “teach” the system the unique position and/or motion of his/her fingers for a given character. This information may be stored in a database for reference during operation.
The step of reading the operator's finger positions may be accomplished in a variety of ways. For example, the electrical signal generated by the optical sensor receiving unit 50 and transmitted to microprocessor 60 may be substantially continuously read (e.g., every tenth, hundredth, or thousandth of a second) by microprocessor 60 and a character determined after the signal changes. For example, when the operator's fingers are positioned stationary over the “home keys” or “asdf” and “jkl;”, the signal from the optical sensor receiving unit 50 remains constant and no character data would be transmitted to the equipment 70. If the operator desires to input the letter “j” the right index finger positioned on that letter would be raised and then repositioned on the “j”. After the motion is complete, the microprocessor 60 would detect a change in the signal from the optical sensor receiving unit 50, compare the new signal to the previous signal and a look-up compiled database, and correctly read the newly inputted character. If the new signal does not match any signal stored in the database, no data or character is input during that particular cycle.
Once the operator is aligned with the image, the virtual data input device 10 begins to repetitively attempt to read new character data in a series of detection cycles. Thus, after a preset delay time (e.g., 10 milliseconds), the optical sensor receiving unit 50 receives a pattern of light reflected off of the operator's fingers that represents the position of the operator's fingers within an input zone, i.e., on the image. As described earlier, the reflected light may originate from an optical sensor light source 40 or from ambient light in the operator's surroundings. The reflected light pattern is converted to an electrical signal that is then compared to the electrical signal generated in the last detection cycle. If the electrical signal differs from the signal in the previous cycle, the signal is then used to determine the character (or key) associated with such signal. If the signal corresponds to a recognizable character, the microprocessor 60 accepts such character and inputs it to the equipment 70.
The operation of the virtual data input device 10 may be described with reference to
The method, which is similar in many respects to the method illustrated in
A microprocessor may be used to execute an algorithm to determine which of a plurality of selection options (e.g., keys) is selected. For example, the microprocessor may compare the electrical signal to a stored database of signals representing unique character data selections. Alternatively, the microprocessor may compare the electrical signal to the previous electrical signal to relate the position of a pointing device on a display screen.
As one will understand upon reading the disclosure of the present invention, the use of a projected image is solely for the operator's benefit. Just as many people can readily type on a keyboard without looking down to “hunt and peck” for keys, one may use the virtual data input device without the use of the protected image. It is preferred that there is a mechanism to align the operator's fingers to the virtual data input device 10. Alternatives to the projected image include a simple sheet of paper that is located at a predetermined distance and position from the equipment (e.g., the equipment is rested on a spot marked on the paper). However, since the system is ultimately comparing the relative position of the operator's finger(s), there is an acceptable range of space for the operator to place his/her fingers and, therefore, the alignment of the device 10 is not a necessity.
In another embodiment, the virtual data input device 10 may include the ability to “learn” additional keystrokes. Thus, the operator may assign certain finger positions to a character or set of characters similar to a macro in popular word processing and spreadsheet software programs. Additionally, the system may incorporate a “Hot Key” that allows the operator to switch between modes (e.g., from keyboard to mouse) by simply moving his/her fingers to a predetermined position.
The principles of the present invention may be used as a character input device for various forms of portable communications and computer equipment, including cell phones, Internet-ready cell phones, palm-size computers, personal computers, laptop computers, notebook computers, personal digital assistants, and the like. The invention may also be used to input character data into portable hybrid communications and computing equipment such as palm-size computer telephones, standard desktop computers, hand-held data receiving or entry devices, GPS receivers, inventory management devices, and portable computing equipment integrated with automobiles, aircraft and the like. In addition, the virtual data input device may be used to detect the position of other forms of input means beyond an operator's fingers, such as a writing stylus or a computer mouse. Moreover, the optical sensor light source 40 and the optical sensor receiving unit 50 may be configured to detect the position of an operator's fingers or other objects on a computer display screen. Thus, the features of the present invention can allow the virtual data input device to act as a form of touch sensitive screen.
Although the present invention has been described in considerable detail with reference to certain presently preferred embodiments thereof, other embodiments are possible without departing from the spirit and scope of the present invention. Therefore the appended claims should not be limited to the description of the preferred versions contained herein.
Patent | Priority | Assignee | Title |
8552983, | Jul 11 2007 | Intelligent robotic interface input device | |
8959013, | Sep 27 2010 | Apple Inc | Virtual keyboard for a non-tactile three dimensional user interface |
9305229, | Jul 30 2012 | Method and system for vision based interfacing with a computer |
Patent | Priority | Assignee | Title |
4177354, | Apr 17 1978 | Bell Telephone Laboratories, Incorporated | Graphic communications apparatus |
4713535, | Sep 04 1985 | Optical keyboard | |
4980685, | Feb 24 1986 | Scanning optical keyboard | |
5450148, | Apr 18 1994 | LIN, YU SHU; NI, RAY-ING; LIN, FANG-ING; LIAUH, WEI-CHANG WAYNE | Laser pointer with selectable pointer patterns |
5457454, | Sep 22 1992 | RPX Corporation | Input device utilizing virtual keyboard |
5581484, | Jun 27 1994 | Finger mounted computer input device | |
5718496, | Jun 25 1996 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | Projection pointer |
5757361, | Mar 20 1996 | International Business Machines Corporation | Method and apparatus in computer systems to selectively map tablet input devices using a virtual boundary |
5785439, | Aug 24 1992 | Product Engineering & Manufacturing, Inc. | Environmentally safe machine control security switch |
5815126, | Oct 22 1993 | Kopin Corporation | Monocular portable communication and display system |
5909210, | Jun 07 1995 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Keyboard-compatible optical determination of object's position |
5938308, | Jun 25 1996 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | Projection pointer |
5977935, | Aug 12 1993 | Seiko Epson Corporation | Head-mounted image display device and data processing apparatus including the same |
6002390, | Nov 25 1996 | IRONWORKS PATENTS LLC | Text input device and method |
6022126, | Jul 04 1996 | Sekinos Co., Ltd. | Laser pointer |
6037882, | Sep 30 1997 | Apple Inc | Method and apparatus for inputting data to an electronic system |
6097374, | Mar 06 1997 | HARMONIC RESEARCH, INC | Wrist-pendent wireless optical keyboard |
6167469, | May 18 1998 | RPX Corporation | Digital camera having display device for displaying graphical representation of user input and method for transporting the selected digital images thereof |
6218967, | Apr 01 1996 | Arrangement for the optical remote control of apparatus | |
6266048, | Aug 27 1998 | Qualcomm Incorporated | Method and apparatus for a virtual display/keyboard for a PDA |
6281878, | Nov 01 1994 | VIRDEX, INC | Apparatus and method for inputing data |
6611252, | May 17 2000 | PINEAPPLE34, LLC | Virtual data input device |
6614422, | Nov 04 1999 | Microsoft Technology Licensing, LLC | Method and apparatus for entering data using a virtual input device |
20050024324, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Apr 29 2006 | Tree Frog Technologies, LLC | P MILTON HOLDINGS, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 022607 | /0835 | |
Jun 20 2006 | DUFAUX, DOUGLAS P | Tree Frog Technologies, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 022607 | /0824 | |
Sep 28 2006 | P. Milton Holdings, LLC | (assignment on the face of the patent) | / | |||
Aug 11 2015 | P MILTON HOLDINGS, LLC | BENHOV GMBH, LLC | MERGER SEE DOCUMENT FOR DETAILS | 037094 | /0504 | |
Mar 16 2020 | BENHOV GMBH, LLC | INTELLECTUAL VENTURES ASSETS 137 LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 052623 | /0004 | |
May 05 2020 | INTELLECTUAL VENTURES ASSETS 137 LLC | PINEAPPLE34, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 055280 | /0898 |
Date | Maintenance Fee Events |
Feb 24 2012 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Feb 23 2016 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Aug 25 2012 | 4 years fee payment window open |
Feb 25 2013 | 6 months grace period start (w surcharge) |
Aug 25 2013 | patent expiry (for year 4) |
Aug 25 2015 | 2 years to revive unintentionally abandoned end. (for year 4) |
Aug 25 2016 | 8 years fee payment window open |
Feb 25 2017 | 6 months grace period start (w surcharge) |
Aug 25 2017 | patent expiry (for year 8) |
Aug 25 2019 | 2 years to revive unintentionally abandoned end. (for year 8) |
Aug 25 2020 | 12 years fee payment window open |
Feb 25 2021 | 6 months grace period start (w surcharge) |
Aug 25 2021 | patent expiry (for year 12) |
Aug 25 2023 | 2 years to revive unintentionally abandoned end. (for year 12) |