A tactile sensor includes a photosensing structure, a volume of elastomer capable of transmitting an image, and a reflective skin covering the volume of elastomer. The reflective skin is illuminated through the volume of elastomer by one or more light sources, and has particles that reflect light incident on the reflective skin from within the volume of elastomer. The reflective skin is geometrically altered in response to pressure applied by an entity touching the reflective skin, the geometrical alteration causing localized changes in the surface normal of the skin and associated localized changes in the amount of light reflected from the reflective skin in the direction of the photosensing structure. The photosensing structure receives a portion of the reflected light in the form of an image, the image indicating one or more features of the entity producing the pressure.
|
0. 44. A sensor comprising:
a roller having an outer surface and an inner space;
a photosensing structure disposed inside the inner space of the roller;
a volume of elastomer capable of transmitting an image disposed on the outer surface of the roller;
the volume of elastomer having a reflective surface, the reflective surface reflecting light incident on the reflective surface from within the volume of elastomer, the reflective surface being geometrically altered in response to pressure applied by an entity touching the reflective surface, the geometrical alteration causing localized changes in the surface normal of the reflective surface and associated localized changes in the amount of light reflected from the reflective surface in the direction of the photosensing structure, and
the photosensing structure being positioned to receive a portion of the reflected light in the form of an image, the image indicating one or more features of the entity producing the pressure.
0. 27. A method of performing tactile sensing, comprising:
providing a roller having an outer surface and an inner space;
providing a volume of elastomer capable of transmitting an image disposed on the outer surface of the roller, the volume of elastomer having an inner surface in contact with the outer surface of the roller and a reflective surface, opposite the inner surface, that reflects light incident on the reflective surface from within the volume of elastomer;
illuminating the reflective surface through the volume of elastomer with a light source, at least a portion of the light being reflected by the reflective surface;
rolling the reflective surface along a surface of an entity, thereby sequentially contacting adjacent portions of the reflective surface with adjacent portions of the entity, the contact producing pressure that geometrically alters the contacted portion of the reflective surface, the alteration causing localized changes in the contacted portion of the reflective surface, and the localized changes in the contacted portion of the reflective surface causing associated localized changes in the light reflected from the contacted portion of the reflective surface; and
positioning a photosensing structure inside the inner space of the roller to receive a portion of the light reflected from the reflective surface in the form of a sequence of images resulting from the sequential contact with portions of the entity, each image of the sequence indicating one or more features of the portion of the entity contacting the reflective surface.
0. 1. A tactile sensor comprising:
a photosensing structure;
a volume of elastomer capable of transmitting an image; and
a reflective skin covering said volume of elastomer, said reflective skin being illuminated through said volume of elastomer by one or more light sources, said reflective skin having particles that non-directionally reflect light incident on the reflective skin from within the volume of elastomer, said reflective skin being geometrically altered in response to pressure applied by an entity touching said reflective skin, said geometrical alteration causing localized changes in the surface normal of said skin and associated localized changes in the amount of light reflected from said reflective skin in the direction of said photosensing structure; wherein
said photosensing structure is positioned to receive a portion of said reflected light in the form of an image, said image indicating one or more features of the entity producing said pressure.
0. 2. The tactile sensor of
0. 3. The tactile sensor of
0. 4. The tactile sensor of
0. 5. The tactile sensor of
0. 6. The tactile sensor of
0. 7. The tactile sensor of
0. 8. The tactile sensor of
0. 9. The tactile sensor of
0. 10. The tactile sensor of
0. 11. The tactile sensor of
0. 12. The tactile sensor of
0. 13. The tactile sensor of
0. 14. A method of performing tactile sensing, comprising:
(a) providing a volume of elastomer capable of transmitting an image;
(b) covering the elastomer with a reflective skin having an inner surface facing the elastomer and an outer surface, wherein the reflective skin comprises particles that non-directionally reflect light incident on the inner surface from within the volume of elastomer;
(c) illuminating the reflective skin through the volume of elastomer with one or more light sources, wherein at least a portion of the light is reflected by the inner surface of the reflective skin;
(d) contacting the outer surface of the reflective skin with an entity, wherein the contact produces pressure that geometrically alters the reflective skin, wherein the alteration causes localized changes in the inner surface of the reflective skin, and wherein the localized changes in the inner surface of the reflective skin cause associated localized changes in the light reflected from the inner surface of the reflective skin;
(e) positioning a photosensing structure to receive a portion of the light reflected from the inner surface of the reflective skin in the form of an image indicating one or more features of the entity contacting the outer surface of the reflective skin.
0. 15. The method of
0. 16. The method of
0. 17. The method of
0. 18. The method of
0. 19. The method of
0. 20. The method of
0. 21. The method of
0. 22. The method of
0. 23. The method of
0. 24. The method of
0. 25. The method of
0. 26. The method of
0. 28. The method of claim 27, further comprising combining the sequence of images into a single image.
0. 29. The method of claim 28, wherein the combining includes stitching the sequence of images into a panoramic single image.
0. 30. The method of claim 27, the rolling the reflective surface along a surface of an entity comprising holding the roller substantially stationary and moving the entity along the reflective surface.
0. 31. The method of claim 27, the rolling the reflective surface along a surface of an entity comprising holding the entity substantially stationary and moving the reflective surface along the entity.
0. 32. The method of claim 27, further comprising:
providing a drive wheel disposed in proximity to the reflective surface;
disposing the entity between the drive wheel and the reflective surface; and
rotating the drive wheel to cause the adjacent portions of the entity to pass between the drive wheel and the adjacent portions of the reflective surface, thereby rolling the reflective surface along the surface of the entity.
0. 33. The method of claim 27, the reflective surface comprising reflective particles that reflect light substantially uniformly in all directions, the reflective particles causing the reflective surface to reflect light substantially non-directionally.
0. 34. The method of claim 27, the reflective surface comprising flakes that exhibit at least one of surface roughness, irregular shape, and random alignment relative to each other, the flakes causing the reflective surface to reflect light moderately directionally.
0. 35. The method of claim 27, the reflective surface comprising flakes that are substantially flat, have a mirror-like surface, and are substantially well aligned relative to each other, the flakes causing the reflective surface to reflect light highly directionally.
0. 36. The method of claim 27, the volume of elastomer comprising at least one of silicone rubber, polyurethane, plastisol, thermoplastic elastomer, natural rubber, polyisoprene, polyvinyl chloride.
0. 37. The method of claim 27, the volume of elastomer comprising a Shore A hardness between 5 and 90.
0. 38. The method of claim 27, the photosensing structure comprising a camera.
0. 39. The method of claim 27, the photosensing structure comprising an array of sensing elements.
0. 40. The method of claim 27, the one or more features comprising roughness of the entity.
0. 41. The method of claim 27, the one or more features comprising the location, amplitude, or direction of the applied pressure.
0. 42. The method of claim 27, the one or more features comprising at least one of the shape, size, and profile of the entity producing the pressure.
0. 43. The method of claim 27, the reflective surface being illuminated by at least two light sources of different colors.
0. 45. The sensor of claim 44, further comprising a drive wheel disposed in proximity to the reflective surface, the drive wheel positioned such that rotation of the drive wheel causes the entity to pass between the drive wheel and a portion of the reflective surface, thereby causing the roller to rotate and causing adjacent portions of the reflective surface to sequentially contact adjacent portions of the entity.
0. 46. The sensor of claim 44, the reflective surface comprising reflective particles that reflect light substantially uniformly in all directions, the reflective particles causing the reflective surface to reflect light substantially non-directionally.
0. 47. The sensor of claim 44, the reflective surface comprising flakes that exhibit at least one of surface roughness, irregular shape, and random alignment relative to each other, the flakes causing the reflective surface to reflect light moderately directionally.
0. 48. The sensor of claim 44, the reflective surface comprising flakes that are substantially flat, have a mirror-like surface, and are substantially well aligned relative to each other, the flakes causing the reflective surface to reflect light highly directionally.
0. 49. The sensor of claim 44, the volume of elastomer comprising at least one of silicone rubber, polyurethane, plastisol, thermoplastic elastomer, natural rubber, polyisoprene, polyvinyl chloride.
0. 50. The sensor of claim 44, the volume of elastomer comprising a Shore A hardness between 5 and 90.
0. 51. The sensor of claim 44, the photosensing structure comprising a camera.
0. 52. The sensor of claim 44, the photosensing structure comprising an array of sensing elements.
0. 53. The sensor of claim 44, the one or more features comprising roughness of the entity.
0. 54. The sensor of claim 44, the one or more features comprising the location, amplitude, or direction of the applied pressure.
0. 55. The sensor of claim 44, the one or more features comprising at least one of the shape, size, and profile of the entity producing the pressure.
|
FIG. 13 is a schematic diagram illustrating the elements of a rolling scanner.The sensor skin can be made by adding reflective particles to the elastomer when it is in a liquid state, via solvent or heat, or before curing. This makes a reflective paint that can be attached to the surface by standard coating techniques such as spraying or dipping. The skin may be coated directly on the surface of the bulk elastomer, or it may be first painted on a smooth medium such as glass and then transferred to the surface of the bulk material and bound there. Also, the particles (without binder) can be rubbed into the surface of the bulk elastomer, and then bound to the elastomer by heat or with a thin coat of material overlaid on the surface. Also, it may be possible to evaporate, precipitate, sputter, other otherwise attach thin films to the surface.
The reflective particles in the skin may reflect light directionally or non-directionally. If the particles reflect light uniformly in all directions regardless of the light's angle of incidence, the resulting skin will behave like a Lambertian surface, which is entirely non-directional. Titanium dioxide powder, as is used in white paint, leads to a largely Lambertian reflectance. If the reflective particles are comprised of fine metal flakes, and if these flakes tend to be aligned with each other, then the skin will reflect light directionally, meaning that, for a given angle of incident light, there will be a non-uniform distribution of reflected light. If the metal flakes are flat and mirror-like, and if they are well aligned with each other, the distribution of reflected light will be highly directional. If the metal flakes are rough or irregular, or if there is randomness in their alignment, then the distribution of reflected light will be moderately directional, with an appearance resembling sandblasted metal. Directional reflectance can also be obtained with flakes of other materials such as mica. In addition there are pigments comprising flakes covered with multilayer interference coatings that can have different directionality for different wavelengths of light.
Further still, metal flake powder of aluminum can be used.
Skin with highly directional reflectance illuminated by a highly directional light source yields a device that is sensitive to small variations in pressure. This sensitivity can be increased by recording the skin's image in its resting state, and using this as a baseline image that is subtracted from images recorded when pressure is applied to the skin. Softer elastomers lead to devices that are more sensitive to low amplitude pressure patterns.
Another example is shown in
The image pixel values do not directly encode pressure. If spatially uniform pressure is applied to the entire skin surface, there will be no change in surface normal and thus no observable variation in the image. The image pixel values depend on surface normal, which in turn depends on the spatial derivative of pressure. Thus, it is the pattern of pressure variation across the surface that is encoded in the image.
Pressure can be applied to the skin by a rigid object or a non-rigid object. In the case of a non-rigid object, such as a fingertip, both the object and the skin will deform, and the skin's shape will depend on the balance of pressures that the skin and the object exert on each other. Pressure can also be applied by a liquid or gas. For example, a stream of water striking the skin causes it to deform, and the pattern of deformation is visible in the image. If the skin and the elastomer are made of very soft gel-like materials, and if a froth of soap bubbles is placed in contact with the skin, one can visualize the forces exerted by the soap bubble walls.
To make the structure compact, it may be preferable to introduce the light at the edge of the support.
In some applications it is desirable that the light source and the camera be placed at optical infinity so that the angle of incidence and reflectance are parallel when the device is in its resting state. This causes the devices optical properties to be spatially uniform across the recorded image.
In accordance with another exemplary embodiment of the invention, it is desirable to reconstruct the 3-D shape of the deformed surface. In
The use of multiple lights to get multiple images is useful even when 3D reconstruction is not being performed. Each light brings out surface normal variation along one axis, but not along the orthogonal axis. By using two or more lights, the lights can be arranged so that one light reveals the relief that is missed by another light. This makes it possible to distinguish a wide range of surface normals in different directions. The preferred method of using two or more lights is to have them be different colors, so that a color camera will separate the information about the different lights into different color channels.
It is also possible to use standard stereoscopic techniques as well, in which multiple cameras are placed in different azimuths relative to the skin of the sensor. The techniques set forth herein of sensing changes in luminance are believed to yield better sensitivity and resolution in many applications relative to known techniques. Furthermore, stereopsis can be used in combination with the techniques disclosed herein. The microtexture of the skin and/or the deformation image can be used to establish correspondence.
In some applications it is desirable to make a sensor surface that covers a large area. For example one may require a touchpad that covers an entire desktop. If the device is simply scaled up, then the camera must be placed at a large distance from the surface, making the device undesirably large. One way to ameliorate this problem is to use the methods of folded optics that are used, for example, in many rear projection televisions. Another way is to use a tiled array of cameras, as shown in
In another embodiment, a sensor is formed into a cylinder, which can be rolled over the surface of the object. In one illustrative configuration, the sensor would look like a brayer or paint roller. As the roller rolls over the surface, a video camera inside the roller is aimed continuously at the portion that is in contact with the surface of the object. The series of images so obtained can be combined into a single image by the same methods that are used to obtain panoramic photographs from a series of smaller photographs.
Alternatively, the roller could be in a fixed position while the surface of interest was pulled over it. For example, as shown in FIG. 13, an illustrative embodiment of a rolling scanner is a counterfeit banknote detector 300 which has a clear elastomer 310 formed into a cylinder, and a reflective skin 320. The detector 300 also has a slot 330 for feeding a banknote 340. A roller 350, driven by a motor, pulls in the banknote 340 and forms an image of the banknote's surface shape based on an embossed surface of the banknote's paper and/or the raised level of printed ink. A genuine banknote will have a known profile, and a mismatch would indicate a counterfeit banknote.
In some applications it is advantageous for the skin to have a texture rather than being smooth. In some situations one wishes to study the distribution of pressures across a region of human skin. For example, when a skin care product is applied to the skin, the application process produces a certain distribution of pressure on the skin which changes over time. In order to estimate this changing distribution, a sensor can be made that mimics the texture, elasticity, and other properties of human skin. When a skin care product is, for example, wiped across the artificial skin, it causes the skin to distort in a manner similar to that of human skin. The pattern of distortion can be assessed by making a tactile sensor with mechanical properties emulating human skin. This means that the reflective skin has texture and elasticity like the upper layer of human skin, and the clear elastomer beneath the skin has mechanical properties like the deeper layers of human skin. Multiple layers of clear elastomer with different mechanical properties are required to mimic the complex properties of human skin. When a skin care product is applied to this device, the reflective skin distorts in response to the mechanical forces applied to it, and this distortion is viewed by a camera looking through the clear elastomer layers.
In some applications it may be desired to study the distribution of pressure over the surface of a specific object, such as a tire or the sole of a shoe. It is possible to form the tactile sensor into the same shape as this specific object, and with the same hardness or other mechanical properties as this specific object.
In other embodiments, it is not necessary that the image be formed by a camera. Many flatbed scanners use a Contact Image Sensor, or CIS, which is a linear array of lenses and photosensors placed in close contact with the object being scanned. No image forming lens is required.
In another embodiment, a multitouch touchscreen device is made in conjunction with a flat panel LED display. A thin sheet of clear elastomer, covered with a semi-reflective skin, covers the front surface of the display. Most of the light that is emitted by the LED's passes through the skin and is seen by a viewer. A portion of the light is reflected by the skin back toward the LEDs. LEDs have the ability to act as photosensors, and thus can be used to measure the amount of reflected light. Pressure variation on the skin causes local changes in the surface normal, which changes the amount of light reflected toward any given LED in the array. The LED photosensing responses comprise an image that is indicative of the pattern of pressure on the skin. This image indicates where the user is touching the screen. In addition, because this is an inherently high resolution image, it is possible to detect the fingerprint of the user. This allows each finger of each user to be distinguished.
In another exemplary embodiment, the device is used to measure fluid flow.
There are applications for which high resolution is not needed and not desirable. An extra layer of elastomer on top of the skin acts as a mechanical lowpass filter. For example, a 1 mm thick layer reduces the resolution to be on the order of 1 mm.
A fluorescent pigment can be used in the skin, illuminated by Ultraviolet (UV) light or blacklight. If the blacklight comes at a grazing angle, it can readily reveal variations in surface normal. The material will be fairly close to Lambertian. To reduce interreflections, one would select a surface that appears dark to emitted wavelengths. This principle is true with ordinary light as well. If one is using a Lambertian pigment in the skin, it is better for it to be gray than white, to reduce interreflections.
Blacklight or UV can be used to illuminate a fluorescent surface, which would then serve as a diffuse source. In some cases, it would be useful to use a single short flash (for instance, recording the instantaneous deformation of an object against the surface) or multiple periodic (strobed) flashes (to capture rapid periodic events or to modulate one frequency down to another frequency.)
Although the present invention has been shown and described with respect to several preferred embodiments thereof, various changes, omissions and additions to the form and detail thereof, may be made therein, without departing from the spirit and scope of the invention.
Patent | Priority | Assignee | Title |
10837916, | Mar 09 2017 | Spirit AeroSystems, Inc. | Optical measurement device for inspection of discontinuities in aerostructures |
Patent | Priority | Assignee | Title |
4120585, | Nov 19 1976 | FINGERMATRIX, INC , A NY CORP | Fingerprint identification system using a pliable optical prism |
4340300, | Aug 11 1980 | Siemens Corporation | Input sensor unit for a fingerprint identification system |
4481815, | Dec 23 1982 | Tactile sensor | |
4549093, | Sep 29 1982 | Emi Limited | Tactile array sensor |
4584625, | Sep 11 1984 | Capacitive tactile sensor | |
4588348, | May 27 1983 | AT&T Bell Laboratories | Robotic system utilizing a tactile sensor array |
4775961, | Jun 24 1985 | VYSOKA SKOLA DOPRAVY A SPOJOV V ZILINE, A CORP OF 20 MOYZESOVA ZILINA, CZECHOSLOVAKIA | Tactile sensor |
5357799, | Jul 14 1992 | Regents of the University of California, The | Method and apparatus for determining abrasion potential of tire treads |
5448649, | May 24 1994 | Apparatus for imaging fingerprint or topographic relief pattern on the surface of an object | |
5616839, | Jan 29 1996 | Ford Motor Company | Apparatus and method for measuring contact force distribution of a tire |
5737071, | Aug 16 1996 | Safariland, LLC | Method and apparatus for enhancing live-scan fingerprint reader images |
5967940, | Sep 17 1997 | Toyota Jidosha Kabushiki Kaisha | Method and apparatus for reducing backlash sound in gear mechanism |
5967990, | Aug 13 1998 | President and Fellows of Harvard College | Surgical probe comprising visible markings on an elastic membrane |
6144757, | Feb 28 1997 | NEC Corporation | System and method for identifying an organism |
6234031, | Nov 27 1997 | NEC Corporaion | Fingerprint detection apparatus |
6909084, | Aug 31 2000 | Toudai TLO, Ltd | Optical tactile sensor having a transparent elastic tactile portion |
7707001, | Mar 09 2004 | Nagoya Industrial Science Research Institute | Control of object operating force, object gripping force and robot hands |
20030178556, | |||
20040237669, | |||
20040252867, | |||
20040255128, | |||
20060119837, | |||
20070146536, | |||
20070288186, | |||
20080106258, | |||
20080219521, | |||
20080284925, | |||
20100284565, | |||
JP2005121858, | |||
JP2007520040, | |||
JP2008008746, | |||
WO2005085766, | |||
WO2006098719, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Oct 03 2013 | Massachusetts Institute of Technology | (assignment on the face of the patent) | / | |||
May 05 2015 | Massachusetts Institute of Technology | NATIONAL SCIENCE FOUNDATION | CONFIRMATORY LICENSE SEE DOCUMENT FOR DETAILS | 035635 | /0406 |
Date | Maintenance Fee Events |
Oct 29 2014 | ASPN: Payor Number Assigned. |
Oct 02 2020 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Oct 02 2024 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Dec 02 2017 | 4 years fee payment window open |
Jun 02 2018 | 6 months grace period start (w surcharge) |
Dec 02 2018 | patent expiry (for year 4) |
Dec 02 2020 | 2 years to revive unintentionally abandoned end. (for year 4) |
Dec 02 2021 | 8 years fee payment window open |
Jun 02 2022 | 6 months grace period start (w surcharge) |
Dec 02 2022 | patent expiry (for year 8) |
Dec 02 2024 | 2 years to revive unintentionally abandoned end. (for year 8) |
Dec 02 2025 | 12 years fee payment window open |
Jun 02 2026 | 6 months grace period start (w surcharge) |
Dec 02 2026 | patent expiry (for year 12) |
Dec 02 2028 | 2 years to revive unintentionally abandoned end. (for year 12) |