This method and apparatus optically samples numerous points on the surface of an object to remotely sense its shape utilizing two stages. The first stage employs a moveable non-contact scanner, which in normal operation sweeps a narrow beam of light across the object, illuminating a single point of the object at any given instant in time. The location of that point relative to the scanner is sensed by multiple linear photodetector arrays behind lenses in the scanner. These sense the location by measuring the relative angular parallax of the point. The second stage employs multiple fixed but widely separated photoelectronic sensors, similar to those in the scanner, to detect the locations of several light sources affixed to the scanner, thereby defining the absolute spatial positions and orientations of the scanner. Individual light sources are distinguished by time-multiplexing their on-off states. A coordinate computer calculates the absolute spatial positions where the scanner light beam is incident on the object at a given instant and continuously on a real time basis to generate a computer model of the object.

Patent
   RE35816
Priority
Mar 30 1995
Filed
Mar 30 1995
Issued
Jun 02 1998
Expiry
Jun 02 2015
Assg.orig
Entity
Large
285
32
all paid
1. Optical mensuration apparatus for mapping and recording the locations of points on a surface of a three dimensional object comprising:
a mounting structure, and an object positioned in immovable relation to said mounting structure, and a three dimensional coordinate system defined in fixed relation to said mounting structure;
scanning means a scanner for projecting a scanning beam onto the a surface of the an object to illuminate a plurality of spots on the surface of the object;
said scanning means scanner being hand holdable and freely moveable by hand in relation to both said mounting structure and said object and not connected mechanically or structurally to either said mounting structure and said object;
a spot detector means mounted to said scanning means scanner for detecting the positions locations of the illuminated spots on the surface of the object in relation to said scanning means scanner;
a position detecting means mounted on said mounting structure and detector remotely located from both said object and said scanning means for detecting the position of said scanning means scanner in known relation to the said coordinate system, which position detector is adapted to determine the position of said scanner in relation to said three dimensional coordinate system; and
computing means a computer connected to said scanning means scanner and to said position detecting means detector for determining and recording the positions locations of said illuminated spots on the surface of the object in relation to the coordinate system by correlating the positions locations of said illuminated spots in relation to said scanning means scanner with the respective positions of said scanning means scanner in relation to said coordinate system when each a respective spot is illuminated.
2. The optical mensuration apparatus of claim 1, wherein said spot detector means comprises a plurality of one dimensional spot sensing means sensors in spaced apart relation for sensing the position locations of the illuminated spots on the surface of the object.
3. The optical mensuration apparatus of claim 2, wherein each of said one dimensional spot sensing means sensors comprises:
a linear photodetector; and
a lens positioned between said linear photodetector and said illuminated spot on the object for focusing light from said illuminated spot onto said linear photodetector.
4. The optical mensuration apparatus of claim 3, wherein said position detecting means detector comprises:
a plurality of pilot light source means sources mounted on said scanning means scanner for projecting a plurality of pilot light rays; and
a plurality of one-dimensional pilot light sensing means sensors in spaced apart relation remotely located from said scanning means position detector for sensing the positions locations of each of said plurality of pilot light source means sources.
5. The optical mensuration apparatus of claim 4, wherein each said one-dimensional pilot light sensing means sensors comprises:
a linear photodetector; and
a lens positioned between said linear photodetector and said plurality of pilot light source means sources for focusing light from said plurality of pilot light source means sources onto said linear photodetector.
6. The optical mensuration apparatus of claim 5, wherein each of said plurality of light source means sources is strobed off and on in a predetermined manner.
7. The optical mensuration apparatus of claim 5, wherein said scanning means scanner comprises:
at least one light source means for producing said scanning beam; and
a corresponding number of scanning beam direction means directors for directing said scanning beam over the surface of the object.
8. The optical mensuration apparatus of claim 7, wherein said light source means for producing said scanning beam is a laser.
9. The optical mensuration apparatus of claim 7, wherein said scanning beam direction means director is a rotating mirror having at least three sides.
10. The optical mensuration apparatus of claim 9, wherein each said lens of each said one-dimensional spot sensing means sensor is a cylindrical lens.
11. The optical mensuration apparatus of claim 9, wherein each said lens of each said one-dimensional pilot light sensing means sensor is a cylindrical lens.
12. A method of determining and mapping the locations of surface points on an object in relation to a mounting structure three dimensional coordinate system, comprising the steps of:
defining a three dimensional coordinate system in fixed relation to said mounting structure;
positioning said object in a fixed spatial relation to said mounting structure coordinate system;
projecting a scanning scannable illuminating beam from a beam projector, mounted on a hand holdable and freely moveable scanning device that is not connected mechanically or structurally to either said mounting structure or the object, and moving the scanning device by hand in relation to said object in such manner as to illuminate a plurality of spots on the a sufficient portion of a surface of the object to map said surface;
scanning said surface with a hand holdable and freely moveable scanner to detect a sufficient portion of said projected beam illuminations to map said illuminated surface portion;
detecting the positions locations of the respectively illuminated spots on portions of the surface of the object in relation to the respective positions of the scanning device scanner when each respective spot portion of the surface is illuminated;
projecting a plurality of pilot light rays from a plurality of pilot light sources positioned in fixed spatial relation to each other on said scanning device scanner substantially simultaneously with the steps of projecting said scanning illuminating beam and detecting the positions locations of the illuminated spots surface portions;
detecting the plurality of pilot rays with a plurality of detectors mounted on said mounting structure in fixed in known relation to said coordinate system and in fixed known, spaced apart relation to each other substantially simultaneously with the step of detecting the positions locations of said illuminated spots surface portions on said object in relation to said scanning device scanner to determine the positions locations of the plurality of pilot light sources and said scanning device scanner in relation to the coordinate system; and
computing the positions locations of the illuminated spots on portions of the surface of the object in relation to the coordinate system by correlating the positions locations of said illuminated spots surface portions in relation to the scanning device scanner with the position locations of the scanning device scanner in relation to said coordinate system. 13. Optical mensuration apparatus for mapping and recording the locations of points on a surface of a three dimensional object as claimed in claim 1 further comprising a mounting structure, fixedly positioned in said three dimensional coordinate system, to which said object is immovably related, and wherein said position detector is mounted on said mounting structure.14. An optical system as claimed in claim 13 further comprising:
multiple energy emitters disposed on said spot detector;
emitted energy detectors disposed in known relationship to said three dimensional coordinate system sufficient to detect energy emitted by said energy emitters; and
a computer operatively associated with said energy detectors adapted to calculate the position and orientation of said light detector in said three dimensional coordinate system.15. An optical system for determining locations of a plurality of points on a portion of a surface of a three dimensional object, in relation to a three dimensional coordinate system in which said object resides, wherein the number of points on said surface portion is sufficient to map said surface portion, said system comprising:
at least one three dimensional object having at least one surface positioned in a three dimensional coordinate system;
a beam projector unconnected mechanically or structurally to said object, and freely moveable in relation to said object;
at least one scannable beam adapted to be projected from said projector onto a surface of said object and to thereby illuminate said plurality of points on said surface portion;
a scanner comprising at least one light detector, unconnected mechanically or structurally to said object, and freely moveable in relation to said object, and adapted to detect locations of said illuminated points on said surface of said object in said three dimensional coordinate system in relation to said spot detector;
means to maintain said object in a substantially stationary condition during said illumination and detection of at least three of said plurality of spots sufficient in number to map said surface portion of said object;
at least one light detector locator disposed in known position in said three dimensional coordinate system, for optically detecting the position and orientation of said light detector in relation to said three dimensional coordinate system; and
a computer connected to said light detector and to said detector locator for correlating respectively the locations of said illuminated portions of said surface of said object in relation to said light detector and the location of said light detector in relation to said three dimensional coordinate system;
whereby indirectly determining the locations of each of said illuminated surface portions with respect to said three dimensional coordinate system, and therefore mapping said portion of said surface of said
object.16. An optical system as claimed in claim 15 wherein said light detector is located together with said illuminating beam projector.17. An optical system as claimed in claim 15 wherein said light detector locator is located together with said
object.18. An optical system as claimed in claim 15 wherein said scanner is hand held and is moved by hand.19. An optical system as claimed in claim 15 further comprising said locator being adapted to locate said light detector at substantially the same time as the locations of each of said illuminated surface portions are being detected by said light detector.20. An optical system as claimed in claim 15 wherein said object is in a fixed position in said three dimensional coordinate system.21. An optical system as claimed in claim 20 wherein said object is attached to a mounting structure which is in fixed spatial relationship to said three dimensional coordinate system, wherein said scanner has said light detector affixed thereto, and wherein said light detector locator in fixed relationship to
said mounting structure.22. A method of mapping at least a portion of a surface on an object, which object is in a known position and orientation in a three dimensional coordinate system, comprising:
disposing an object, comprising at least one surface, in a known position and orientation in a three dimensional coordinate system;
disposing a hand holdable scanner in said coordinate system, unconnected mechanically or structurally to said object and freely moveable within said three dimensional coordinate system, so positioned that it can scan said surface;
projecting a plurality of spots onto said surface portion;
detecting the locations on said surface portion, in relation to a spot detector, of a sufficient number of spots to map at least said portion of the surface;
at substantially the same time as the locations of the respective spots are being determined, determining the location of said spot detector in said coordinate system;
correlating the locations of said respective spots with the position and orientation of said spot detector; thereby
indirectly determining the locations of said illuminated spots in said three dimensional coordinate system; and
mapping said surface.

1. Field of the Invention

This invention relates to optical mensuration devices in general, and in particular to an improved method and apparatus for the optical mensuration of the surface shape of a three-dimensional object.

2. Brief Description of the Prior Art

Numerous mensuration systems exist in the prior art for sensing the locations of surface points on three-dimensional solid objects in relation to a predefined fixed reference frame or coordinate system for input into an application system, such as a computer or other device for measurement or analysis. For example, one type of mensuration system that can be used to determine the location of a single point on the surface of an object includes the use of a narrow projected beam of light to illuminate a tiny area or spot on the surface of the object. A lens in the system is positioned on an optical axis oblique to the axis of the projected beam and is used to focus the reflected light from the illuminated spot onto a photoelectric sensor or onto a linear array of sensors. Since the optical axis of the lens and sensor assembly in that type of system is not coincident with the axis of the projected beam, the position of the image of the illuminated spot on the sensor will depend on the location of the particular illuminated surface point with respect to the illuminating beam. Therefore, the location of the illuminated point with respect to the predetermined reference frame can be determined by computing the distance of the illuminated surface point from the origin of the light beam which, of course, is known. Examples of such point illumination optical mensuration systems are found in the following U.S. Pat. Nos. 4,660,970; 4,701,049; 4,705,395; 4,709,156; 4,733,969; 4,743,770; 4,753,528; 4,761,072; 4,764,016; 4,782,239; and 4,825,091.

Of course, to determine the overall shape of an object, numerous individual surface points, along with their respective locations, must be measured and recorded. Such optical measurement of multiple surface points of an object is typically accomplished by mounting the beam projector on a moveable scanning head capable of being moved from point-to-point with very high precision, such as the type commonly found on numerically controlled milling machines. By precisely moving the beam projector mounted on the scanning head in a raster-like scanning pattern, it is possible to measure the surface shape of the object being scanned by measuring the individual locations of surface points individually illuminated by the point-like scanning beam as it is scanned over the object's surface. Alternatively, the object itself can be moved while the scanning head remains stationary. One disadvantage of this type of system is that only one side of the object may be scanned at any one time, since other sides of the object are hidden by the side being scanned. Scanning of these hidden sides can only be accomplished by relocating either the scanning head or the object to expos the previously hidden surfaces to the scanning beam. Obviously, such a relocation requires time and precision equipment to keep track of the changed position of the scanning head, or the object in relation to the fixed reference frame so that the new surface data will correspond to the previously obtained surface data. Helical or three-dimensional scanning heads solve this problem by allowing the entire object to be scanned at once. However, such helical systems are relatively expensive, since they require complex mechanical apparatus to move the scanning head around the object in three-dimensions.

Regardless of the scanning method used, however, deep holes, overhangs, undercuts, and surfaces nearly parallel to the axis of the scanning beam reduce the accuracy of the system, since it is difficult to accurately measure these points, if they can even be illuminated by the scanning beam at all. For example, such systems cannot completely scan the inside, outside, and handle details of a coffee cup without requiring the scanning apparatus to be relocated or the object to be reoriented so that the inside surfaces or other surfaces previously hidden from the scanning beam can be illuminated by the beam, thus measured and recorded. As discussed earlier, such re-locations or re-orientations have the disadvantage of having to recalibrate the scanning apparatus, or otherwise recorrelate the new surface points with respect to the original coordinate system. Moreover, even if such relocations or reorientations are not required, such as in the case of a helical scanning apparatus, there is still a severe loss of accuracy when scanning near the top or bottom of a rounded object, unless the scanning head and detector are relocated to better illuminate and detect such points. Furthermore, these types of systems are not very portable or adaptable since they require high precision electro-mechanical or other apparatus to accurately move the scanning heads (or the object) and define their positions in relation to the predetermined reference frames. Therefore, all these prior art scanning systems will usually require some type of relocation of the scanning apparatus or reorientation of the object to completely measure and record all of the surface details.

A variant of the above-described systems projects a thin beam of light in a single plane which, of course, is incident as a line, as opposed to a point, on the surface of the object being scanned. The intersection of this plane of light with the object's surface thus forms a brightly illuminated contour line. A two-dimensional electronic video camera or similar device whose optical axis is not coincident with the axis of the illuminating beam, detects the image of this contour line. Again, since the optical axis of the camera is not coincident with the axis of the illuminating light beam, it views the contour line from an oblique angle, thus allowing location of the contour line to be precisely determined in relation to the known position of the beam projector. Examples of inventions using this type of system are found in the following U.S. Pat. Nos. 4,821,200; 4,701,047; 4,705,401; 4,737,032; 4,745,290; 4,794,262; 4,821,200, 4,743,771; and 4,822,163.

To measure more than one contour line of an object, either the measuring apparatus or the object is panned along (or rotated about) an axis through the object. While these line scanning devices share similar drawbacks with the point scanning devices previously described, they do operate much faster, gathering a larger number of sample points during a given scanning interval. Unfortunately, the accuracy of each surface sample point is limited by the relatively low resolution of the two-dimensional charge coupled device (CCD) sensors found in most video cameras, which is typically in the range of 1 part in 512. Even worse, these systems still suffer the disadvantages of the point scanning systems in that either the scanning head or the object must be relocated or re-oriented to completely and accurately record all of the surface details of an object.

Still other mensuration systems track the positions of specific points in three-dimensional space by using small radiating emitters which move relative to fixed receiving sensors, or vice versa. Such radiation emitters may take the form of sound, light, or nutating magnetic fields. Another mensuration system uses a pair of video cameras plus a computer to calculate the position of homologous points in the pair of stereographic video images. See, for example, U.S. Pat. Nos. 4,836,778 and 4,829,373. The points tracked by this system may be passive reflectors or active light sources. The latter simplifies finding and distinguishing the points.

Additional prior art relevant to this patent application are found in the following references:

Burton, R. P.; Sutherland, I. E.; "Twinkle Box--a three dimemsional computer input device", National Computer Conference, AFIPS Proceedings, v 43, 1974, p 513-520;

Fischer, P.; Mesqui, F.; Kaeser, F.; "stereometric measurement system for quantification of object forms", SPIE Biostereometrics 602, 1985, p 52-57;

Fuchs, H.; Duran, J.; Johnson, B.; "Acquisition and Modeling of Human Body Form Data", Proc. SPIE, v 166, 1978, p 94-102;

Macellari, V.; "A Computer Peripheral Remote Sensing Device for 3-Dimensional; Monitoring of Human Motion", Med. & Biol. Eng. & Comput., 21, 1983, p 311-318;

Mesqui, F.; Kaeser, F.; Fischer, P.; "real-time, noninvasive recording and 3-d display of the functional movements of an arbitrary mandible point", SPIE Biostereometrics 602, 1985, p 77-84;

Yamashita Y.; Suzuki, N.; Oshima, M.; "Three-Dimensional Stereometric Measurement System Using Optical Scanners, Cylindrical Lenses, and Line Sensors", Proc. SPIE, v. 361, 1983, p. 67-73.

In particular, the paper by Fuchs, et al, (1978) describes a basic method of tracking a light source in three-dimensional space. The method is based on using three or more one-dimensional sensors, each consisting of a cylindrical lens and a linear array of photodetectors, such as charge coupled devices (CCDs), to determine the location of the currently radiating source.

Numerous other methods have been devised and patented for determining the position of a point along a line, within a plane, or in three-dimensional space. Devices employing these methods include photographic camera rangefinders, tablet digitizers, coordinate measuring machines, and surveying tools. Some exploit sound, magnetic fields, or mechanical apparatus for mensuration, and there are other devices employing x-rays, nuclear magnetic resonance, radar, sonar, and holography to sense the shapes of objects.

Unfortunately, each of the above mensuration systems has its own set of drawbacks, which include high cost, poor accuracy, poor resolutions, awkward or difficult use, limitations on geometrical complexity, excessive numerical computation, or slow measurement speed. Experience has shown that no single prior art system best suits all three-dimensional measurement applications. For example, there is no existing mensuration device that can perform even straightforward anatomical measurements of a person without significant drawbacks.

Thus, there remains a need for a non-contact, three-dimensional optical mensuration system which is capable of accurate, speedy, convenient, and inexpensive sensing of three-dimensional geometric shapes or objects. Ideally, the scanning head of such an improved system should be hand-held to allow the operator to easily move the scanning beam over some of the more complex surface details of the object while dispensing with the need for the expensive, cumbersome, and high precision scanning head positioning apparatus currently required. Such a hand-held scanner must also provide the accuracy and precision associated with currently available optical mensuration systems, that is, it must be able to accurately measure and precisely locate the surface details of the object in relation to the predetermined reference frame.

Accordingly, it is an object of the present invention to provide an improved, non-contact, three-dimensional optical mensuration system capable of accurately sensing the surface shapes of three-dimensional objects without the numerous drawbacks associated with the prior art systems.

It is another object of this invention to provide an optical mensuration system that is inexpensive, portable, and easy to use.

It is a further object of this invention to provide a three-dimensional optical mensuration system which can quickly scan the surface of the object without the need for expensive, complicated, and high precision mechanical positioning apparatus to position either the scanning head or the object being scanned.

A still further object of this invention is to provide a portable, hand-held, and hand-maneuverable scanner for the three-dimensional, non-contact shape-scanning and/or mensuration of three-dimensional objects.

Additional objects, advantages, and novel features of this invention shall be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following or may be learned by the practice of the invention. The objects and the advantages of the invention may be realized and attained by means of the instrumentalities and in combinations particularly pointed out in the appended claims.

To achieve the foregoing and other objects and in accordance with the purposes of the present invention, as embodied and broadly described herein, the apparatus for three-dimensional, non-contact shape sensing of this invention may comprise a hand held scanning head with a light source for projecting a scanning light beam over the surface of the object being scanned. Two spot detectors mounted on the hand-held scanning head are operative to detect the position of the illuminated spot on the surface of the object in relation to the scanning head. Three pilot light detectors, the positions of which are known with respect to a predetermined coordinate system, detect the positions of the three pilot light emitters positioned in spaced-apart relation on the scanning head as pilot light emitters are strobed in sequence. A coordinate computer connected to the scanning head and to the pilot light detectors receives data from the spot detectors and calculates the position of the illuminated spot with respect to the scanning head. The coordinate computer then calculates the various positions and orientations of the scanning head in relation to the predetermined coordinate system on a real time basis from the data received from the pilot light detectors. Finally, the coordinate computer calculates the position of the illuminated spot in relation to the predetermined coordinate system by correlating the position of the illuminated spot in relation to the scanning head with the position of the scanning head in relation to the predetermined coordinate system.

The method of this invention includes the steps of sweeping a scanning beam projected from the hand held scanning head over the surface of the object being scanned to illuminate a spot on the surface of the object, detecting the position of the illuminated spot with respect to the scanning head, detecting the position of the scanning head in relation to a predetermined coordinate system, and computing the position of the illuminated spot in relation to the predetermined coordinate system by correlating the position of the illuminated spot in relation to the scanning head with the position of the scanning head in relation to the predetermined coordinate system.

The accompanying drawings, which are incorporated herein and form a part of the specification illustrate preferred embodiments of the present invention, and together with the description, serve to explain the principles of the invention. In the drawings:

FIG. 1 is a block diagram of the optical mensuration apparatus of the present invention showing the major components;

FIG. 2 is a perspective view of the hand held scanning head of the present invention, showing how it can be positioned to direct the scanning beam onto the surface of the object being scanned;

FIG. 3 is a plan view of the scanning head of the present invention with the top surface broken away to more clearly show the arrangement of the optical projecting apparatus and the spot detectors;

FIG. 4 is a schematic perspective representation of one of the one-dimensional photodetectors of the present invention;

FIG. 5 is a schematic block diagram of the optical mensuration apparatus of the present invention showing in detail the functions and operations of the control unit and coordinate computer; and

FIG. 6 is a graph of signal strength vs. location on the detector surface for a typical light detector used by the optical mensuration apparatus of the present invention.

The optical mensuration apparatus 10 of the present invention is shown schematically in FIG. 1 and comprises a hand-held or moveable scanning head 12 housing light beam projecting apparatus 14 (not shown in FIG. 1, but shown in FIG. 3), two one-dimensional spot sensors or detectors 16, 18, and three pilot light emitters 20, 22, and 24. Three remotely located, one-dimensional pilot light sensors 26, 28, and 30 are mounted in fixed, spaced-apart relation to each other and are located at known positions with respect to a predetermined reference coordinate system or frame 80. These three pilot sensors 26, 28, and 30 sense the light projected by the individual pilot light emitters 20, 22, and 24 and generate electrical output signals from which are derived the location of the scanning head 12 with respect to the fixed coordinate system 80. A control unit 32 connected to the moveable scanning head 12 via data line 46 and connected to the remotely located sensors 26, 28, and 30 via data lines 48, 50, and 52, respectively, synchronizes the time multiplexing of the three pilot emitters 20, 22, and 24, controls the operation of the beam projecting apparatus 14, and receives data from the two spot sensors 16, 18 on scanning head 12, as will be completely described below. A coordinate computer 34, connected to control unit 32 by data line 54 calculates the three-dimensional spatial coordinates of the illuminated spot 36 in relation to the predetermined coordinate reference frame 80, which position information can then be used by an application system 82.

In operation, the light beam projecting apparatus 14 housed in the hand held scanner head 12 directs a narrow beam of light or scanning beam 42 onto the surface 40 of object 38 to illuminate a small portion or spot 36 on the surface 40. Reflected light 43 from illuminated spot 36 is detected by the two one-dimensional spot sensors or detectors 16, 18 mounted on scanner head 12. These sensors 16, 18 sense the location of the illuminated spot 36 with respect to the position of the moveable scanner 12 by measuring the relative angular parallax of the reflected light 43 from illuminates spot 36. Next, the spatial position and orientation of the moveable scanner head 12 at that same instant are determined by measuring the locations of the three time multiplexed pilot light emitters 20, 22, and 24 relative to the known positions of the pilot light sensors 26, 28, and 30. Finally, the parallax data from each of the sensors 16, 18, 26, 28, and 30 are ultimately fed to the coordinate computer 34, which determines the position of the illuminated spot 36 with respect to the predetermined reference frame by correlating the position of the illuminated spot 36 in relation to the scanner head 12 with the position of the scanner 12 in relation to the fixed pilot light sensors 26, 28, and 30, which are positioned in relation to the predetermined reference frame 80 at precisely predetermined locations at conveniently spaced distances from each other and from the object 38 and the hand-held scanner 12. If the computer can make these location or position calculations very fast, the operation can be performed over and over again in sequence as the scanner head 12 moves in relation to the object, thus resulting in effectively real time mensuration of the object as the scanner head 12 moves.

By using this two-stage measurement system, i.e., first measuring the location of the illuminated spot 36 in relation to the scanning head 12 at a particular instant in time, and then determining the position of the scanning head 12 in relation to the predetermined reference frame at that same instant in time, the optical mensuration apparatus 10 of the present invention dispenses with the need for high precision head positioning apparatus and the complex and expensive mechanical structure typically associated therewith. Further, the hand-held scanner, 12 is easily manipulated by the operator to direct the scanning beam 42 over complex, interior, or blind surface details, which would otherwise be difficult to scan, thus speeding the scanning operation.

The details of the optical mensuration apparatus 10 of the present invention are best understood by referring to FIGS. 2, 3, and 4 simultaneously. Essentially, the hand-held scanner head 12 houses the light beam projecting apparatus 14 (FIG. 3), the two one-dimensional spot sensors or detectors 16, 18, and the three pilot light emitters 20, 22, and 24. A handle 44 allows the scanner head 12 to be easily manipulated by the operator to guide the scanning beam 42 over the various shapes and hidden contours of the surface 40 of object 38.

In the preferred embodiment, the light beam projecting apparatus comprises a helium-neon (He-Ne) laser 56 to generate collimated scanning beam 42. Of course, other devices could be used to produce the spot-like scanning beam as would be readily apparent to persons having ordinary skill in the art. For example, laser 56 could be replaced by a light emitting diode (LED) and associated collimating lens. Other sources and lens combinations are possible so long as the apparatus is capable of projecting a small, well defined beam of light on the surface of the object. A planar mirror 58, which could be optionally pivotally mounted as shown in FIG. 3, directs beam 42 to a rotating many-faceted mirror 60, which directs, or scans beam 42 over the surface 40 in a single plane relative to the scanner 12 (i.e., the plane of the paper in FIG. 3). Of course, the number of sides of the rotating, many-faceted mirror 60 determines the angle through which scanning beam 42 sweeps. For example, the pentagonal mirror shown in FIG. 3 will sweep the beam through a 144-degree angle. More sides will sweep the beam through smaller angles. Moreover, other scanning paths are possible by using irregularly shaped mirrors or multiple rotating mirrors, and the present invention should not be regarded as limited by the particular scanning apparatus shown and described herein.

While the rotating mirror 60 can be rotated in either direction with equal effectiveness, the rotating mirror 60 in the preferred embodiment 10 is rotated in the direction indicated by arrow 62 by a simple, unsynchronized motor (not shown). As mentioned above, planar mirror 58 may be optionally pivotally mounted such that it can be swung out of the beam path to position 58' (shown in broken lines in FIG. 3) to inhibit the scanning action of the beam 42. With the mirror at position 58' the beam 42 will exit straight out aperture 64 in scanner 12 which can then be used as a point-type scanner or as a noncontact pointer for identifying some single point of interest on the surface 40 of object 38.

The details of the one-dimensional spot detectors 16, 18 are best understood by referring to FIG. 4. Actually, all of the one-dimensional sensors 16, 18, 26, 28, and 30 used in the preferred embodiment 10 of the present invention are identical to the one-dimensional spot detector 16 in every respect. Therefore, for the purpose of giving a detailed description of this embodiment, only the sensor 16 is shown and described in detail since the remaining sensors 18, 26, 28, and 30 have identical features.

Referring now to FIG. 4, the one-dimensional sensor 16 comprises a cylindrical lens 66 that has a longitudinal axis 74 which is orthogonal to the optical axis 76 of the sensor 16. A linear photodetector 68, such as a charge coupled device (CCD) with several thousand elements, or a similar device capable of linear light detection with an elongated aperture 78 is positioned in such a manner that optical axis 76 passes through aperture 78 and such that the long axis of aperture 78 is orthogonal to the plane containing the longitudinal axis 74 of lens 66. The incident light beam 43 reflected from illuminated spot 36 is then focused by the cylindrical lens 66 into a real image line 72 on the surface 70 of linear photodetector 68, which is a characteristic of this type of lens.

The CCD detector 68 then generates a signal, such as the one shown in FIG. 6, that is related to the position of real image line 72 on the surface 70 of photodetector 68, thus characterizing the location of the image itself. That is, those elements of the detector 68 illuminated by the real image line 72 will generate a strong signal, while those not illuminated will generate a weak signal. Thus, a graph of signal strength vs. location on the surface of the CCD will resemble the signal peak curve 100 shown in FIG. 6. Note that the "zero" signal level 102 is never quite zero due to the effects of background light and other imperfections in the sensor. In any event, since the image of illuminated spot 36 is focused into line 72, only the horizontal displacement of spot 36 from optical axis 76 is measured by detector 68, hence the designation "one-dimensional detector."

Thus, a single one-dimensional detector 16 can only locate the plane on which spot 36 particular beam lies, but detector 16 cannot, by itself, determine the unique location or position in space on which point 36 is located. To precisely locate the location in space of point 36 would require three such detectors postitioned in spaced-apart relation to each other, since the intersection of three planes defines a point. However, if the plane containing the aperture 78 of detector 16 is in the same plane as the scanning beam 42, only two detectors are required to uniquely locate the position of spot 36. Therefore, in the preferred embodiment 10 of the present invention, the apertures 78 of the respective photodetectors 16, 18, lie in the same plane as the scanning beam 42, thereby allowing the exact point in space of illuminated spot 36 to be determined with only two detectors 16, 18.

The three pilot light emitters 20, 22, and 24 (FIGS. 1-3) can be high intensity light emitting diodes (LEDs), which are preferably time multiplexed or strobed by control unit 32 in a predetermined manner such that only one pilot light LED is "on" or emitting light at any one time. The light emitted from any one of these emitters 20, 22, and 24 is detected by each of the three pilot light detectors 26, 28, and 30, which then determine the position of that particular emitter in relation to the known positions of the detectors 26, 28, and 30 at the instant in time that it is strobed or illuminated. To locate the position of a particular illuminated one of emitters 20, 22, 24, the pilot light detectors 26, 28, and 30 are mounted so that their optical axes are not collinear. In the preferred embodiment, two pilot light detectors, such as detectors 26, 30 in FIG. 1, are situated such that their respective axes 74 (FIG. 4) are in parallel spaced-apart relation, with the third detector 28 situated between the first two, but with its axis 74 perpendicular to the first two. As described above, each of the detectors 26, 28, and 30 then determine a unique plane in which the given pilot emitter lies, the intersection of which defines the exact location of that illuminated emitter.

While this process of detecting the position of a given illuminated pilot emitter 20, 22, 24 can locate the exact position of the illuminated emitter, it cannot determine the particular orientation of the entire scanner head 12 in three-dimensions. To do so requires the detection of the locations of at least three spaced-apart emitters whose orientations with respect to one another are known. Therefore, the optical mensuration system 10 of the present invention determines the orientation of the scanning head 12 in three-dimensional space by using the three (3) pilot emitters 20, 22, and 24, whose relative positions on the scanning head 12 are fixed and known. Consequently, when each of the emitters 20, 22, and 24 are rapidly turned on in sequence, the sensors 26, 28, and 30 can detect the exact position of each emitter in turn, thus determine the exact location and orientation of the scanning head 12. Since only one of the pilot light emitters 20, 22, 24 is on at any one time, the detectors 26, 28, 30 locate the position of that particular illuminated pilot light only. If the strobe rate, that is, the frequency at which the emitters 20, 22, 24 are turned on and off in sequence, is fast enough, the detectors 26, 28, and 30 can, for all practical purposes, determine the position and orientation of the scanning head 12 at any instant in time.

Note that the detectors 26, 28, 30, need only distinguish which of the pilot light emitters 20, 22, 24 is "on" or illuminated at any one time. In the preferred embodiment 10 of the present invention, this function is accomplished by strobing or illuminating each of the emitters 20, 22, 24 in sequence. However, other methods could be used to allow the detectors 26, 28, 30 to distinguish the respective pilot light emitters 20, 22, 24 from one another. For example, different colors of light could be used in conjunction with detectors capable of distinguishing those particular colors or wavelengths of light. Alternatively, the respective pilot light emitters 20, 22, 24 could be modulated with a unique "tone" for each emitter. The control unit 32 or coordinate computer 34 could then be programmed to demodulate the tone, thus determine to which particular emitter 20, 22, or 24 the position signal belongs. Numerous other methods of distinguishing the pilot light emitters 20, 22, and 24 are possible and would be readily apparent to persons having ordinary skill in the art. Therefore, the present invention should not be regarded as limited to the particular strobing method shown and described herein.

The details of the structure and operation of the control unit 32 are best seen in FIG. 5. Specifically, control unit 32 supplies power to the light beam projecting apparatus or source 14, the beam spot sensors 16, 18, the pilot light emitters or sources 20, 22, and 24, and the pilot light sensors 26, 28, and 30. The control and synchronization unit 84 and light source sequencer 86 time multiplexes or strobes the beam projecting apparatus 14 and the pilot lights 20, 22, and 24 individually, as described above, so that the position and orientation of the scanning head 12 can be determined from the signals received from pilot light sensors 26, 28 and 30. The angular data signals received from the pilot light sensors 26, 28, and 30 and from the spot sensors 16, 18, are converted by analog to digital converter 88. Actually, five analog to digital converters are used, as shown in FIG. 5, but only one is labeled and described herein for brevity, since the other four analog to digital converters are identical and are used to convert the signals from sensors 28 and 30 and 16 and 18, respectively.

The control and synchronization unit 84 also controls five switches, of which switch 90 is typical, which store all digital data received from the sensors 26, 28, and 30 and 16 and 18 when the pilot light emitters and scanning beam 42 are "off," and stores these data in background memory 92. Then, when the pilot light sources and scanning beam are illuminated in sequence by light source sequencer 86, the control and synchronization unit 84 changes the state of switch 90, which then redirects the data from the five sensors to the subtraction unit 94. Subtraction unit 94 substracts the "background" data from the illuminated data, thus resulting in a signal relatively free from background noise signal 102 (FIG. 6), since it has been subtracted from the signal.

Referring now to FIGS. 4 and 6 in conjunction with FIG. 5, the first-last over-threshold unit 96 computes the location of the real image line 72 on the CCD sensor 68 (FIG. 4) by measuring the locations of the edges 104, 106 of the signal blip 100 (FIG. 6) generated by the CCD sensor based on a predetermined threshold signal level. The first-last over-threshold unit 96 then averages the distance between the two edges to find the center of the signal peak, which is often dipped, as shown in FIG. 6. This particular method of determining the center of the signal peak is well known in the art and will not be described in further detail. Moreover, numerous other methods of determining the location of the signal peak are known in the art, and would be obvious to those having ordinary skill in the art. The particular method used would depend on the signal characteristics of the particular light sensor used, as well as the characteristics of the lens system used to focus the light onto the surface of the detector, as well as other parameters. Those practicing this invention with the various alternates described herein would have no trouble selecting a signal detection algorithm best suited to the particular characteristics of the sensors.

Finally, control unit 32 (FIG. 5) transmits the position data to the coordinate computer 34. That is, when the coordinate computer 34 is ready to compute the current location of the illuminated spot 36 on the object, the latest angular data from all sensors are provided for analyzation. If the spot sensors 16, 18, or the pilot light sensors 26, 28, and 30, generate data faster than the control unit 32 can process them, the angular data are simply discarded.

The details of the coordinate computer 34 are also best seen in FIG. 5. Essentially, the coordinate computer 34 calculates one-dimensional positions for each light source based on the location of the signal peak from each respective sensor. These one-dimensional positions are then used to calculate the three-dimensional spatial coordinates for the illuminated spot 36 and for the scanning head 12 in relation to the predetermined coordinate system 80, by coordinate transformation methods which are well-known in the art. The output from the coordinate computer 34 can be in any form desired by the operator or required by the application system 80, such as XYZ coordinate triples based upon some predetermined stationary rectangular coordinate system.

The operation of the optical mensuration apparatus of the present invention is as follows. Upon illumination of a spot 36 on the surface 40 of object 38, the two spot sensors 16, 18 inside the scanner head 12 sense the angular position of the illuminated spot 36 at a given instant in time. The signals from these spot sensors 16, 18, are directed to the control unit 32 via data line 46. Next, the pilot light detectors 26, 28, and 30 are used to sense the individual positions of the three pilot light emitters 20, 22, 24 in sequence as described above. That is, each pilot light detector 26, 28, 30, measures the angle of rays from each of three pilot light emitters 20, 22, 24, mounted on the scanner 12. The angular data from each of these sensors 26, 28, and 30 are also directed to control unit 32 via data lines 48, 50, and 52.

As described above, the control unit 32 converts the angular data from each of the sensors 16, 18, 26, 28, and 30, which is in analog form, to digital data and tags these data with information identifying their respective sources. These converted digital data are then processed by removing the background noise and by using known signal detection methods to determine the center of the signal peak, thus the location of the image line 72 on the detector 68. These position locations of the centers of the respective signal peaks from each detector 16, 18, 26, 28, and 30 are then directed to coordinate computer 34 via data line 54, which then computes the current location of the illuminated spot 36 with respect to the predetermined coordinate system 80. Sequential calculations and beam spot position determination can be made as fast as the computer can do so, thus many such points on the surface of the object can be determined as they are scanned almost on a real time basis. These position data can be stored in computer memory, recalled, and correlated together to produce an image of the object in precise reproduction detail, or various points or other features on the object can be mensurated or used in any manner desired.

This completes the detailed description of the method and apparatus of the optical mensuration apparatus 10 of the present invention. While some of the obvious and numerous modifications and equivalents have been described herein, still other modifications and changes will readily occur to those skilled in the art. For instance, the preferred embodiment uses visible light since human operators can readily observe if the light sources are operative or whether they are causing troublesome reflections. Clearly, other wavelengths of electromagnetic radiation could be used without departing from the spirit and scope of this invention. Further, it would be possible to include circuitry in the detectors which would subtract out the ambient light, thus improve the detection efficiency of the invention. Other modifications to the detector optics and lenses are possible which would alter the image characteristics on the detectors. For example, cylindrical lenses could be used which have been longitudinally curved along an arc with a radius equal to the focal length of the lens. Similarly, the surfaces of the photodetectors could also be curved, thus allowing the images of distant light sources to remain in sharp focus regardless of their positions. Various measurements of the detector outputs are also possible. For example, the angle of peak intensity, the intensity-weighted average, or the average of the minimum and maximum angles where the intensity is over some predetermined threshold value could be used. Finally, numerous enhancements of the digital data are possible by programming the coordinate computer to make the appropriate enhancements, as would be obvious to those persons having ordinary skill in the art.

The foregoing is considered illustrative only of the principles of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalents may be resorted to as falling within the scope of the invention as defined by the claims which follow.

Schulz, Waldean A.

Patent Priority Assignee Title
10070801, Jul 10 2008 Covidien LP Integrated multi-functional endoscopic tool
10080617, Jun 27 2011 Board of Regents of the University of Nebraska On-board tool tracking system and methods of computer assisted surgery
10096126, Jun 03 2008 Covidien LP Feature-based registration method
10105149, Mar 15 2013 Board of Regents of the University of Nebraska On-board tool tracking system and methods of computer assisted surgery
10154798, Apr 08 2009 Covidien LP Locatable catheter
10209059, Apr 21 2010 Faro Technologies, Inc. Method and apparatus for following an operator and locking onto a retroreflector with a laser tracker
10219811, Jun 27 2011 Board of Regents of the University of Nebraska On-board tool tracking system and methods of computer assisted surgery
10285623, Jun 06 2008 Covidien LP Hybrid registration method
10302413, Apr 15 2011 Faro Technologies, Inc. Six degree-of-freedom laser tracker that cooperates with a remote sensor
10321803, May 01 2005 Covidien LP System and method for image-based alignment of an endoscope
10383509, Sep 15 2003 Covidien LP System of accessories for use with bronchoscopes
10390686, Sep 27 2007 Covidien LP Bronchoscope adapter and method
10418705, Oct 28 2016 Covidien LP Electromagnetic navigation antenna assembly and electromagnetic navigation system including the same
10426555, Jun 03 2015 Covidien LP Medical instrument with sensor for use in a system and method for electromagnetic navigation
10446931, Oct 28 2016 Covidien LP Electromagnetic navigation antenna assembly and electromagnetic navigation system including the same
10478092, Jun 06 2008 Covidien LP Hybrid registration method
10478254, May 16 2016 Covidien LP System and method to access lung tissue
10480929, Apr 21 2010 Faro Technologies, Inc. Method and apparatus for following an operator and locking onto a retroreflector with a laser tracker
10517505, Oct 28 2016 Covidien LP Systems, methods, and computer-readable media for optimizing an electromagnetic navigation system
10582834, Jun 15 2010 RIOJA LLC; Covidien LP Locatable expandable working channel and method
10597178, Jan 18 2006 Medtronic Navigation, Inc. Method and apparatus for providing a container to a sterile environment
10615500, Oct 28 2016 Covidien LP System and method for designing electromagnetic navigation antenna assemblies
10638952, Oct 28 2016 Covidien LP Methods, systems, and computer-readable media for calibrating an electromagnetic navigation system
10674936, Jun 06 2008 Covidien LP Hybrid registration method
10722311, Oct 28 2016 Covidien LP System and method for identifying a location and/or an orientation of an electromagnetic sensor based on a map
10743748, Apr 17 2002 Covidien LP Endoscope structures and techniques for navigating to a target in branched structure
10751126, Oct 28 2016 Covidien LP System and method for generating a map for electromagnetic navigation
10792106, Oct 28 2016 Covidien LP System for calibrating an electromagnetic navigation system
10898153, Mar 01 2000 Medtronic Navigation, Inc. Multiple cannula image guided tool for image guided procedures
10912487, Jul 10 2008 Covidien LP Integrated multi-function endoscopic tool
10952593, Jun 10 2014 Covidien LP Bronchoscope adapter
10980400, Sep 27 2007 Covidien LP Bronchoscope adapter and method
11006914, Oct 28 2015 Medtronic Navigation, Inc. Apparatus and method for maintaining image quality while minimizing x-ray dosage of a patient
11074702, Jun 03 2008 Covidien LP Feature-based registration method
11116574, Jun 16 2006 Board of Regents of the University of Nebraska Method and apparatus for computer aided surgery
11160617, May 16 2016 Covidien LP System and method to access lung tissue
11219489, Oct 31 2017 Covidien LP Devices and systems for providing sensors in parallel with medical tools
11234611, Jul 10 2008 Covidien LP Integrated multi-functional endoscopic tool
11241164, Jul 10 2008 Covidien LP Integrated multi-functional endoscopic tool
11331150, Oct 28 1999 Medtronic Navigation, Inc. Method and apparatus for surgical navigation
11672604, Oct 28 2016 Covidien LP System and method for generating a map for electromagnetic navigation
11684491, Jan 30 2003 Medtronic Navigation, Inc. Method and apparatus for post-operative tuning of a spinal implant
11707363, Jan 30 2003 Medtronic Navigation, Inc. Method and apparatus for post-operative tuning of a spinal implant
11759264, Oct 28 2016 Covidien LP System and method for identifying a location and/or an orientation of an electromagnetic sensor based on a map
11783498, Jun 03 2008 Covidien LP Feature-based registration method
11786314, Oct 28 2016 Covidien LP System for calibrating an electromagnetic navigation system
11786317, May 16 2016 Covidien LP System and method to access lung tissue
11801024, Oct 28 2015 Medtronic Navigation, Inc. Apparatus and method for maintaining image quality while minimizing x-ray dosage of a patient
11857265, Jun 16 2006 of Nebraska Method and apparatus for computer aided surgery
11911117, Jun 27 2011 of Nebraska On-board tool tracking system and methods of computer assisted surgery
5969822, Sep 28 1994 ARANZ Healthcare Limited Arbitrary-geometry laser surface scanner
6146390, Apr 21 1992 Sofamor Danek Holdings, Inc. Apparatus and method for photogrammetric surgical localization
6165181, Apr 21 1992 SOFAMOR DANEK HOLDINGS, INC Apparatus and method for photogrammetric surgical localization
6222582, Jul 24 1997 MURATA MANUFACTURING CO , LTD Image capture system
6226548, Sep 24 1997 Medtronic Navigation, Inc Percutaneous registration apparatus and method for use in computer-assisted surgical navigation
6235038, Oct 28 1999 Medtronic Surgical Navigation Technologies System for translation of electromagnetic and optical localization systems
6271918, Feb 04 1999 National Research Council of Canada Virtual multiple aperture 3-D range sensor
6296613, Aug 22 1997 AO Technology AG 3D ultrasound recording device
6324296, Dec 04 1997 PHASESPACE, INC Distributed-processing motion tracking system for tracking individually modulated light points
6363940, May 14 1998 Varian Medical Systems, Inc System and method for bracketing and removing tissue
6370224, Jun 29 1998 Sofamor Danek Group, Inc. System and methods for the reduction and elimination of image artifacts in the calibration of x-ray imagers
6374198, Jul 11 1996 HEXAGON METROLOGY S P A Method for the creation of tridimensional numerical models
6379302, Oct 28 1999 Medtronic Navigation, Inc Navigation information overlay onto ultrasound imagery
6381485, Oct 28 1999 Medtronic Navigation, Inc Registration of human anatomy integrated for electromagnetic localization
6402762, Oct 28 1999 Surgical Navigation Technologies, Inc. System for translation of electromagnetic and optical localization systems
6413084, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT Method and system of scanning
6474341, Oct 28 1999 Medtronic Navigation, Inc Surgical communication and power system
6491702, Apr 21 1992 Sofamor Danek Holdings, Inc. Apparatus and method for photogrammetric surgical localization
6493095, Apr 13 1999 CREAFORM INC Optional 3D digitizer, system and method for digitizing an object
6497134, Mar 15 2000 IMAGE GUIDED TECHNOLOGIES, INC Calibration of an instrument
6499488, Oct 28 1999 SURGICAL NAVIGATION TECHNOLOGIES, INC Surgical sensor
6532299, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT System and method for mapping a surface
6564086, May 03 2000 ROCKY MOUNTAIN BIOSYSTEMS, INC Prosthesis and method of making
6585651, Apr 20 1999 AO Technology AG Method and device for percutaneous determination of points associated with the surface of an organ
6611141, Dec 23 1998 Stryker Corporation Hybrid 3-D probe tracked by multiple sensors
6669635, Oct 28 1999 Surgical Navigation Technologies, Inc. Navigation information overlay onto ultrasound imagery
6694168, Jun 22 1998 AO Technology AG Fiducial matching using fiducial implants
6724947, Jul 14 2000 GLOBALFOUNDRIES U S INC Method and system for measuring characteristics of curved features
6725080, Mar 01 2000 SURGICAL NAVIGATION TECHNOLOGIES, INC Multiple cannula image guided tool for image guided procedures
6725082, Mar 17 1999 AO Technology AG System and method for ligament graft placement
6728423, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT System and method for mapping a surface
6732030, Aug 18 2001 Snap-On Incorporated Three-dimensional mapping systems for automotive vehicles and other articles
6744914, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT Method and system for generating a three-dimensional object
6744932, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT System and method for mapping a surface
6771809, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT Method and system for registering data
6801637, Aug 10 1999 JOLLY SEVEN, SERIES 70 OF ALLIED SECURITY TRUST I Optical body tracker
6812842, Dec 20 2001 Varian Medical Systems, Inc System for excitation of a leadless miniature marker
6822570, Dec 20 2001 Varian Medical Systems, Inc System for spatially adjustable excitation of leadless miniature marker
6838990, Dec 20 2001 Varian Medical Systems, Inc System for excitation leadless miniature marker
6888640, Feb 04 2000 SPINA, MARI J Body spatial dimension mapper
6889833, Dec 30 2002 Varian Medical Systems, Inc Packaged systems for implanting markers in a patient and methods for manufacturing and using such systems
6892090, Aug 19 2002 Surgical Navigation Technologies, Inc. Method and apparatus for virtual endoscopy
6911972, Apr 04 2001 MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD User interface device
6918919, May 14 1998 Varian Medical Systems, Inc System and method for bracketing and removing tissue
6920347, Apr 07 2000 Medtronic Navigation, Inc Trajectory storage apparatus and method for surgical navigation systems
6947786, Feb 28 2002 Medtronic Navigation, Inc Method and apparatus for perspective inversion
6968224, Oct 28 1999 Surgical Navigation Technologies, Inc. Method of detecting organ matter shift in a patient
6990368, Apr 04 2002 Medtronic Navigation, Inc Method and apparatus for virtual digital subtraction angiography
7007699, Oct 28 1999 SURGICAL NAVIGATION TECHNOLOGIES, INC Surgical sensor
7027642, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT Methods for registration of three-dimensional frames to create three-dimensional virtual models of objects
7065462, Jul 24 1998 Merilab, Inc.; MERILAB, INC Vehicle wheel alignment by rotating vision sensor
7068825, Mar 08 1999 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT Scanning system and calibration method for capturing precise three-dimensional information of objects
7068836, Apr 28 2000 SOLAR CAPITAL LTD , AS SUCCESSOR AGENT System and method for mapping a surface
7085400, Jun 14 2000 Surgical Navigation Technologies, Inc. System and method for image based sensor calibration
7120524, Dec 04 2003 Matrix Electronic Measuring Properties, LLC System for measuring points on a vehicle during damage repair
7130676, Aug 20 1998 Sofamor Danek Holdings, Inc. Fluoroscopic image guided orthopaedic surgery system with intraoperative registration
7135978, Sep 14 2001 Varian Medical Systems, Inc Miniature resonating marker assembly
7142312, Dec 31 2002 D4D Technologies, LLC Laser digitizer system for dental applications
7152608, Oct 28 1999 Medtronic Navigation, Inc Surgical communication and power system
7154395, Jul 01 2004 Mitsubishi Electric Research Laboratories, Inc Interactive wireless tag location and identification system
7174202, Jan 27 1993 British Telecommunications Medical navigation apparatus
7176798, Dec 20 2001 Varian Medical Systems, Inc System for spatially adjustable excitation of leadless miniature marker
7184150, Mar 24 2003 D4D Technologies, LLC Laser digitizer system for dental applications
7217276, Apr 20 1999 Surgical Navigational Technologies, Inc. Instrument guidance method and system for image guided surgery
7256899, Oct 04 2006 PATENT ARMORY INC Wireless methods and systems for three-dimensional non-contact shape sensing
7277594, May 03 1999 AO Technology AG System and method for preparing an image corrected for the presence of a gravity induced distortion
7289839, Dec 30 2002 Varian Medical Systems, Inc Implantable marker with a leadless signal transmitter compatible for use in magnetic resonance devices
7313430, Aug 28 2003 Medtronic Navigation, Inc. Method and apparatus for performing stereotactic surgery
7336375, Oct 04 2006 PATENT ARMORY INC Wireless methods and systems for three-dimensional non-contact shape sensing
7342668, Sep 17 2003 D4D Technologies, LLC High speed multiple line three-dimensional digitalization
7355721, May 08 2003 D4D Technologies, LLC Optical coherence tomography imaging
7366562, Oct 17 2003 SURGICAL NAVIGATION TECHNOLOGIES, INC Method and apparatus for surgical navigation
7376492, Dec 04 2003 Matrix Electronic Measuring Properties, LLC System for measuring points on a vehicle during damage repair
7407054, Dec 30 2002 Varian Medical Systems, Inc Packaged systems for implanting markers in a patient and methods for manufacturing and using such systems
7447558, Sep 18 2004 The Ohio Willow Wood Company Apparatus for determining a three dimensional shape of an object
7535363, Sep 14 2001 Varian Medical Systems, Inc Miniature resonating marker assembly
7542791, Jan 30 2003 Medtronic Navigation, Inc. Method and apparatus for preplanning a surgical procedure
7567834, May 03 2004 Medtronic Navigation, Inc Method and apparatus for implantation between two vertebral bodies
7570791, Apr 25 2003 Medtronic Navigation, Inc Method and apparatus for performing 2D to 3D registration
7577474, Apr 05 2000 Brainlab AG Referencing or registering a patient or a patient body part in a medical navigation system by means of irradiation of light points
7599730, Nov 19 2002 Medtronic Navigation, Inc Navigation system for cardiac therapies
7606613, Mar 23 1999 Medtronic Navigation, Inc Navigational guidance via computer-assisted fluoroscopic imaging
7630753, Feb 28 2002 Medtronic Navigation, Inc. Method and apparatus for perspective inversion
7636595, Oct 28 2004 Medtronic Navigation, Inc. Method and apparatus for calibrating non-linear instruments
7657300, Oct 28 1999 Medtronic Navigation, Inc. Registration of human anatomy integrated for electromagnetic localization
7657301, Jun 08 2001 Varian Medical Systems, Inc Guided radiation therapy system
7657302, Jun 08 2001 Varian Medical Systems, Inc Guided radiation therapy system
7657303, Jun 08 2001 Varian Medical Systems, Inc Guided radiation therapy system
7660623, Jan 30 2003 Medtronic Navigation, Inc. Six degree of freedom alignment display for medical procedures
7696876, Dec 20 2001 Varian Medical Systems, Inc System for spatially adjustable excitation of leadless miniature marker
7697972, Nov 19 2002 Medtronic Navigation, Inc Navigation system for cardiac therapies
7725162, Jan 27 2000 Stryker Corporation Surgery system
7751865, Oct 17 2003 Medtronic Navigation, Inc. Method and apparatus for surgical navigation
7763035, Dec 12 1997 Medtronic Navigation, Inc. Image guided spinal surgery guide, system and method for use thereof
7778687, Dec 30 2002 Varian Medical Systems, Inc Implantable marker with a leadless signal transmitter compatible for use in magnetic resonance devices
7797032, Oct 28 1999 SURGICAL NAVIGATION TECHNOLOGIES, INC Method and system for navigating a catheter probe in the presence of field-influencing objects
7818044, Oct 17 2003 Medtronic Navigation, Inc. Method and apparatus for surgical navigation
7831082, Jun 14 2000 Medtronic Navigation, Inc. System and method for image based sensor calibration
7835778, Oct 16 2003 Medtronic Navigation, Inc Method and apparatus for surgical navigation of a multiple piece construct for implantation
7835784, Sep 21 2005 Medtronic Navigation, Inc Method and apparatus for positioning a reference frame
7840253, Oct 17 2003 Medtronic Navigation, Inc Method and apparatus for surgical navigation
7853305, Apr 07 2000 Medtronic Navigation, Inc. Trajectory storage apparatus and method for surgical navigation systems
7869861, Oct 25 2002 Stryker Corporation Flexible tracking article and method of using the same
7881770, Mar 01 2000 Medtronic Navigation, Inc. Multiple cannula image guided tool for image guided procedures
7925328, Aug 28 2003 Medtronic Navigation, Inc. Method and apparatus for performing stereotactic surgery
7953471, May 03 2004 Medtronic Navigation, Inc. Method and apparatus for implantation between two vertebral bodies
7971341, Oct 17 2003 Medtronic Navigation, Inc. Method of forming an electromagnetic sensing coil in a medical instrument for a surgical navigation system
7974677, Jan 30 2003 Medtronic Navigation, Inc. Method and apparatus for preplanning a surgical procedure
7996064, Mar 23 1999 Medtronic Navigation, Inc. System and method for placing and determining an appropriately sized surgical implant
7998062, Mar 29 2004 Covidien LP Endoscope structures and techniques for navigating to a target in branched structure
8007448, Oct 08 2004 STRYKER EUROPEAN HOLDINGS III, LLC System and method for performing arthroplasty of a joint and tracking a plumb line plane
8011508, Dec 30 2002 Varian Medical Systems, Inc Packaged systems for implanting markers in a patient and methods for manufacturing and using such systems
8046052, Nov 19 2002 Medtronic Navigation, Inc. Navigation system for cardiac therapies
8057407, Oct 28 1999 Medtronic Navigation, Inc. Surgical sensor
8060185, Nov 19 2002 Medtronic Navigation, Inc. Navigation system for cardiac therapies
8074662, Oct 28 1999 Medtronic Navigation, Inc. Surgical communication and power system
8105339, Dec 12 1997 Sofamor Danek Holdings, Inc. Image guided spinal surgery guide system and method for use thereof
8112292, Apr 21 2006 Medtronic Navigation, Inc Method and apparatus for optimizing a therapy
8165658, Sep 26 2008 Medtronic, Inc Method and apparatus for positioning a guide relative to a base
8175681, Dec 16 2008 Medtronic Navigation Inc. Combination of electromagnetic and electropotential localization
8196589, Dec 24 2003 Varian Medical Systems, Inc Implantable marker with wireless signal transmitter
8200314, Jan 27 1993 British Telecommunications public limited company Surgical navigation
8239001, Oct 17 2003 Medtronic Navigation, Inc Method and apparatus for surgical navigation
8244330, Jul 23 2004 Varian Medical Systems, Inc Integrated radiation therapy systems and methods for treating a target in a patient
8249332, May 22 2008 Matrix Electronic Measuring Properties, LLC Stereoscopic measurement system and method
8271069, Oct 17 2003 Medtronic Navigation, Inc. Method and apparatus for surgical navigation
8280490, Dec 02 2004 Siemens Healthcare GmbH Registration aid for medical images
8290570, Sep 10 2004 STRYKER EUROPEAN HOLDINGS III, LLC System for ad hoc tracking of an object
8290572, Oct 28 1999 Medtronic Navigation, Inc. Method and system for navigating a catheter probe in the presence of field-influencing objects
8297030, Dec 30 2002 Varian Medical Systems, Inc Methods for manufacturing packaged systems for implanting markers in a patient
8320653, Jun 14 2000 Medtronic Navigation, Inc. System and method for image based sensor calibration
8326022, May 22 2008 Matrix Electronic Measuring Properties, LLC Stereoscopic measurement system and method
8340742, Jul 23 2004 Varian Medical Systems, Inc Integrated radiation therapy systems and methods for treating a target in a patient
8345953, May 22 2008 Matrix Electronic Measuring Properties, LLC Stereoscopic measurement system and method
8359730, Oct 17 2003 Medtronic Navigation, Inc. Method of forming an electromagnetic sensing coil in a medical instrument
8382765, Aug 07 2007 STRYKER EUROPEAN HOLDINGS III, LLC Method of and system for planning a surgery
8401616, Nov 19 2002 Medtronic Navigation, Inc. Navigation system for cardiac therapies
8452068, Jun 06 2008 Covidien LP Hybrid registration method
8452375, May 14 1998 Varian Medical Systems, Inc Systems and methods for locating and defining a target location within a human body
8457719, Oct 25 2002 Stryker Corporation Flexible tracking article and method of using the same
8467589, Jun 06 2008 Covidien LP Hybrid registration method
8467851, Sep 21 2005 Medtronic Navigation, Inc. Method and apparatus for positioning a reference frame
8467853, Nov 19 2002 Medtronic Navigation, Inc. Navigation system for cardiac therapies
8473026, Sep 15 1994 STRYKER EUROPEAN HOLDINGS I, LLC System for monitoring a position of a medical instrument with respect to a patient's body
8473032, Jun 03 2008 Covidien LP Feature-based registration method
8494613, Aug 31 2009 Medtronic, Inc. Combination localization system
8494614, Aug 31 2009 Regents of the University of Minnesota; Medtronic, Inc. Combination localization system
8548565, Oct 28 1999 Medtronic Navigation, Inc. Registration of human anatomy integrated for electromagnetic localization
8549732, Oct 17 2003 Medtronic Navigation, Inc. Method of forming an electromagnetic sensing coil in a medical instrument
8611984, Apr 08 2009 Covidien LP Locatable catheter
8617173, Aug 07 2007 STRYKER EUROPEAN HOLDINGS III, LLC System for assessing a fit of a femoral implant
8617174, Aug 07 2007 STRYKER EUROPEAN HOLDINGS III, LLC Method of virtually planning a size and position of a prosthetic implant
8634897, Apr 07 2000 Medtronic Navigation, Inc. Trajectory storage apparatus and method for surgical navigation systems
8641210, Nov 30 2011 Izi Medical Products Retro-reflective marker including colored mounting portion
8644907, Oct 28 1999 Medtronic Navigaton, Inc. Method and apparatus for surgical navigation
8646921, Nov 30 2011 Izi Medical Products Reflective marker being radio-opaque for MRI
8651274, Nov 30 2011 Izi Medical Products Packaging for retro-reflective markers
8660635, Sep 29 2006 Medtronic, Inc Method and apparatus for optimizing a computer assisted surgical procedure
8661573, Feb 29 2012 Izi Medical Products Protective cover for medical device having adhesive mechanism
8662684, Nov 30 2011 Izi Medical Products Radiopaque core
8663088, Sep 15 2003 Covidien LP System of accessories for use with bronchoscopes
8668342, Nov 30 2011 Izi Medical Products Material thickness control over retro-reflective marker
8668343, Nov 30 2011 Izi Medical Products Reflective marker with alignment feature
8668344, Nov 30 2011 Izi Medical Products Marker sphere including edged opening to aid in molding
8668345, Nov 30 2011 Izi Medical Products Retro-reflective marker with snap on threaded post
8672490, Nov 30 2011 Izi Medical Products High reflectivity retro-reflective marker
8687172, Apr 13 2011 FAUL, IVAN; Ivan, Faul Optical digitizer with improved distance measurement capability
8696548, Apr 17 2002 Covidien LP Endoscope structures and techniques for navigating to a target in branched structure
8696685, Apr 17 2002 Covidien LP Endoscope structures and techniques for navigating to a target in branched structure
8706185, Oct 16 2003 Medtronic Navigation, Inc. Method and apparatus for surgical navigation of a multiple piece construct for implantation
8731641, Dec 16 2008 Medtronic Navigation, Inc. Combination of electromagnetic and electropotential localization
8764725, Feb 09 2004 Covidien LP Directional anchoring mechanism, method and applications thereof
8768437, Aug 20 1998 Sofamor Danek Holdings, Inc. Fluoroscopic image guided surgery system with intraoperative registration
8791997, May 04 2007 Fraunhofer-Gesellschaft zur Foerderung der Angewandten Forschung E V; FRIEDRICH-SCHILLER-UNIVERSITAET JENA Device and method for non-contact recording of spatial coordinates of a surface
8838199, Apr 04 2002 Medtronic Navigation, Inc. Method and apparatus for virtual digital subtraction angiography
8845655, Apr 20 1999 Medtronic Navigation, Inc. Instrument guide system
8857043, Dec 30 2002 Varian Medical Systems, Inc Method of manufacturing an implantable marker with a leadless signal transmitter
8905920, Sep 27 2007 Covidien LP Bronchoscope adapter and method
8932207, Jul 10 2008 Covidien LP Integrated multi-functional endoscopic tool
9007601, Apr 21 2010 Faro Technologies, Inc. Automatic measurement of dimensional data with a laser tracker
9041914, Mar 15 2013 FARO TECHNOLOGIES, INC Three-dimensional coordinate scanner and method of operation
9055881, May 01 2005 Covidien LP System and method for image-based alignment of an endoscope
9072895, Jun 08 2001 Varian Medical Systems, Inc Guided radiation therapy system
9085401, Nov 30 2011 Izi Medical Products Packaging for retro-reflective markers
9089261, Sep 15 2003 Covidien LP System of accessories for use with bronchoscopes
9113813, Apr 08 2009 Covidien LP Locatable catheter
9117258, Jun 03 2008 Covidien LP Feature-based registration method
9146094, Apr 21 2010 Faro Technologies, Inc. Automatic measurement of dimensional data with a laser tracker
9151830, Apr 15 2011 Faro Technologies, Inc.; FARO TECHNOLOGIES, INC Six degree-of-freedom laser tracker that cooperates with a remote structured-light scanner
9157987, Apr 15 2011 Faro Technologies, Inc.; FARO TECHNOLOGIES, INC Absolute distance meter based on an undersampling method
9164173, Apr 15 2011 Faro Technologies, Inc.; FARO TECHNOLOGIES, INC Laser tracker that uses a fiber-optic coupler and an achromatic launch to align and collimate two wavelengths of light
9168102, Jan 18 2006 Medtronic Navigation, Inc Method and apparatus for providing a container to a sterile environment
9207309, Apr 15 2011 Faro Technologies, Inc.; FARO TECHNOLOGIES, INC Six degree-of-freedom laser tracker that cooperates with a remote line scanner
9237860, Jun 05 2008 Varian Medical Systems, Inc Motion compensation for medical imaging and associated systems and methods
9271803, Jun 06 2008 Covidien LP Hybrid registration method
9286506, May 22 2008 Matrix Electronic Measuring Properties, LLC Stereoscopic measurement system and method
9298078, Jul 10 2009 Steropes Technologies, LLC Method and apparatus for generating three-dimensional image information using a single imaging path
9377885, Apr 21 2010 Faro Technologies, Inc.; FARO TECHNOLOGIES, INC Method and apparatus for locking onto a retroreflector with a laser tracker
9395174, Jun 27 2014 Faro Technologies, Inc.; FARO TECHNOLOGIES, INC Determining retroreflector orientation by optimizing spatial fit
9400170, Apr 21 2010 FARO TECHNOLOGIES, INC Automatic measurement of dimensional data within an acceptance region by a laser tracker
9442362, Jul 10 2009 Steropes Technologies, LLC Method and apparatus for generating three-dimensional image information
9449378, May 22 2008 Matrix Electronic Measuring Properties, LLC System and method for processing stereoscopic vehicle information
9453913, Nov 17 2008 Faro Technologies, Inc. Target apparatus for three-dimensional measurement system
9454822, May 22 2008 Matrix Electronic Measuring Properties, LLC Stereoscopic measurement system and method
9482515, May 22 2008 Matrix Electronic Measuring Properties, LLC Stereoscopic measurement system and method
9482746, Apr 15 2011 Faro Technologies, Inc.; FARO TECHNOLOGIES, INC Six degree-of-freedom laser tracker that cooperates with a remote sensor
9482755, Nov 17 2008 Faro Technologies, Inc. Measurement system having air temperature compensation between a target and a laser tracker
9498231, Jun 27 2011 Board of Regents of the University of Nebraska On-board tool tracking system and methods of computer assisted surgery
9504530, Oct 28 1999 Medtronic Navigation, Inc. Method and apparatus for surgical navigation
9575140, Apr 03 2008 Covidien LP Magnetic interference detection system and method
9597154, Sep 29 2006 Medtronic, Inc. Method and apparatus for optimizing a computer assisted surgical procedure
9616248, Jul 23 2004 Varian Medical Systems, Inc. Integrated radiation therapy systems and methods for treating a target in a patient
9642514, Apr 17 2002 Covidien LP Endoscope structures and techniques for navigating to a target in a branched structure
9659374, Jun 03 2008 Covidien LP Feature-based registration method
9668639, Sep 27 2007 Covidien LP Bronchoscope adapter and method
9675424, Jun 04 2001 Surgical Navigation Technologies, Inc. Method for calibrating a navigation system
9682253, Jul 23 2004 Varian Medical Systems, Inc Integrated radiation therapy systems and methods for treating a target in a patient
9757087, Feb 28 2002 Medtronic Navigation, Inc. Method and apparatus for perspective inversion
9772394, Apr 21 2010 Faro Technologies, Inc. Method and apparatus for following an operator and locking onto a retroreflector with a laser tracker
9867721, Jan 30 2003 Medtronic Navigation, Inc Method and apparatus for post-operative tuning of a spinal implant
9964649, Nov 30 2011 Izi Medical Products Packaging for retro-reflective markers
9986895, Sep 27 2007 Covidien LP Bronchoscope adapter and method
D705678, Feb 21 2012 Faro Technologies, Inc. Laser tracker
RE39133, Sep 24 1997 Medtronic Navigation, Inc Percutaneous registration apparatus and method for use in computer-assisted surgical navigation
RE40852, Jun 14 1995 SURGICAL NAVIGATION TECHNOLOGIES, INC Method and system for navigating a catheter probe
RE41066, Jun 14 1995 SURGICAL NAVIGATION TECHNOLOGIES, INC Method and system for navigating a catheter probe
RE42194, Sep 24 1997 Medtronic Navigation, Inc. Percutaneous registration apparatus and method for use in computer-assisted surgical navigation
RE42226, Sep 24 1997 Medtronic Navigation, Inc. Percutaneous registration apparatus and method for use in computer-assisted surgical navigation
RE43328, Nov 20 1997 Medtronic Navigation, Inc Image guided awl/tap/screwdriver
RE43750, Jun 14 1995 Medtronic Navigation, Inc. Method for navigating a catheter probe
RE43952, Oct 05 1989 Medtronic Navigation, Inc. Interactive system for local intervention inside a non-homogeneous structure
RE44305, Sep 24 1997 Medtronic Navigation, Inc. Percutaneous registration apparatus and method for use in computer-assisted surgical navigation
RE45509, Sep 24 1997 Medtronic Navigation, Inc. Percutaneous registration apparatus and method for use in computer-assisted surgical navigation
RE46409, Nov 20 1997 Medtronic Navigation, Inc. Image guided awl/tap/screwdriver
RE46422, Nov 20 1997 Medtronic Navigation, Inc. Image guided awl/tap/screwdriver
Patent Priority Assignee Title
3821469,
3983474, Feb 21 1975 KAISER AEROSPACE & ELECTRONICS CORPORATION, A CORP OF NV Tracking and determining orientation of object using coordinate transformation means, system and process
4209254, Feb 03 1978 Thomson-CSF System for monitoring the movements of one or more point sources of luminous radiation
4585350, Jan 28 1983 LMI TECHNOLOGIES INC Pulsed robotic inspection
4649504, May 22 1984 CAE Electronics, Ltd. Optical position and orientation measurement techniques
4660970, Nov 25 1983 Carl-Zeiss-Stiftung Method and apparatus for the contact-less measuring of objects
4701047, Jun 22 1984 Dornier GmbH Line selection for preparing range images
4701049, Jun 22 1983 B V OPTISCHE INDUSTRIE DE OUDE DELFT Measuring system employing a measuring method based on the triangulation principle for the non-contact measurement of a distance from the surface of a contoured object to a reference level. _
4705395, Oct 03 1984 LMI TECHNOLOGIES INC Triangulation data integrity
4705401, Aug 12 1985 CYBERWARE LABORATORY INC , A CORP OF CALIFORNIA Rapid three-dimensional surface digitizer
4709156, Nov 27 1985 EX-CELL-O CORPORATION, A CORP OF MICHIGAN Method and apparatus for inspecting a surface
4721384, Jan 26 1985 Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raumfahrt e.V. Optical-electronic rangefinder
4721388, Oct 05 1984 Hitachi, Ltd. Method of measuring shape of object in non-contacting manner
4733969, Sep 08 1986 CyberOptics Corporation Laser probe for determining distance
4737032, Aug 26 1985 CYBERWARE LABORATORY, INC , 2062 SUNSET DRIVE, PACIFIC GROVE, CA , 93950, A CORP OF CA Surface mensuration sensor
4743771, Jun 17 1985 GENERAL SCANNING, INC Z-axis height measurement system
4745290, Mar 19 1987 FRANKEL, DAVID Method and apparatus for use in making custom shoes
4753528, Dec 13 1983 Quantime, Inc. Laser archery distance device
4761072, Oct 03 1984 LMI TECHNOLOGIES INC Electro-optical sensors for manual control
4764015, Dec 31 1986 OWENS-ILLINOIS, INC , A CORP OF OH Method and apparatus for non-contact spatial measurement
4764016, Jun 14 1985 BENGTSSON, ANDERS Instrument for measuring the topography of a surface
4767934, Jul 02 1986 Honeywell Inc. Active ranging system
4775235, Jun 08 1984 Robotic Vision Systems, Inc. Optical spot scanning system for use in three-dimensional object inspection
4782239, Apr 05 1985 U S BANK NATIONAL ASSOCIATION, AS COLLATERAL AGENT Optical position measuring apparatus
4794262, Dec 03 1985 SATO, YUKIO Method and apparatus for measuring profile of three-dimensional object
4803645, Sep 19 1985 Tokyo Kogaku Kikai Kabushiki Kaisha Method and apparatus for measuring coordinates
4821200, Apr 14 1986 JONKOPINGS LANS LANDSTING, A CORP OF SWEDEN Method and apparatus for manufacturing a modified, three-dimensional reproduction of a soft, deformable object
4822163, Jun 26 1986 Rudolph Technologies, Inc Tracking vision sensor
4825091, Feb 05 1987 Carl-Zeiss-Stiftung Optoelectronic distance sensor with visible pilot beam
4829373, Aug 03 1987 VEXCEL IMAGING CORPORATION Stereo mensuration apparatus
4836778, May 26 1987 Vexcel Corporation Mandibular motion monitoring system
4982188, Sep 20 1988 Grumman Aerospace Corporation System for measuring positional characteristics of an ejected object
//////////
Executed onAssignorAssigneeConveyanceFrameReelDoc
Mar 30 1995Image Guided Technologies Inc.(assignment on the face of the patent)
Aug 17 1999IMAGE GUIDED TECHNOLOGIES, INC , A CORPORATION OF COLORADO, F K A PIXSYS, INC Silicon Valley BankSECURITY INTEREST SEE DOCUMENT FOR DETAILS 0101880799 pdf
Aug 17 1999SPRINGFIELD SURGICAL INSTRUMENTS, A CORPORATION OF MASSACHUSETTS, F K A BRIMFIELD PRECISION, INC Silicon Valley BankSECURITY INTEREST SEE DOCUMENT FOR DETAILS 0101880799 pdf
Dec 21 2007NORTHERN DIGITAL INC BANK OF MONTREALSECURITY INTEREST SEE DOCUMENT FOR DETAILS 0207940239 pdf
Dec 21 2007NORTHERN DIGITAL INC BMO CAPITAL CORPOORATIONSECURITY INTEREST SEE DOCUMENT FOR DETAILS 0207620157 pdf
Dec 21 2007NORTHERN DIGITAL INC BMO CAPTIAL CORPORATIONSECURITY AGREEMENT0207620131 pdf
Dec 21 2007NORTHERN DIGITAL INC BANK OF MONTREALASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0207620082 pdf
Dec 21 2007NORTHERN DIGITAL INC BMO CAPITAL CORPORATIONCORRECTION OF ASSINGEE INFORMATION FROM BMO CAPTIAL CORPOORATION TO BMO CAPITAL CORPORATION 0208280379 pdf
Aug 04 2010BMO CAPITAL CORPORATIONNORTHERN DIGITAL INC RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS 0250000396 pdf
Aug 04 2010BANK OF MONTREALNORTHERN DIGITAL INC RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS 0249460944 pdf
Date Maintenance Fee Events
Nov 06 2000ASPN: Payor Number Assigned.
Nov 06 2000M181: 7.5 yr surcharge - late pmt w/in 6 mo, Large Entity.
Nov 06 2000M184: Payment of Maintenance Fee, 8th Year, Large Entity.
Nov 13 2000LSM2: Pat Hldr no Longer Claims Small Ent Stat as Small Business.
Sep 30 2004M1553: Payment of Maintenance Fee, 12th Year, Large Entity.


Date Maintenance Schedule
Jun 02 20014 years fee payment window open
Dec 02 20016 months grace period start (w surcharge)
Jun 02 2002patent expiry (for year 4)
Jun 02 20042 years to revive unintentionally abandoned end. (for year 4)
Jun 02 20058 years fee payment window open
Dec 02 20056 months grace period start (w surcharge)
Jun 02 2006patent expiry (for year 8)
Jun 02 20082 years to revive unintentionally abandoned end. (for year 8)
Jun 02 200912 years fee payment window open
Dec 02 20096 months grace period start (w surcharge)
Jun 02 2010patent expiry (for year 12)
Jun 02 20122 years to revive unintentionally abandoned end. (for year 12)