The invention is an optical feature extraction apparatus which uses video display, spatial light modulation, and detection components in conjunction with microlenslet replicating optics, to expedite the recognition of DNA sequences based on their symmetry properties and, specifically, to classify short (6 bases in length) sequences of DNA as palindrome or nonpalindrome. The DNA sequences are symbolically encoded using a novel method. Multichannel operation is achieved through the replication of input scenery, making possible a higher throughput rate than for single channel systems.
|
1. An optical feature extraction apparatus for detecting data comprising:
a filter for decoding the data to be detected comprising a plurality of opaque and transparent regions; a display means for displaying the filter, the displayed filter spatially modulating the light from the display means and creating an intensity distribution of light; a means for replicating the light intensity distribution received from the display means; a mask encoded with the data to be classified, the mask receiving and further modulating the light intensity distribution; and a means for detecting the further modulated light intensity distribution received from the mask.
2. The optical feature extraction apparatus as recited in
3. The optical feature extraction apparatus as recited in
4. The optical feature extraction apparatus as recited in
5. The optical feature extraction apparatus as recited in
6. The optical feature extraction apparatus as recited in
7. The optical feature extraction apparatus as recited in
8. The optical feature extraction apparatus as recited in
9. The optical feature extraction apparatus as recited in
10. The optical feature extraction apparatus as recited in
11. The optical feature extraction apparatus as recited in
12. The optical feature extraction apparatus as recited in
|
This invention was made with Government support under contract No. N00039-91-C-0001 awarded by the U.S. Navy Department. The Government has certain rights in this invention.
The invention relates to a novel optical feature extractor and a method for symbolically encoding DNA bases to permit detection of a class of DNA sequences based on their symmetry.
The degree of computational parallelism available using optics has generated great interest in optical approaches to pattern recognition and computation in general. The performance of optical pattern recognition techniques has in some cases been quite remarkable, while in others the results have been less than satisfactory. The level of performance often has much to do with how well-conditioned a problem is to an optical approach or architecture. For example, problems in image interpretation may be complicated by scale, rotation, and perspective variabilities or distortions that reduce the degree of correlation between reference objects and images under examination. Assembly line inspection, optical character recognition of printed text, and other problems in which image content and structure are constrained are more tractable due to the reduced number of degrees of freedom.
In some cases transformations exist that allow the offending properties of images to be removed (e.g., use of the Mellin transform to convert scale variant objects to translation variant features). In others the investigator is able to exploit the structure or representation of a problem by using optical methods. DNA sequence analysis is a problem that can be exploited by using optical methods.
The sequence of bases along each strand of DNA forms the fundamental genetic information in each individual and organism. While only 4 types of subunits are used in DNA (denoted by the symbols A, C, G, and T in FIG. 1), the number of different possible sequences of length N is 4N. As the length N of the human genome is approximately 3×109 this allows for a rather large number of possible human DNA sequences. An important structural element of DNA is that the sequences on each of the two strands of which DNA is composed are complementary; an A base on one strand is hydrogen bonded to a T base on the opposite strand of the double helix. Likewise, a G on one strand indicates a C on the opposite strand at the same position along the DNA. Thus, knowledge of the sequence of one strand immediately defines the sequence along the complementary strand.
In order to understand the genetic basis of disease and evolutionary relationships between species and individuals, molecular biologists have found it useful to sift through a vast collection of such sequences, looking for themes and relationships between subsequences constituting genes and other genetic landmarks. Their efforts have been impressive yet modest, given the scale of the problem and the computational resources available.
A set of sequences known as "palindromes" are important landmarks, both to the biochemical machinery of the living cell and for navigational purposes to the investigator. They are a class of DNA sequences known to have special regulatory functions in biological systems and distinguished by the antisymmetric arrangement of bases in palindromic sequences. They are DNA sequences that have a 2-fold inversion symmetry (i.e., sequences that conform to the scheme XY=YX).
A system of enzymes ("restriction enzymes") able to recognize and act on palindromic sequences has evolved in many living organisms. The symmetric composition of the DNA restriction sites has been exploited in the evolution of enzymes having in many cases a 2-fold dyadic structure. These enzymes are used by researchers to cut and splice segments of DNA at the sites of the palindromic sequences recognized by the restriction enzymes.
Currently, genetic sequence databases contain on the order of 108 bases of sequence data gathered from organisms of all types, determined primarily in the last 10 years. The rate of sequence determination is increasing exponentially and is expected to do so for years to come. With each newly logged sequence a time consuming search for palindromes is begun. The scientific community needs new approaches to reduce the search time required.
The need for faster searching of DNA sequences has resulted in the development of a multichannel optical architecture which can rapidly search for symbolically encoded sequences of DNA base information and detect palindromic sequences of length-6, a common object of sequence searches. This approach uses video display, spatial light modulation, and detection components in conjunction with microlenslet replicating optics, to expedite the recognition of symbol sequences based on their symmetry properties. Multichannel operation is achieved through the replication of input scenery, making possible a higher throughput rate than for single channel systems. The method of the invention uses a symbolic representation of the DNA subunits to facilitate the classification of sequences as palindromic or nonpalindromic.
A notable feature of this optical approach is the exchanged positions of input scenery and the filter set. The conventional treatment has been to display the input scene on a monitor for projection onto a set of feature extraction vectors realized as amplitude modulated LCTV devices or lithographically prepared masks. Instead, the invention provides the filter set as input to the system and correspondingly places the sequence data in the filter plane of the system, relying on the commutativity of projection to allow this role reversal.
The optical feature extractor of the invention can classify short (6 bases in length) sequences of DNA as palindrome or nonpalindrome. This classification is made on the basis of the sequence symmetry, independent of base composition.
FIG. 1 illustrates a tract of a double stranded DNA molecule containing a length-6 palindromic sequence (in the box) within a longer sequence. Each strand is read in the direction of the arrows.
FIG. 2, consisting of FIGS. 2a, 2b, and 2c, illustrates the DNA base representation method of the invention as follows: FIG. 2a: symbols for individual bases: FIG. 2b: a nonpalindromic sequence (AATTCG) of length 6: FIG. 2c: a length-6 palindrome. (Note the 2-fold rotation symmetry of FIG. 2c.) The stippled cells correspond to transparent regions of the mask on which the DNA sequences are encoded, while white cells correspond to opaque regions.
FIG. 3 illustrates schematically the optical components of the feature extractor of the invention: (a) display monitor, (b) lenslet array, (c) mask, (d) tapered fiber bundle, (e) collection lens, and (f) CCD detector array.
FIG. 4 illustrates imaging of one of the display filters (number 4) comprising g(x,y) onto a single copy of a sequence block of tm (x,y)' by a lenslet of a 2×2 array. Each lenslet images the complete filter set onto 4 copies of a sequence block in the mask plane. Four blocks are shown in the figure.
FIG. 5 illustrates a two-dimensional view of the arrangement of the filter set f(x,y), lenslet array L, and mask tm (x,y)' of the optical feature extractor of the invention. The most divergent ray of the optical system is shown as a dotted line at an angle Ω/2 to the z-axis. Lenslet diameter is denoted by d1, lenslet interspacing by Δy, and filter height by Y. Larger numbers of lenslets or increased lenslet interspacing would result in pronounced vignetting due to a consequent increase Ω/2. A TFB with sufficient numerical aperture can receive all rays exiting the mask.
FIG. 6 illustrates a region of a CCD detector in the optical feature extractor of the invention that is illuminated by a filter-mask product. Each of the regions a, b, c, and d is integrated to a single signal for subsequent processing.
The optical architecture of the invention described herein serves as the main computational engine of a DNA sequence analysis system. Such a system uses dynamic spatial light modulators in the optical architecture and integrates this architecture with a rapid access DNA database and a post-detection decision processor.
The basic goal is to use a sequence representation that allows a determination whether any 6-base sequence is palindromic or not. This is done by examining the sequence for the 2-fold symmetry that characterizes palindromes rather than by examining the sequence composition on a position-by-position basis. This method saves the searcher from having to compare each sequence with a list of length-6 palindromes, of which there are 64. Thus the search is carried out by probing for a single symmetry feature common to all 64 possible palindromes rather than by performing a comparison between the sequence in question and each sequence on this list. This reduces the computational complexity of the search tremendously.
Symmetry searching is made possible by use of a binary DNA base representation (FIG. 2a) that has been designed in cognizance both of the base pairing rules governing DNA sequences and of the goals of the searches undertaken. Each symbol is made up of four cells (square in this embodiment) that are used to spatially modulate the intensity of an impinging beam of light. In this representation, the symbol for an A base is just the complement of the symbol for a T, while similarly the symbol for a G is the complement of C. The 4-bit representation used here also ensures that the A symbol is the same as the T symbol in reverse order and likewise for G and C. Finally, a design has been selected in which each of the four symbols has 2 opaque cells and 2 transparent cells; this balance gives each type of base equal weighting and ensures that complementarity can be encoded in the representation.
A sequence of bases can be represented by stacking these base symbols vertically (FIG. 2b). Stacks of six symbols, representing tracts of six consecutive bases in a DNA sequence, will have perfect 2-fold symmetry if and only if the sequence is palindromic. Examples of nonpalindromic and palindromic sequences encoded in this representation appear in FIGS. 2b and 2c.
The filter set used in this work is designed to extract the features necessary to classify the sequence blocks of interest here (palindromes). The optical architecture is more general--different searches and classification schemes can be implemented by use of different filter sets and data representations.
The invention utilizes replicating micro-optics to achieve multichannel correlation between the DNA sequence blocks under investigation and a set of filters for the purpose of forming optical products between the filters and sequence blocks. Subsequent optical components transfer these products to a charge coupled device (CCD) detector array for measurement. This embodiment is shown in FIG. 3.
A filter set, composed of dark (opaque) and bright (transparent) square regions, is displayed on a television screen and replicated by a rectilinear array of plano-convex lenslets. The filter intensity distribution is described by the function g(x,y). Imaging of the display object g(x,y) by the array of lenslets onto a mask encoded with blocks of sequence data is shown in FIG. 4.
A subsampled hexagonal array made by the Corning FOTOFORM® process is used to perform multichannel classification. Each lenslet in the array has a diameter of d1 =450 μm, with a center-to-center spacing of 584 μm. Photolithographic mask material is used to block light from all but a 4×4 subarray of lenslets on a Cartesian lattice. The lenslet interspacing distances of this subarray Δx and Δy are equal to 7.008 mm and 6.069 mm, respectively. For a focal length f(20 mm for the lenslets used in this work), the object and image distances so and si obey the Gaussian lens equation:
1/f=1/so +1/si, (1)
and
M=1/μ=-si /so. (2)
Consider the display object g(x,y) centered on the origin, normal to the optical axis z, and of width X and height Y in the x-y plane. A lenslet array is placed normal to the z-axis at z=so, with the z-axis passing through a point equally spaced between lenslets separated horizontally by Δx and vertically by Δy. The position of each lenslet in the array is indexed by the numbers nx and ny ; positive values for nx and ny indicate a position to the right of and above the origin, respectively. The center of the image from lenslet (nx,ny) has coordinates (xo,yo)'
x0 =(1+|M|)·(2nx -1)·Δx /2,
y0 =(1+|M|)·(2ny -1)·Δy /2, (3)
in the image (primed) plane z=s0 +si, as determined by Eq. 1. The light intensity in the image plane for an ideal lossless case is
I(x,y)'=μ2 ·g[μ·(x-x0)',μ·(y-y0)'](4)
The factor μ2 accounts for the change in area between object and image. A sequence block encoded as a binary transmission object and centered at these coordinates in the focal plane of the lenslet array will further modulate the light intensity. The intensity distribution representing the filter-mask product Im (x,y)' available just beyond this mask is given by
Im (x,y)'=I(x,y)'·tm (x,y)', (5)
where tm (x,y)' is the transmission function of the mask. An array with focal length f=20 mm and an object distance so =70 mm is used, yielding an image distance of si =28 mm. For |so |>2f, 0>M>-1, an inverted, minified image of the display filters results.
In fact, the displayed function g(x,y) is made up of 4 subfilters. The mask tm (x,y)' is made up of 16 cells laid out on a 4×4 lattice. Each such cell contains 4 copies of a length-6 sequence block. Thus, each of the 16 lenslets projects a set of 4 filters onto 4 copies of a sequence block. The intensity distribution Im (x,y)' due to a single lenslet then contains the information necessary for classification of the sequence block subsequent to detection by a CCD array.
An important result here is that by selective subsampling of the lenslet array, different, even variable spacing of the mask plane patterns can be accommodated for a given magnification. This allows some flexibility in the use of mask devices having varying scales and spatial resolutions, which can be accommodated by revision of object and image distances and the use of different lenslet array subsamplings. It is possible to select values of so, si, nx, and ny such that the lenslets yield focused, over-lapping images in the mask plane. Such imaging is undesirable in this application but might prove useful for other correlation studies. The condition for no overlap in the x' or y' dimension is
|M|·X<Δx, |M|·Y<Δy. (6)
For any given values of so, si, nx, and ny overlap can be averted by reducing X or Y, but at the cost of reduced use of the space-bandwidth product available for a display of given resolution and extent.
FIG. 5 illustrates the problem of divergence due to an off-axis object. Because each lenslet sees the display as an off-axis object, the chief rays of each of the lenslets diverge. The divergence is a function of both object-lenslet-image dimensions and the lenslet interspacings. Chief rays of each lenslet diverge from the optical axis of the system at a half angle of
φx /2=tan-1 [Δx ·(2nx -1)/(2so)],
φy /2=tan-1 [Δy ·(2ny -1)/(2so)]. (7)
A single angle Ω can be used to express divergence in both x- and y-directions. Correspondingly, diagonal distances (using s- to replace x- and y-directions) are used in the two-dimensional formulae. Divergence of light from each lenslet about its chief ray increases the system overall divergence half angle to
Ω/2=tan-1 {[|M|·[Δs /2·(2ns -1)+S/2]+d1 /2]/si }, (8)
where
ns =(nx2 +ny2)1/2,S=(X2 +Y2)1/2, and Δs =(Δx2 +Δy2)1/2. (9)
A significant part of this divergence results from the lenslet spacings. This is effectively equivalent to a high divergence source placed in the object plane, and increases the demand for fast collection optics to image the mask plane onto the detector array. The approach here has been to reduce the divergence of the system by using an imaging tapered fiber bundle (TFB) placed in contact with the mask. The basic operation of the TFB is to reduce the scale of the object Im (x,y)' by a factor r, so that the light is redistributed at the fiber bundle output plane (x,y)" as
Ib (x,y)"=k·r2 ·Im (rx,ry)',(10)
where k is the transmission efficiency of the TFB. A TFB allowing a scale reduction of r=2 was placed in contact with the sequence mask. The diameters of individual fibers comprising the TFB are 6 μm at the input (large) end and 3 μm at the output end. These fibers are arranged as a hexagonal close packed array. The smallest opaque or transparent region of the distribution Im (x,y)' illuminates approximately 7000 fibers, ensuring adequate resolution of the input image by the TFB and subsequent optics.
The transmission efficiency k of the TFB is approximately 50% (some of the light impinging on the TFB is lost between fibers and the remainder lost to reflection and attenuation within the bundle). The reduction in area by a factor of 4 in transiting the TFB combines with the light loss to yield an approximate doubling of the output irradiance.
The reduced size of the filter-mask product is not sufficient using this TFB to match the dimensions of the CCD array detector (8.8 mm horizontal×6.6 mm vertical). Moreover, the numerical aperture of the TFB output is large (approximately 1.0). In transiting the CCD array faceplate of thickness ∼1.0 mm, light from each fiber would broaden to the extent that poor image formation at the CCD detector array would result, with consequent loss of resolution.
To circumvent this shortcoming, a lens system to match the TFB output to the dimensions of the detector is used. A further benefit of the TFB is that all fibers in the bundle have parallel axes at the output end, reducing the divergence and lessening the demands on the collection optics. The numerical aperture of the collection optics need only accept the divergence resulting from the assemblage of individual parallel fibers.
The described optical system is designed to achieve faithful reproduction of the filter-mask products from each DNA sequence block at the plane of the CCD detector array. In order to classify each sequence block as palindrome or nonpalindrome, subsequent processing of the optical signals must be performed. The CCD signals are processed by a frame grabber and the raster signal passed to image processing software. This software segments the data for each filter-mask product into 4 regions (see FIG. 6). The signals within each region are integrated by software, resulting in a set of 16 signals for each sequence under investigation. Because of the cellular composition of the filter-mask products and the shape of the integration regions, the signal arising from an integrated region will have values falling into 4 ranges. Ideally, these signals can be scaled to quantized values of only 0, 1, 2, or 3. These correspond to the number of illuminated regions. Thus, the grayscale resolution required of the detector is not great (2 bits). In reality, distortion, noise, and unequal illumination and transmission across the optical components cause these signal levels to broaden. Thresholding is used in the post-detection processor to assign scaled, quantized levels to the measured integrated signals. Palindromic sequences result in equal signals among all pairs of symmetrically placed integration regions while nonpalindromic sequences do not. Equality of these pairs is determined by computer software, which then performs the sequence classification.
The filter set described above permits probing the DNA sequences for the information necessary to classify sequences correctly as palindromic or nonpalindromic. This problem represents a specific case of a larger set of analyses relevant to DNA research. The strategy employed here can be applied to many other DNA sequence analyses through the development of different filter sets for the extraction of features of importance to those analyses. Where necessary, different sequence representations can be developed to carry the required sequence information.
The correlator-classifier architecture described uses replicating lenslet arrays to present multiple copies of an off-axis input image to the remainder of the optical system. The principal benefits of such arrays are the uniform magnification and spacing of the images, and the possibility of varying the spacing by subsampling the array. A property constraining their use is the vignetting that can occur for significantly off-axis objects. Increasing the lenslet diameter, decreasing the lenslet spacing, and reducing the sequence mask features would all help to keep rays close to the system optical axis.
A TFB with sufficiently large image reduction could be butted directly against the CCD detector active area, eliminating the need for collection lenses to image the mask plane onto the CCD. Such a TFB-detector device would greatly reduce the length of the optical system. This configuration would also reduce the numerical aperture (and size) of the collection optics by receiving the mask output before its divergence is further broadened. The diameter of fibers in TFBs are typically small compared to CCD pixel dimensions. Coordinating the fiber and CCD pixel dimensions and arrangements would maintain optimal resolution.
An efficient and general DNA sequence analysis system can use the described architecture. Several requirements of such a system are clear. Improvement in the optical architecture will result from the use of dynamic spatial light modulators (SLM) for both filter and sequence input. Speed of the system will result from transferring the contents of a DNA sequence library (GenBank release 61) stored on CD-ROM to the SLM in place of the litho-graphically produced mask described herein. Reducing the vignetting associated with large lenslet arrays to allow a larger number of channels is central. Also of interest are tradeoffs between speed, spatial resolution, dynamic range, and information density, and how to find the proper balance between the limitations of available devices. The processing rates for a multi-channel correlation system incorporating dynamic spatial light modulators are projected as follows:
______________________________________ |
Current Projected |
Architecture |
Architecture |
______________________________________ |
Frame Rate 30 Hz 100 Hz |
Number of channels |
16 1024 |
Correlations/second |
480 105 |
Binary operations/second |
2.3 × 104 |
4.8 × 106 |
Genbank search time |
2.4 days 17 minutes |
______________________________________ |
This projection is based on use of a 4×4 lenslet array and video frame rates (30 Hz) for the current configuration. Use of a parallel CCD detection array device in place of the frame rate device is assumed in the projections. Future realizations would employ high frame rate magneto-optic spatial light modulators and increased lenslet numbers.
Palindrome detection takes place in a single pass through the system. This architecture can be used more generally to perform a broader set of analyses or classification operations by inclusion of a feedback system. The post-detection processor used here to perform classification is capable of selecting alternate filter sets based on the signals extracted during the first pass. Interrogation of the sequence blocks during subsequent passes can extract the signals necessary to make other common DNA sequence classifications.
Boone, Bradley G., Christens-Barry, William A., Terry, David H.
Patent | Priority | Assignee | Title |
10009556, | Aug 25 2004 | Intellectual Ventures II LLC | Large dynamic range cameras |
10142548, | Aug 25 2004 | Intellectual Ventures II LLC | Digital camera with multiple pipeline signal processors |
10148927, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
10694162, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
11412196, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
11425349, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
11706535, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
6620586, | Feb 20 2001 | Applied Gene Technologies | Methods and compositions for analyzing nucleic acids |
6707020, | Dec 28 1999 | MZA Associates Corporation | Adaptive dynamic range wavefront sensor |
7714262, | Jul 01 2005 | Intellectual Ventures II LLC | Digital camera with integrated ultraviolet (UV) response |
7772532, | Jul 01 2005 | Intellectual Ventures II LLC | Camera and method having optics and photo detectors which are adjustable with respect to each other |
7795577, | Aug 25 2004 | Intellectual Ventures II LLC | Lens frame and optical focus assembly for imager module |
7884309, | Aug 25 2004 | Intellectual Ventures II LLC | Digital camera with multiple pipeline signal processors |
7916180, | Aug 25 2004 | Intellectual Ventures II LLC | Simultaneous multiple field of view digital cameras |
7964835, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
8124929, | Aug 25 2004 | Intellectual Ventures II LLC | Imager module optical focus and assembly method |
8198574, | Aug 25 2004 | Intellectual Ventures II LLC | Large dynamic range cameras |
8304709, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
8334494, | Aug 25 2004 | Intellectual Ventures II LLC | Large dynamic range cameras |
8415605, | Aug 25 2004 | Intellectual Ventures II LLC | Digital camera with multiple pipeline signal processors |
8436286, | Aug 25 2004 | Intellectual Ventures II LLC | Imager module optical focus and assembly method |
8526824, | Mar 21 2011 | Adtran, Inc. | Correlation systems and methods with error compensation |
8598504, | Aug 25 2004 | Intellectual Ventures II LLC | Large dynamic range cameras |
8629390, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
8664579, | Aug 25 2004 | Intellectual Ventures II LLC | Digital camera with multiple pipeline signal processors |
9232158, | Aug 25 2004 | Intellectual Ventures II LLC | Large dynamic range cameras |
9294745, | Aug 25 2005 | Intellectual Ventures II LLC | Digital cameras with direct luminance and chrominance detection |
9313393, | Aug 25 2004 | Intellectual Ventures II LLC | Digital camera with multiple pipeline signal processors |
Patent | Priority | Assignee | Title |
3064519, | |||
3227034, | |||
3248552, | |||
3726997, | |||
3785736, | |||
4637055, | May 27 1983 | PA Consulting Services Limited | Adaptive pattern recognition |
4862511, | Jun 15 1987 | NIPPON SHEET GLASS CO , LTD | Local feature analysis apparatus |
5086483, | Aug 31 1989 | The Boeing Company | Optical processor including electronic processing array |
5131055, | Feb 16 1990 | The United States of America as represented by the Administrator of the | Auto and hetero-associative memory using a 2-D optical logic gate |
5148502, | Feb 23 1988 | Olympus Optical Co., Ltd. | Optical image input/output apparatus for objects having a large focal depth |
5216484, | Dec 09 1991 | The United States of America as represented by the Administrator of the | Real-time imaging spectrometer |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Nov 13 1992 | The Johns Hopkins University | (assignment on the face of the patent) | / | |||
Nov 19 1992 | TERRY, DAVID H | Johns Hopkins University, The | ASSIGNMENT OF ASSIGNORS INTEREST | 006387 | /0078 | |
Nov 19 1992 | CHRISTENS-BARRY, WILLIAM A | Johns Hopkins University, The | ASSIGNMENT OF ASSIGNORS INTEREST | 006387 | /0078 | |
Nov 19 1992 | BOONE, BRADLEY G | Johns Hopkins University, The | ASSIGNMENT OF ASSIGNORS INTEREST | 006387 | /0078 |
Date | Maintenance Fee Events |
Jun 14 2002 | M2551: Payment of Maintenance Fee, 4th Yr, Small Entity. |
Jul 02 2002 | REM: Maintenance Fee Reminder Mailed. |
Jul 19 2002 | ASPN: Payor Number Assigned. |
Jun 15 2006 | M2552: Payment of Maintenance Fee, 8th Yr, Small Entity. |
Jul 19 2010 | REM: Maintenance Fee Reminder Mailed. |
Dec 15 2010 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
Date | Maintenance Schedule |
Dec 15 2001 | 4 years fee payment window open |
Jun 15 2002 | 6 months grace period start (w surcharge) |
Dec 15 2002 | patent expiry (for year 4) |
Dec 15 2004 | 2 years to revive unintentionally abandoned end. (for year 4) |
Dec 15 2005 | 8 years fee payment window open |
Jun 15 2006 | 6 months grace period start (w surcharge) |
Dec 15 2006 | patent expiry (for year 8) |
Dec 15 2008 | 2 years to revive unintentionally abandoned end. (for year 8) |
Dec 15 2009 | 12 years fee payment window open |
Jun 15 2010 | 6 months grace period start (w surcharge) |
Dec 15 2010 | patent expiry (for year 12) |
Dec 15 2012 | 2 years to revive unintentionally abandoned end. (for year 12) |