The present invention provides a measurement system and an image processing system for quantitatively figuring out the fiber bundles which are passing through any voi. A static magnetic field and an RF signal are applied to a subject, and a nuclear magnetic resonance signal is received from the subject (401). Diffusion tensor is calculated from the nuclear magnetic resonance signals (402). As to a target area for receiving the nuclear magnetic resonance signal from the subject, fiber bundles passing through multiple predetermined origins, respectively, are extracted in a form of a group of coordinate points for each of the fiber bundles, based on the diffusion tensor calculated by the calculating means (406). At least one voi is set for the target area for receiving the nuclear magnetic resonance signal (408). Out of the multiple fiber bundles extracted by the fiber bundle extracting means, the fiber bundles having at least one coordinate point of the group of coordinate points being included in the voi are discriminated and the number of which is counted (409).
|
4. An nmr image processing system which obtains an optimum range of a voi (Volume Of Interest) size, comprising:
a memory means for storing nuclear magnetic resonance signal data previously acquired from a subject;
a calculating means for calculating a diffusion tensor from the nuclear magnetic resonance signal data;
a fiber bundle extracting means for extracting fiber bundles as a group of coordinate points that represents each of the fiber bundles included in a target range which receives the nuclear magnetic resonance signal from the subject, based on the diffusion tensor calculated by the calculating means;
a voi selecting means for setting at least one voi within the target range which receives the nuclear magnetic resonance signal;
a fiber bundle number discriminating means for discriminating the fiber bundles which have at least one coordinate point of the group of coordinate points included in the voi, from the fiber bundles extracted by the fiber bundle extracting means, and configured for counting the number of the fiber bundles; and where
the voi selecting means arranges various sizes of VOIs, allows the fiber bundle number discriminating means to count the number of the fiber bundles included in each of the various sizes of VOIs, and performs a predetermined computation as to the number of fiber bundles of the various sizes of VOIs, whereby an optimum range of the voi size is obtained.
1. An nmr measurement system which obtains an optimum range of a voi (Volume Of Interest) size, comprising:
a magnetic field application means for applying a static magnetic field and a radio frequency field to a subject;
a receiving means for receiving a nuclear magnetic resonance signal from the subject;
a calculating means for calculating a diffusion tensor from the nuclear magnetic resonance signals;
a fiber bundle extracting means for extracting fiber bundles as a group of coordinate points that represents each of the fiber bundles included in a target range which receives the nuclear magnetic resonance signal from the subject, based on the diffusion tensor calculated by the calculating means;
a voi selecting means for setting at least one voi within the target range which receives the nuclear magnetic resonance signal;
a fiber bundle number discriminating means for discriminating the fiber bundles which have at least one coordinate point of the group of coordinate points included in the voi, from the fiber bundles extracted by the fiber bundle extracting means, and configured for counting the number of the fiber bundles; and where
the voi selecting means arranges various sizes of VOIs, allows the fiber bundle number discriminating means to count the number of the fiber bundles included in each of the various sizes of VOIs, and performs a predetermined computation as to the number of fiber bundles of the various sizes of VOIs, whereby an optimum range of the voi size is obtained.
2. The nmr measurement system according to
the voi selecting means sets a voi group including at least one voi, and a discrimination condition indicating a neural path of the fiber bundles to be discriminated against with respect to the at least one voi in the set voi group, and wherein
the fiber bundle number discriminating means discriminates the fiber bundles and counts the number thereof within the set voi group, according to the discrimination condition.
3. The nmr measurement system according to
a comparator configured for comparing the numbers of the fiber bundles with one another, when the voi selecting means sets at least two of the VOIs or at least two of the voi groups and wherein the number of the fiber bundles is counted with respect to each of the VOIs or the voi groups, which have then been set by the voi selecting means.
5. The nmr image processing system according to
the voi selecting means sets a voi group including at least one voi, and a discrimination condition indicating a neural path of the fiber bundles to be discriminated against with respect to the at least one voi in the set voi group, and wherein
the fiber bundle number discriminating means discriminates the fiber bundles and counts the number thereof within the set voi group, according to the discrimination condition.
6. The nmr image processing system according to
a comparator configured for comparing the numbers of the fiber bundles with one another, when the voi selecting means sets at least two of the VOIs or at least two of the voi groups and wherein the number of the fiber bundles is counted with respect to each of the VOIs or the voi groups, which have then been set by the voi selecting means.
|
The present application claims priority from Japanese application JP 2006-318981 filed on Nov. 27, 2006, the content of which is hereby incorporated by reference into this application.
1. Field of the Invention
The present invention relates to a measurement system and an information processing system using a nuclear magnetic resonance. More particularly, it relates to the measurement system and the image processing system that quantitatively evaluate neural fiber bundles being extracted based on diffusion tensor.
2. Description of the Related Art
In recent years, tractography has been developed, which represents a fiber bundle such as a white matter fiber, by utilizing a nuclear magnetic resonance imaging (hereinafter, referred to as “MRI”). This technique is now becoming established as a strong tool for a brain scientific research. In addition, this technique is expected to be applied to a diagnosis of lesion of the central nervous system, a preoperative examination of a brain surgical operation, and the like.
The tractography is based on a diffusion anisotropic measurement. In this measurement, MPGs (Motion Probing Gradient), being a gradient magnetic field that enhances a change of signal amount due to the molecular diffusion, is applied in at least seven directions so as to measure diffusion-weighted images, and diffusion tensor corresponding to each voxel of these diffusion-weighted images is calculated. In a fiber-like tissue such as a white matter made up of neural fibers, a direction in which the internal water molecules diffuse is restricted by the fiber, and this indicates anisotropy. Therefore, by using information of an eigenvalue and an eigenvector, which can be obtained by diagonalizing the diffusion tensor, a pixel having high diffusion anisotropy is sequentially traced along a direction in which a diffusion coefficient is maximized (a direction of the eigenvector having a maximum eigenvalue), thereby enabling an imaging of the fiber bundles. A technique for imaging the fiber bundles based on the diffusion tensor is described, for example, in the Japanese Unexamined Patent Application Publication No. 11000320, hereinafter referred to as “patent document 1”, and in “PROCEEDINGS OF INTERNATIONAL SOCIETY OF MAGNETIC RESONANCE IN MEDICINE, 320 (1999)”, hereinafter referred to as “non-patent document 1”.
The fiber bundles are traced as the following; voxels included in any area on the diffusion tensor image are set as origins, and fiber bundles passing through respective origins are traced, and a series of image data constituting each fiber bundle is stored. The area for the voxels being the origins is referred to as a seed area. In selecting an area of interest, an operator may specify any position in a magnetic resonance image, by way of example. Alternatively, an area having high diffusion anisotropy may be extracted based on the diffusion tensor, a brain area extracted from a result of a brain functional measurement such as fMRI (functional magnetic resonance image) may be used, or a specific portion obtained from priori information may be used.
In the brain functional measurement such as the fMRI, a brain activated area associated with a particular impulse is created as an image. In order to understand functions of the brain, it is important to know anatomical connectivity between these brain activated areas. There is a method being used frequently, which displays a three-dimensional image of neural fiber bundles between the brain activated areas being obtained by the brain functional measurement, and visually evaluates the connectivity of each neural path. In many cases, in order to figure out a positional relationship between the area of interest within the brain and each neural path, data of the neural fiber bundles is displayed in superimposing manner on an anatomical image such as a nuclear magnetic image.
One of the methods to quantitatively evaluate the connectivity is described in “MAGNETIC RESONANCE IN MEDICINE, 1077-1088 (2003)”, hereinafter, referred to as “non-patent document 2”. In drawing the fiber bundles, there exists uncertainty due to a noise, artifact, incomplete modeling of diffusion signals, and the like. In this method, the above uncertainty is represented in the form of local probability density function based on the diffusion model, and by using this probability density function, a probability of existence of fiber bundle connection between any two points is estimated.
However, in the conventional method that displays the neural fiber bundle data in superimposing manner on the anatomical image, it is not possible to quantitatively figure out the neural fiber bundles. In the method to estimate the probability of existence of fiber bundle connection between any two points, as global connectivity by using the probability density function, it is not possible to compare the connectivity intensity between any of the neural paths, with others.
The present invention has been made to solve the problems shown in the conventional techniques as described above, and an object of the present invention is to provide a measurement system and an image processing system in which fiber bundles passing through any VOI (volume of interest) are quantitatively figured out.
In order to solve the above problem, according to a first aspect of the present invention, a measurement system as described below is provided. In other words, the measurement system includes, a magnetic field application means for applying a static magnetic field and a radio frequency field to a subject, a nuclear magnetic resonance signal receiving means for receiving a nuclear magnetic resonance signal from the subject, a calculating means for calculating diffusion tensor from the nuclear magnetic resonance signal, a fiber bundle extracting means for selecting seed areas confined to measurement area for the nuclear magnetic resonance signals from the subject, and for extracting fiber bundles by using the seed areas as starting points of the extraction based on the diffusion tensor calculated by the calculating means, and for keeping a group of coordinate points for each of the fiber bundles, a VOI selecting means for setting at least one VOI as to the target range for receiving the nuclear magnetic resonance signal, and a fiber bundle number discriminating means for discriminating the fiber bundles at least one coordinate point of the group of coordinate points of which is included in the VOI, from multiple fiber bundles extracted by the fiber bundle extracting means, and for counting the number of the fiber bundles. With the configuration above, it is possible to count the number of the fiber bundles passing through the VOI.
The VOI selecting means is capable of setting a VOI group including at least one VOI, and a discrimination condition that indicates what kind of neural path the fiber bundles to be discriminated are taking, when passing through at least one of the VOIs constituting the VOI group. The fiber bundle number discriminating means discriminates the fiber bundles and counts the number thereof as to the VOI group, according to the discrimination condition. Accordingly, it is possible to quantitatively figure out the connectivity in a predetermined neural path in the VOI group, in the form of the number of fiber bundles.
By way of example, the discrimination condition can be configured in such a manner as including a first condition in which the fiber bundles pass through at least one of two VOIs specified in the VOI group, a second condition in which the fiber bundles pass through both the two VOIs specified in the VOI group, and a third condition in which the fiber bundles do not pass through one VOI specified in the VOI group. The fiber bundle number discriminating means is capable of discriminating the fiber bundles satisfying the discrimination condition and count the number thereof as the following manner, when each of the above conditions is set; when the first condition is set, the fiber bundles having at least one coordinate point of the group of coordinate points that is included in at least either one of the two VOIs are discriminated and the number of which is counted; when the second condition is set, the fiber bundles having at least any of coordinate points of the group of coordinate points that are included in at least either one of the two VOIs and any other coordinate points of the same group of coordinate points being included in the other VOI are discriminated and the number of which is counted; and when the third condition is set, the fiber bundles whose coordinate points are included in the VOI being specified are discriminated and excluded from a target of counting.
The aforementioned VOI selecting means is capable of setting a combination condition of two or more of the first condition, the second condition, and the third condition, as to the VOI group including at least three VOIs. Accordingly, it is possible to discriminate the fiber bundles passing through a predetermined neural path.
When the VOI selecting means sets at least two VOIs or VOI groups, it is possible to compare the numbers of the fiber bundles counted with respect to each of the VOI and the VOI groups. Accordingly, the connectivity between at least two VOI groups can be compared.
When the numbers of fiber bundles are compared as to the VOIs or the VOI groups, a predetermined evaluation function value or a statistical analytic value is calculated for the comparison. As the evaluation function value, it is possible to employ a ratio between the number of fiber bundles being counted and the number of fiber bundles included in a predetermined background area. An area obtained by adding multiple VOIs can be used as the predetermined background area. A result of the comparison may be displayed on a display.
The aforementioned VOI selecting means is capable of arranging VOIs having various sizes respectively at multiple different positions, and allowing the fiber bundle number discriminating means to count each of the numbers of the fiber bundles respectively included in the VOIs of various sizes, and display the result on the display. Accordingly, by referring to the result being displayed, the operator is capable of determine a VOI having appropriate size. On this occasion, it is further possible that the VOI selecting means executes a predetermined computation as to the number of fiber bundles in the VOIs of various sizes, whereby an optimum range of the VOI size is obtained, and the obtained optimum range is displayed on a display.
According to the second aspect of the present invention, an image processing system as the following is provided. In other words, this image processing system includes a memory means for storing nuclear magnetic resonance signal data previously acquired from a subject, a calculating means for calculating diffusion tensor from the nuclear magnetic resonance signal data previously acquired, a fiber bundle extracting means for extracting fiber bundles as a group of coordinate points for each of the fiber bundles included in a target range for receiving a nuclear magnetic resonance signal from the subject, based on the diffusion tensor calculated by the calculating means, a VOI selecting means for setting at least one VOI as to the target range for receiving the nuclear magnetic resonance signal, and a fiber bundle number discriminating means for discriminating the fiber bundles at least one coordinate point of the group of coordinate points of which is included in the VOI, from multiple fiber bundles extracted by the fiber bundle extracting means, and for counting the number of the fiber bundles. Accordingly, it is possible to provide the image processing system that is capable of counting the number of fiber bundles, based on the nuclear magnetic resonance signal data acquired from a separate device. As for the operations of each of the elements above may be the same as those of the first embodiment.
According to the third aspect of the present invention, a program as the following is provided. In other words, it is an image processing program that allows a computer to function as a calculating means for calculating diffusion tensor from the nuclear magnetic resonance signal data previously acquired from a subject, a fiber bundle extracting means for extracting fiber bundles as a group of coordinate points for each of the fiber bundles included in a target range for receiving a nuclear magnetic resonance signal from the subject, based on the diffusion tensor calculated by the calculating means, a VOI selecting means for setting at least one VOI as to the target range for receiving the nuclear magnetic resonance signal, and a fiber bundle number discriminating means for discriminating the fiber bundles at least one coordinate point of the group of coordinate points of which is included in the VOI, from multiple fiber bundles extracted by the fiber bundle extracting means, and for counting the number of the fiber bundles. By allowing the computer to execute this program, an image processing similar to the second aspect of the invention can be implemented, and thereby the number of fiber bundles can be quantitatively figured out.
According to the measurement system and the image processing system of the present invention, the following effects can be achieved:
Hereinafter, preferred embodiments of the present invention will be explained with reference to the accompanying drawings. It is to be noted these embodiments here described will not restrict the scope of the present invention.
The measurement system of the present invention as a first embodiment will be explained. This measurement system includes a nuclear magnetic resonance imager 116 and an image processing means 125, and the configuration incorporating these elements is shown in
The nuclear magnetic resonance imager 116 is provided with a coil 101 for generating a static magnetic field in an imaging space in which a subject 103 is placed, a coil 102 for generating a gradient magnetic field, an RF transmitter 109 for transmitting an RF magnetic field to the subject 103, and the receiver 110 for receiving the nuclear magnetic resonance signal generated from the subject 103.
A gradient magnetic field power supply 105 is connected to the gradient magnetic field coil 102. An RF amplifier 108, an RF modulator 107, and an RF oscillator 106 are connected to the RF transmitter 109. A high frequency wave generated by the RF oscillator 106 is modulated to a predetermined frequency by the RF modulator 107, and amplified by the RF amplifier 108. Thereafter, it is supplied to the RF transmitter 109 and a given RF magnetic field is irradiated to the subject 103. An amplifier 111, a phase detector 112, and an A/D converter 113 are connected to the receiver 110. A nuclear magnetic resonance signal generated from the subject is converted into an electrical signal by the receiver 110, and thereafter, amplified by the amplifier 111, and detected by the phase detector 112. Then, the signal is converted into a digital signal by the A/D converter 113.
The nuclear magnetic resonance imager 116 is provided with a CPU 114, a sequencer 104, and a memory 115. The sequencer 104 activates the gradient magnetic field power supply 105 and the RF oscillator 106 at a specified timing and allows the A/D converter 113 and the phase detector 112 to perform detection, whereby a predetermined imaging pulse sequence directed by the CPU 114 is executed. In this example here, neural fiber bundles are extracted by using diffusion tensor in the image processing means 125. Therefore, an imaging pulse sequence for acquiring a diffusion-weighted image is executed.
The CPU 114 receives an output signal from the A/D converter 113, and performs a signal processing such as an image reconstruction, and thereby, a desired reconstructed image of the subject 103, such as a nuclear magnetic resonance image (e.g., a tomographic image) is obtained according to the nuclear magnetic resonance signal. The nuclear magnetic resonance signal and the nuclear magnetic resonance image are stored in the memory 115, as necessary.
On the other hand, the image processing means 125 is provided with a calculator 117, a fiber bundle extracting means 118, a fiber bundle number discriminating means 120, a comparator 121, a display 122, and a VOI selecting means 119.
The calculator 117 calculates the diffusion tensor from the nuclear magnetic resonance image obtained by a series of measurements, and further by diagonalizing the diffusion tensor, an eigenvalue and an eigenvector are calculated. The fiber bundle extracting means 118 receives from an operator, a specification of a seed area within a target range from where the nuclear magnetic resonance image of the subject is acquired. Then, using one voxel within the seed area as an origin, voxels positioned in the direction where the diffusion coefficient is the largest according to the information of the eigenvector, i.e., in the direction of a principal vector of the fiber bundle, are sequentially traced in a nuclear magnetic resonance image. Therefore, a series of voxels (i.e., a group of coordinate points) constituting one fiber bundle can be extracted. The operation above is performed by setting as an origin, each of all the voxels or multiple voxels selected according to a certain condition within the seed area, whereby all the fiber bundles passing through the seed area can be extracted, as a group of coordinate points with respect to each of the fiber bundles. The fiber bundle extracting means 118 records a series of coordinate points being extracted, in the built-in memory.
The VOI selecting means 119 sets at least one VOI specified by the operator, within the range for acquiring the nuclear magnetic resonance image of the subject 103. In the fiber bundle number discriminating means 120, the group of coordinate points constituting one fiber bundle is compared with the coordinate range of the VOI. Then, it is determined whether or not at least one coordinate of the coordinate point group is included in the coordinate range of the VOI. If it is included, this one fiber bundle is determined as a fiber bundle that passes through the VOI. This determination is made with respect to all the fiber bundles recorded by the fiber bundle extracting means 118. After the determinations are made as to all the fiber bundles, the number of the fiber bundles included in the VOI is counted. If multiple VOIs are set in the VOI selecting means 119, the fiber bundles included in these VOIs are discriminated with respect to each VOI, and the number of fiber bundles is counted. In the comparator 121, if multiple VOIs are set in the VOI selecting means 119, the numbers of fiber bundles included in each of those multiple VOIs are compared as needed, and a result of the comparison is displayed on the display 122. Further on the display 122, an image of the fiber bundles is generated by representing a series of voxels (a group of coordinate points) constituting the fiber bundles, in a given color, and it is displayed with the nuclear magnetic resonance image, and the like, in superimposed manner one on another.
It is to be noted that the image reconstruction can be executed by the calculator 117 of the image processor 125, instead of executing the image reconstruction by the CPU 114 of the nuclear magnetic resonance imager 116. In the case above, the image reconstruction is not performed in the CPU 114, and the nuclear magnetic resonance signal is stored in the memory means 115. The nuclear magnetic resonance signal being stored is transferred to the calculator 117, and the image reconstruction is performed in the calculator 117.
Next, as an imaging pulse sequence for acquiring a diffusion-weighted image, a pulse sequence according to a diffusion-weighted echo-planar will be explained with reference to
Before and after the application of inverse RF pulses 301 and 303, a gradient magnetic field (motion probing gradient, MPG) pulse for detecting a spin motion is applied, in a slice direction, a readout direction, and a phase encoding direction. Pulse 309 represents a first MPG in the slice direction, pulse 310 represents a second MPG in the slice direction, pulse 311 represents a first MPG in the readout direction, pulse 312 is a second MPG in the readout direction, pulse 313 represents a first MPG in the phase encoding direction, and pulse 314 represents a second MPG in the phase encoding direction. The first MPG and the second MPG in each of the directions have the same gradient waveform.
In order to obtain a diffusion coefficient within the subject according to the pulse sequence shown in
It is to be noted that in the pulse sequence as shown in
Here, a principle of a method will be briefly explained, so as to obtain a diffusion coefficient from the nuclear magnetic resonance image. Signal attenuation according to the MPG is exponential as shown in the formula 1.
S(b)/S(0)=exp(−bD) [FORMULA 1]
In this formula, S(b) represents signal intensity at the time of applying the MPG, S(0) represents signal intensity without the MPG, and D represents a diffusion coefficient. Here, b represents b-factor representing a degree of the signal attenuation according to the MPG, and it is obtained by the formula 2. Here, TE represents an echo time, γ represents a gyromagnetic ratio, and G(τ) represents gradient magnetic field intensity.
In the diffusion anisotropic measurement, the diffusion coefficient D represents a tensor quantity as shown in the formula 3, and it is referred to as diffusion tensor. Since the number of independent elements of the tensor is six, in order to obtain the diffusion tensor, at least six nuclear magnetic resonance images using a combination of different MPGs and a nuclear magnetic resonance image obtained without applying the MPGs are necessary as to an identical slice. Each element of the diffusion tensor functions as a component of the diffusion coefficients being different by direction.
The coordinate system based on a principal axis of the diffusion tensor is different from the observational coordinate system based on an imaging axis of MRI. Therefore, it is necessary to conduct a coordinate conversion. Therefore, the diffusion tensor obtained from the above measured value is diagonalized and eigenvalues λ1, λ2, and λ3 are calculated. Then, the diffusion tensor having been diagonalized is obtained (formula 4).
A coordinate system after the conversion is expressed by the eigenvectors μ1, μ2, and μ3, respectively associated with their eigenvalues. Since the direction of the fiber bundle corresponds to the direction in which water molecules are the most active, the direction of the eigenvector (principal vector), where the eigenvalue becomes the maximum, agrees with the direction of the fiber bundle.
As an index value indicating the level of the diffusion anisotropy, the fractional anisotropy (FA value) expressed by the formula 5 is utilized. The FA value is an index representing a deviance from the isotropic diffusion, and if it is completely isotropic, the index becomes zero. As the anisotropy is increased, the index is approaching 1 (one). In the area where fibers exist, such as a white matter within a brain, the diffusing direction of the water molecules within a tissue is restricted by the fibers. Therefore, the FA value becomes larger. On the other hand, in the area where no fibers exist, such as a gray matter, the FA value becomes smaller.
In the measurement system according to the present embodiment, the fiber bundles are extracted by using the eigenvector and FA value obtained according to the formulas described above. Hereinafter, by using the flow as shown in
Firstly, the nuclear magnetic resonance imager 116 executes the pulse sequence as shown in
The calculator 117 calculates the diffusion tensor based on the formula 3 with respect to each voxel, from at least seven nuclear magnetic resonance images as to each slice (step 402). Subsequently, the diffusion tensor obtained for each voxel is diagonalized, and simultaneously eigenvalues and eigenvectors are calculated (step 403). The eigenvalues obtained here represent diffusion coefficients in three orthogonal directions. The directions of the eigenvectors respectively agree with the three orthogonal directions, and one of the directions indicates a direction of the fiber bundle. Since the direction of the fiber bundle corresponds to the direction in which water molecules are the most active, the direction of the eigenvector (principal vector), in which the eigenvalue becomes the maximum, agrees with the direction of the fiber bundle. The calculator 117 calculates the FA value from the eigenvalues as necessary (step 404). An appropriate threshold value is set, and by extracting an area where the FA value exceeds this threshold value, an area where the fiber bundles exist can be extracted. Here, it is assumed that the threshold value is set to be 0.6, by way of example.
Next, the fiber bundle extracting means 118 accepts from an operator, any specification of a seed area (step 405). As a method for specifying the seed area, for example, the operator selects any area by using a pointing device, or the like on the nuclear magnetic resonance image. Alternatively, if the spatial transformation to the standard brain coordinate has been conducted, a coordinate value or a combination of coordinate values of the standard brain coordinate system may be specified based on a priori knowledge. In the case where the measurement is performed targeting a brain, the operator may select a brain activated area that is obtained by a brain functional measuring method such as fMRI (functional magnetic resonance imaging). An alternative configuration is possible such as extracting an area by a threshold process using the FA value obtained in step 404, and selecting this area. Furthermore, the operator may select any area using a pointing device, from the multiple areas being extracted, or select from the extracted area, a coordinate value or a combination of coordinate values of the standard brain coordinate system based on a priori knowledge.
Next, the fiber bundle extracting means 118 extracts a fiber bundle, setting one voxel as a starting point for calculation among the voxels included in the seed area (step 406). In other words, in the case where the FA value at the origin (the FA value obtained in step 404) exceeds a given threshold value, and an adjacent voxel in the direction of the principal vector of the origin (i.e., the eigenvector obtained in step 403) exists within the image data space, the adjacent voxel in the direction of the principal vector is assumed as a coordinate point (voxel) along the fiber bundle and this coordinate point is traced. With regard to this adjacent voxel, when the FA value exceeds a given threshold value and another adjacent voxel in the principal vector direction exists in the image data space, this second adjacent voxel in the principal vector direction is assumed as a coordinate point (voxel) along the fiber bundle, and this second adjacent voxel is traced. The operation above is repeated until reaching a voxel where the FA value becomes equal to or less than the given threshold value, or until the adjacent voxel in the principal vector direction goes out of the image data space. A group of coordinate points obtained as a result of the tracing is recorded in the built-in memory (step 407). The operations in steps 406 and 407 are performed for all the voxels included in the seed area. Accordingly, all the fiber bundles passing through the seed area can be extracted.
Next, the VOI selecting means 119 accepts from the operator a setting of VOI (step 408). A shape of the VOI may be a sphere, a rectangular solid, or the like. It is also possible to set a “VOI group” that incorporates multiple VOIs. The VOI selecting means 119 also accepts from the operator a setting of the discriminating conditions as the following, under which the fiber bundle number discriminating means 120 discriminates the fiber bundles; the condition (AND) that the fiber bundles pass through both two VOIs, the condition (OR) that the fiber bundles pass through either one of the two VOIs, and the condition (NOT) that the fiber bundles passing through the VOI is excluded.
An operation in step 408 of the VOI selecting means 119 will be specifically explained with reference to the flowchart in
Next, in each of the operation screens 706, the operator sets the number M of VOIs, which is at least one, to be included in the VOI group (VOI group A in the example of
When the operator specifies a spherical area or a cubic area by using a pointing device or the like, or specifies any area in the Brodmann's map, “APPLY” button 604 arranged in the operation screen 602 is pressed (i.e., selected by the pointing device, or the like), whereby the VOI 702 being specified is stored in the memory means within the VOI selecting means 119 (step 1310 and step 1311). By pressing “SAVE” button 606, a group of coordinates constituting the VOI can be stored in the memory means of the VOI selecting means 119.
When a spherical area, a cubic area, or the like, is selected via the aforementioned “SPHERE (Text Entry)” or “CUBE (Text Entry)”, or the selection is made by using the aforementioned coordinate file, the “APPLY” button 604 is pressed, and then, a screen for inputting a text or a screen for selecting a coordinate file is displayed.
In the operation screen 602, when the “END” button 607 is pressed, the operation screen 602 for the VOI (1) is ended. As shown in
As shown in
When the operator inputs a logical formula into the logical formula input part 712 as appropriate, and selects “END” button 709, the VOI selecting means 119 stores the logical formula 712 in the built-in memory means (step 1313 and step 1314). The actions as described above are performed as to all the VOI groups (step 1315, step 1302, step 1303, and step 1304). The image display part 701 displays the VOIs. As for the display color of the VOIs, different colors may be respectively assigned to the VOI groups, thereby enabling an easy recognition of the VOI groups on the image.
In addition to the nuclear magnetic resonance image and Brodmann's map, as an image to be displayed on the image display part 701, it is possible to use another image such as an anatomical image obtained by X-ray CT (X-ray computed tomography) and the like, a brain functional image obtained by fMRI, PET (positron emission tomography), an electroencephalography, a magnetoencephalography, an optical measurement instrument for living body, SPECT (single photon emission computed tomography), or the like, and a standard brain model such as MNI and Talairach.
As the operation screen 602 for selecting the VOI, it is possible to have different operation screens for each of the VOIs 702 and 703 as shown in
Next, the operation proceeds with step 409 in
The fiber bundle number discriminating means 120 discriminates the fiber bundles and counts the number thereof with respect to each VOI group. Firstly, the fiber bundle number discriminating means 120 receives from the fiber bundle extracting means 118, the number L of fiber bundles being extracted, simultaneously receives from the VOI selecting means 119, the number M of VOIs included in the VOI group A (here, M=4) (step 1401 and step 1405), and determines whether or not at least one of the coordinate points that define the first fiber bundle (l=1) is included in the VOI (1) (step 1409). If it is included, it is determined that the first fiber bundle is included in the VOI (1), and the number “1” is assigned to R(1) (R(1)=1) (step 1411). If it is not included, R(1) is set to zero (step 1410). Next, it is determined whether or not this first fiber bundle is included in the VOI (2) (step 1412, 1409). If it is included, R(2) is set to one, and if it is not included R(2) is set to zero (step 1411 and step 1410). The operation above is repeated as to all the VOIs (1) to (4) in the VOI group A, and R(1) to R(4) are obtained (step 1406 and step 1407). Thereafter, the operation proceeds with step 1415 in
In step 1415, the logical formula is read, which is input in the operation screen 706 of the VOI group A, from the memory means of the VOI selecting means 119, the logical formula being RA=(R(1)×R(2)+R(3)−R(4)) in
Next, the operation returns to step 1413 in
In step 1416, if the number of the VOI set in the VOI group is one and there is no logical formula being input, RA is set to R(1), whereby the number of fiber bundles passing through this one VOI is counted.
Subsequently, in the comparator 121, the fiber bundle numbers being discriminated by the fiber bundle number discriminating means 120 are compared, and the comparison result is displayed in the display 122 (step 410 and step 411 in
Specifically, the comparator 121 performs the processing as the flow shown in
Next, the comparator 121 creates a display 1701 on the display 122 as shown in
If the comparing means accepted in step 1602 is the “the number of fiber bundles”, the comparator 121 reads from the fiber bundle number discriminating means 120, each number of fiber bundles in at least two VOI groups (VOIs) being selected (step 1603). If the numerical value table is selected as the display method, a table showing a relationship between the VOI group (VOI) and the number of fiber bundles is displayed as the numerical table 901 shown in
According to the example as shown in
If the “color-coded display” is selected, the numbers of fiber bundles are assigned to the colors of fiber bundles displayed in the image, just like the color-coded display as shown in
If the comparing means accepted in step 1602 in
If the comparing means accepted in step 1602 is the “statistical analysis” (step 1605), a setting of parameters used in the statistical analysis is accepted from the operator (step 1608), and the statistic analysis is conducted. For example, a result of the statistical analysis is displayed, such as p-value (significance probability) and F-value of the analysis of variance, ANOVA. Specifically, the numbers of fiber bundles included in multiple VOIs or in multiple VOI groups are obtained from multiple subjects. Then, the analysis of variance is conducted as to three factors, “VOI (VOI group)”, “subject”, and “interaction therebetween”. Then, p-value, F-value, and the like are calculated, indicating whether or not a difference being statistically significant exists. If there is a statistically significant difference, multiple comparison or the like is made, to find out a combination of factors that shows an actual difference.
The calculated statistical value is displayed in the display method selected by the operator, such as the numerical value table 902 shown in
Here, the evaluation function used in step 1606 and step 1607 will be specifically explained. As the evaluation function, a relative evaluation function can be employed, which is calculated from a ratio between the number of fiber bundles included in the VOI or in the VOI group, and the number of fiber bundles included in another VOI or in another VOI group. An example of the evaluation function will be explained with reference to
As an example of the evaluation function, S1 can be used, which is obtained by dividing Rall by the mean value of R1 and R2, as shown in the formula 6. In the formula 6 to formula 8, N represents the number of the VOIs, and N is set to two in the present embodiment.
In another example of the evaluation function, Rall is divided by the minimum value either R1 or R2, and obtained S2 can be employed.
As further alternative example of the evaluation function, S3 can be used, which is obtained by calculating a mean value of the ratio between Rall and R1, and the ratio between Rall and R2.
Here, the explanation has been made in the case where N=2, however, the present invention is not limited thereto and a similar evaluation function can be used in the case where N=3 or more.
By using the evaluation function as described above, it is possible to evaluate connectivity, which cannot be evaluated properly if the numbers of the fiber bundles are directly compared. For example, when the numbers of neural fiber bundles connecting particular areas in a white matter are compared between individuals, it is anticipated that a proper evaluation result cannot be obtained by the direct comparison of the neural fiber bundles, because there is an individual difference in the degree of brain nerve development. However, by introducing a relative evaluation function, which is calculated from the number of fiber bundles, and by comparing the values of the evaluation function, instead of the number of the fiber bundles directly, thereby enabling a proper evaluation excluding influence from the individual difference. By way of example, in the case of the white matter, a relative evaluation function calculated from the ratio between the number of fiber bundles included in the “VOI or VOI group”, and the number of fiber bundles included in “another VOI or another VOI group” can be employed. An example of the above “another VOI” may be the whole white matter, or the whole white matter excluding “the VOI or the VOI group”, for instance.
As described above, according to the measurement system of the present embodiment, the fiber bundles passing through the VOI set by the operator, or the fiber bundles passing through multiple VOIs under the condition represented by the logical formula, are discriminated and the number thereof can be counted. The fiber bundles being counted can be compared using the number of fiber bundles, the evaluation function, or the statistical analysis, and then displayed. Accordingly, it is possible to make a quantitative comparison as to the fiber bundles passing through any VOI, as well as a quantitative comparison as to the connectivity of the fiber bundles which are connecting any VOIs.
A clinical application example of the measurement system of the present embodiment will be explained. Here, an explanation will be made as to an example where a part of pyramidal tract is damaged due to cerebral infarction or the like, and connectivity of the fiber bundles is checked against a patient who is suffering from dyskinesia, so that an optimum rehabilitation can be selected.
The pyramidal tract is a neural path that delivers a motor command from the cerebral cortex of the motor area, via the capsula interna, brainstem, spinal cord, and peripheral nerve, up to a muscle periphery, and the pyramidal tracts on the left and right sides crosses each other at the medulla oblongata of the brainstem part. Therefore, it is known that a lesion on the part upper than the medulla oblongata may cause a disorder of upper and lower extremities on the opposite side, and a lesion on the part lower than the medulla oblongata may cause such disorder on the same side. When a patient who is suffering from dyskinesia due to damage on a part of the pyramidal tract, caused by the cerebral infarction or the like, and undergoing rehabilitation, it is expected that the brain activity against an exercise load is changed, at the stage where the motor function is gradually recovered by the rehabilitation. Therefore, if the change of the brain activity is monitored and its result can be given as a feedback, it is expected that the patient is encouraged and a more efficient rehabilitation effect can be achieved.
However, depending on the damaged position and its degree of damage, a portion where the brain activity is to be monitored may be different. In other words, if it is damaged to a slight degree, the patient may recover from dyskinesia, when the brain function of the lesion portion recovers. However, if the pyramidal tract is severely damaged, it is not possible to recover the brain function of the lesion portion, and this may give rise to the phenomenon that a part different from the portion originally supposed to act, substitutes for the brain function being damaged. Accordingly, there is a possibility that a process of recovering of the brain function may be different, and also a menu for an optimum rehabilitation may be different.
Given the situation above, by using the measurement system of the present embodiment, it is measured at which position of the pyramidal tract and to what degree it is damaged, whereby it helps specifying in which part the brain activity is to be monitored, enabling a customization of an optimum rehabilitation method. The situation above will be specifically explained, with reference to
Firstly, according to step 401 to 407 in
Subsequently, in the fiber bundle number discriminating means 120, the number of the fiber bundles included in the VOI groups is obtained. According to a directive from the operator, a value of relative evaluation function is calculated based on the number of fiber bundles. In the comparator 121, based on the number of the fiber bundles or the value of the relative evaluation function, a difference of connectivity between the left and right pyramidal tracts is calculated. The comparator 121 displays, in the display 122, values of the evaluation function that indicates the connectivity between the left and right pyramidal tracts, according to the numerical table 1008.
The comparator 121 uses the predetermined threshold values A and B (A<B) to make a judgment as the following; when the difference in connectivity between the left and right pyramidal tracts 1002 and 1003 is smaller than the threshold value A, the damage is mild, when it is equal to or larger than the threshold value A and smaller than the threshold B, the damage is moderate, and when it is equal to or larger than the threshold value B, the damage is serious. According to the degree of the damage, a predetermined rehabilitation menu is assigned, and it is displayed in the display 122 in the form of table 1009. Here, it is to be noted that without using the threshold value, the degree of the damage may be assigned to sequential values according to the value of the connectivity. It is further possible that in response to the result such as the connectivity value and the difference in connectivity, an optimum portion for monitoring the brain activity, being previously determined, is assigned and this portion to be monitored is displayed for the operator.
As thus described, by using the measurement system according to the present embodiment, it is possible to make a quantitative comparison as to the number of fiber bundles (connectivity) passing through given neural paths (e.g., pyramidal tract), and therefore, a damaged position in the brain can be figured out and a suitable rehabilitation menu can be suggested.
The first embodiment of the present invention is directed to a configuration where the operator inputs a logical formula form, representing under which condition the fiber bundles to be discriminated and counted, are passing through multiple VOIs included in the VOI group. However, the present invention is not limited to the configuration where the logical formula is input. Another configuration is possible, for instance, in the operation screen 706 in
As a second embodiment, a measurement system will be explained, which is provided with a function to set the VOI to an optimum size. In the configuration of the measurement system according to the second embodiment, a function for setting the VOI with an optimum size is added to the VOI selecting means 119, but the configuration and operations are the same as the measurement system of the first embodiment. Therefore, hereinafter, among the operations of the VOI selecting means 119 and the fiber bundle number discriminating means 120, only a part different from the first embodiment will be explained, and the operations and the configuration being the same as the first embodiment will not be tediously explained.
It is anticipated that the size of the VOI may influence the number of fiber bundles, a value of relative evaluation function, and their comparison result. Therefore, it is desirable to select a size of the VOI that hardly influences these values. In the present embodiment, as to the VOI of various sizes (volumes), the number of fiber bundles and the value of the evaluation function are calculated, and these results or comparison results are displayed, thereby allowing the operator to select a size of the VOI that is optimum. Here, with reference to
In step 1308 of
In the minimum radius value input part 1102, a minimum radius value of the VOI is input, among multiple VOI volumes desired by the operator. In the maximum radius value input part 1103, a maximum radius value of the VOI is input, among multiple VOI volumes desired by the operator. In the radius step input part 1104, the amount of change of the VOI radius is input. With the inputs as described above, it is possible to determine the coordinates of the center of the VOI and what kind of radius the VOI has.
After the above inputting is completed for the text input screen 1101, the “APPLY” button 614 is pressed and thereby the VOI is set in the VOI selecting means 119. In addition, by pressing the “SAVE” button 615, the combination of the coordinates constituting the VOI, and the minimum and maximum values of radius, and the amount of change thereof are stored in the memory means of the VOI selecting means 119.
As to the VOIs having various types of radius set in the VOI selecting means 119, the fiber bundle number discriminating means 120 discriminates the fiber bundles respectively passing through the VOIs having various types of radius, and counts the number thereof, similar to the case where one VOI is set in the VOI group in the flows shown in
In addition, the VOI selecting means 119 is also capable of providing information to help the judgment, when the operator selects the optimum radius of the 2VOI. By way of example, the VOI selecting means 119 sequentially executes the following computations, thereby obtaining information for helping the judgment;
With the procedure above, the operator is allowed to select an optimum radius, according to the numerical values in the numerical table 1201 displayed on the display 122 and the coloring that represents the optimum range. Alternatively, the operator may choose an automatic selection setting, and the radius which the VOI selecting means 119 has determined as the optimum range can be automatically set without waiting the selection by the operator.
In the case of the automatic setting, the VOI selecting means 119 automatically sets the radius selected by the operator or the radius within the optimum range, as the radius of the VOI, and the fiber bundle number discriminating means 120 and the comparator 121 perform the discrimination and comparison of the number of fiber bundles, and display thereof, similar to the first embodiment (step 409 to step 411 in
As thus described, according to the measurement system of the second embodiment, it is possible to set the VOI to an optimum size, which has little effect upon the number of fiber bundles, a value of the relative evaluation function, and the comparison result thereof. Therefore, the effect due to the size of the VOI is reduced, and more accurate quantitative evaluation is possible as to the number of fiber bundles and their connectivity.
Next, an image processing system according to the third embodiment will be explained, with reference to
Since the configuration of the image processor 125 is the same as the first embodiment, tedious explanation will not be made. The memory means 201 stores a diffusion-weighted image that is separately taken. Similar to the first embodiment, as the diffusion-weighted image, nuclear magnetic resonance images using a combination of at least six different MPGs, and a nuclear magnetic resonance image obtained without applying the MPG are stored.
As indicated by the flow shown in
According to the image processing system of the third embodiment, it is possible to quantitatively figure out the number of fiber bundles in the VOI and the connectivity between the VOIs, by using the diffusion-weighted images that is taken separately, without the nuclear magnetic resonance imager being incorporated.
It is further possible that the image processing system according to the third embodiment is provided with the function to set the VOI to an optimum size, which is explained as the second embodiment.
As the fourth embodiment, a computer that implements the image processing function will be explained. In the first and the third embodiments, the calculator 117, the fiber bundle extracting means 118, the discriminating means for fiber bundles 120, the comparator 121, and the VOI selecting means 119 are separately configured, which are incorporated in the image processing means 125. However, in the fourth embodiment, a CPU of the computer reads and executes programs, thereby performs the same function as those of the calculator 117, the fiber bundle extracting means 118, the fiber bundle number discriminating means 120, the comparator 121, and the VOI selecting means 119.
As shown in
The CPU 181 reads and executes the programs within the program storing means 183, whereby the functions of the calculator 117, the fiber bundle extracting means 118, the fiber bundle number discriminating means 120, the comparator 121, and the VOI selecting means 119 are performed, and these operations are executed. Specifically, the processing in the steps from step 402 to 411 shown in
The hardware configuration of the computer having the image processing function of the fourth embodiment is the same as the configuration of a commercially available personal computer, or the like, and by storing the programs and allowing the computer to execute the programs, the operations similar to those of the image processing system according to the third embodiment can be implemented.
Also in the measurement system according to the first embodiment, the image processor 125 may have the same configuration as the computer of the fourth embodiment.
By using the measurement system and the image processing system according to the first to the fourth embodiments described above, the number of neural fiber bundles within a particular area and the connectivity of the neural fiber bundles between the areas are measured, and thereby the state of the neural fiber bundles can be figured out quantitatively. Therefore, there is a possibility that customization for an optimum rehabilitation method can be implemented. With the configuration above, it is possible to monitor a change of the brain activity of a damaged-brain patient who is a rehabilitant, and to feedback a result of the monitoring, expecting to obtain more efficient rehabilitation effect, even in the case where a degree of recovery of the brain function is different by patient depending on the state of the surviving neural fiber bundles, and an optimum rehabilitation method is different by patient.
By using the measurement system and the image processing system according to the first to the fourth embodiments, the connectivity between particular brain areas can be quantitatively evaluated. Therefore, if a relationship between the connectivity of particular brain areas and a specific ability is figured out in the future brain science research, the present invention may be applied to a development of learning-effect monitoring tool and a support tool for development of educational materials, in order to foster this ability.
In each of the above embodiments, neural fiber bundles of a brain have been taken as an example. However, a target for the measurement and the image processing in the measurement system and the image processing system of the present invention is not restricted to the brain. Measurement and image processing of muscle fibers existing in the muscle are also available. With the application above, the present invention may be useful for monitoring a rehabilitation effect on a patient whose muscle is damaged.
Yamamoto, Yukari, Sakai, Kuniyoshi L.
Patent | Priority | Assignee | Title |
10504229, | Oct 28 2016 | Canon Medical Systems Corporation | Medical image processing apparatus and medical image processing method |
10672126, | Sep 10 2015 | Canon Medical Systems Corporation | Image processing apparatus and magnetic resonance imaging apparatus |
10996303, | Sep 16 2011 | The United States of America, as represented by the Secretary, Department of Health and Human Services | MRI tractography based transit time determination for nerve fibers |
11273283, | Dec 31 2017 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to enhance emotional response |
11318277, | Dec 31 2017 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to enhance emotional response |
11364361, | Apr 20 2018 | Neuroenhancement Lab, LLC | System and method for inducing sleep by transplanting mental states |
11452839, | Sep 14 2018 | Neuroenhancement Lab, LLC | System and method of improving sleep |
11478603, | Dec 31 2017 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to enhance emotional response |
11717686, | Dec 04 2017 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to facilitate learning and performance |
11723579, | Sep 19 2017 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement |
11786694, | May 24 2019 | NeuroLight, Inc. | Device, method, and app for facilitating sleep |
7902825, | May 19 2008 | The Board of Trustees of the Leland Stanford Junior University | Motion corrected tensor magnetic resonance imaging |
7994784, | Jul 20 2007 | AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC | Systems and methods for rescaling image intensities with diffusion MRI to synthesize complex diffusive geometries |
8076937, | Dec 11 2006 | Koninklijke Philips Electronics N V | Fibre tracking on the basis of macroscopic information |
8098068, | Jul 20 2007 | AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC | Systems and methods for parametric mapping for correction of noise-based systematic bias of DTI metrics, using a DTI mapping phantom |
8134363, | Jul 20 2007 | AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC | Test object for use with diffusion MRI and system and method of synthesizing complex diffusive geometries using novel gradient directions |
8169216, | Jul 20 2007 | AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC | System and method for synthesizing crossing ADC distributions via reassembly of multiple k-spaces |
8274283, | Apr 27 2009 | SIEMENS HEALTHINEERS AG | Method and apparatus for diffusion tensor magnetic resonance imaging |
8334695, | Mar 25 2010 | Board of Regents of the University of Texas System | Method and apparatus for improving the quality of MR images sensitized to molecular diffusion |
Patent | Priority | Assignee | Title |
6526305, | Nov 25 1998 | The Johns Hopkins University School of Medicine | Method of fiber reconstruction employing data acquired by magnetic resonance imaging |
6724190, | May 15 2002 | KONINKLIJKE PHILIPS ELECTRONICS N V | Retrospective selection and various types of image alignment to improve DTI SNR |
6806705, | May 15 2002 | Koninklijke Philips Electronics N V | Diffusion tensor magnetic resonance imaging including local weighted interpolation |
7268551, | Apr 14 2004 | McLean Hospital Corporation | Inter-subject coherence in DT-MRI |
7355597, | May 06 2002 | Brown University Research Foundation | Method, apparatus and computer program product for the interactive rendering of multivalued volume data with layered complementary values |
7505806, | Aug 29 2002 | GE Medical Systems Global Technology Company, LLC | Fiber rendering apparatus |
7570049, | Apr 14 2004 | McLean Hospital Corporation | Inter-subject coherence in DT-MRI |
7657071, | Apr 11 2005 | Siemens Medical Solutions USA, Inc | Systems, devices, and methods for bundle segmentation in diffusion tensor magnetic resonance imaging |
20030214289, | |||
20030214290, | |||
20030234781, | |||
20050101857, | |||
20050240097, | |||
20060229856, | |||
20060281987, | |||
20070273373, | |||
20080122440, | |||
20080180097, | |||
20080205733, | |||
20090232374, | |||
20100004527, | |||
JP11000320, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Nov 12 2007 | YAMAMOTO, YUKARI | Hitachi, LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 020215 | /0878 | |
Nov 12 2007 | YAMAMOTO, YUKARI | UNIVERSITY OF TOKYO, THE | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 020215 | /0878 | |
Nov 14 2007 | SAKAI, KUNIYOSHI | Hitachi, LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 020215 | /0878 | |
Nov 14 2007 | SAKAI, KUNIYOSHI | UNIVERSITY OF TOKYO, THE | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 020215 | /0878 | |
Nov 27 2007 | Hitachi, Ltd. | (assignment on the face of the patent) | / | |||
Nov 27 2007 | The University of Tokyo | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Aug 03 2011 | ASPN: Payor Number Assigned. |
Aug 03 2011 | RMPN: Payer Number De-assigned. |
Apr 16 2014 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
May 03 2018 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Jul 04 2022 | REM: Maintenance Fee Reminder Mailed. |
Dec 19 2022 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
Date | Maintenance Schedule |
Nov 16 2013 | 4 years fee payment window open |
May 16 2014 | 6 months grace period start (w surcharge) |
Nov 16 2014 | patent expiry (for year 4) |
Nov 16 2016 | 2 years to revive unintentionally abandoned end. (for year 4) |
Nov 16 2017 | 8 years fee payment window open |
May 16 2018 | 6 months grace period start (w surcharge) |
Nov 16 2018 | patent expiry (for year 8) |
Nov 16 2020 | 2 years to revive unintentionally abandoned end. (for year 8) |
Nov 16 2021 | 12 years fee payment window open |
May 16 2022 | 6 months grace period start (w surcharge) |
Nov 16 2022 | patent expiry (for year 12) |
Nov 16 2024 | 2 years to revive unintentionally abandoned end. (for year 12) |