The invention relates to a detection system for automatic detection of bone cancer metastases from a set of isotope bone scan images of a patients skeleton, the system comprising a shape identifier unit, a hotspot detection unit, a hotspot feature extraction unit, a first artificial neural network unit, a patient feature extraction unit, and a second artificial neural network unit.
|
1. A detection system for automatic detection of bone cancer metastases from a set of isotope bone scan images of a patients skeleton, the system comprising:
a shape identifier unit for identifying anatomical structures of the skeleton pictured in the set of bone scan images, forming an annotated set of images;
a hotspot detection unit for detecting areas of high intensity in the annotated set of images based on information from the shape identifier regarding the anatomical structures corresponding to different portions of the skeleton of the images;
a hotspot feature extraction unit for extracting a set of hotspot features for each hot spot detected by the hotspot detection unit;
a first artificial neural network unit arranged to calculate a likelihood for each hot spot of the hotspot set being a metastasis based on the set of hotspot features extracted by the hotspot feature extraction unit;
a patient feature extraction unit arranged to extract a set of patient features based on the hotspots detected by the hotspot detection unit and on the likelihood outputs from the first artificial neural network unit; and
a second artificial neural network unit arranged to calculate a likelihood that the patient has one or more cancer metastases, based on the set of patient features extracted by the patient feature extraction unit; and
an input image memory,
wherein the shape identifier unit accesses the set of isotope bone scan images from the input image memory and wherein, upon extraction of the set of patient features, the patient feature extraction unit stores the set of patient features in a patient feature memory for accessing by the second artificial neural network to calculate the likelihood that the patient has one or more cancer metastases.
10. A method automatically detecting bone cancer metastases from an isotope bone scan image set of a patient, the method comprising the following steps:
identifying accessing, by a computerized image processing system, an isotope bone scan image set, wherein each image in the isotope bone scan image set comprises a plurality of pixels with a value of each pixel corresponding to an intensity;
automatically segmenting, by the computerized image processing system, each image in the isotope bone scan image set to identify one or more anatomical structures of the a skeleton pictured in the set of each image in the isotope bone scan images image set, thereby forming an annotated set of images;
automatically detecting areas, by the computerized image processing system, a set of hotspots comprising one or more hotspots, each hotspot corresponding to an area of high intensity in the annotated set of images based on information regarding the anatomical structures corresponding to different portions of the skeleton of the images, said detecting of the set of hotspots comprising iteratively:
identifying one or more skeletal image elements, each of the skeletal image elements corresponding to a region of an image in the annotated image set, wherein the region is associated with one of the anatomical structures;
detecting one or more hotspots in the annotated set of images, each hotspot corresponding a region of high intensity relative to its surroundings;
for each of the skeletal image elements, determining whether each of the skeletal image element comprises a detected hotspot;
calculating an average intensity of the skeletal image elements determined not to comprise a detected hotspot;
calculating a normalization factor, wherein a product of the normalization factor and the average intensity is a pre-defined intensity level; and
multiplying the value of each pixel in the annotated set of images by the normalization factor;
for each hotspot in the set of hotspots, extracting, by the computerized image processing system, a set of hotspot features for each hot spot detected associated with the hotspot; and
feeding, to a first artificial neural network unit arranged to calculate for each hotspot in the set of hotspots, calculating, by the computerized image processing system, a first likelihood value corresponding to a likelihood for each hot spot of the hotspot set being a metastasis, based on the set of hotspot features extracted;
extracting a set of patient features based on the hotspots detected and on the likelihood outputs from the first artificial neural network unit; and
feeding, to a second artificial neural network unit arranged to calculate a likelihood that the patient has one or more cancer metastases, the set of patient features extracted associated with the hotspot.
2. The detection system as recited in
3. The detection system as recited in
4. The detection system as recited in
5. The detection system as recited in
6. The detection system as recited in
7. The detection system as recited in
8. The detection system as recited in
9. The detection system as recited in
11. The method of
a value describing the eccentricity of each hotspot;
a value describing the skeletal volume occupied by an extracted hotspot region;
a value describing the maximum intensity calculated from all hotspots on the corresponding normalized image;
a value describing the hotspot localization relative to a corresponding skeletal region;
a value describing distance asymmetry which is only calculated for skeletal regions with a natural corresponding contralateral skeletal region(s); and
a number of hotspots in one or more certain anatomical region(s).
12. The method of
identifying a number of anatomical structures in the bone scan image(s); detecting a set of hotspots comprising one or more hotspots comprises:
detecting hotspots in each anatomical region by comparing the value of each pixel with a threshold value, different for each anatomical region; and
decide deciding, for each hotspot, which anatomical region it belongs to.
13. The method of
for each hotspot: determining the number of pixels having an intensity above a predetermined threshold level.
14. The method of claim 12 wherein the step of identifying a number of anatomical structures in the 10, comprising segmenting each image in the isotope bone scan image(s) further includes the step of segmenting the bone scan image(s) image set by a segmentation-by-registration method.
15. The method of
comparing a each image of the bone scan image set with an a corresponding atlas image of an atlas image set, the each atlas image having anatomical regions marked; and
for each image of the bone scan image set, adjusting a copy of the corresponding atlas image set to the bone scan image set, such that anatomical regions of the atlas image can be superimposed on the bone scan image of the bone scan image set.
16. The method of
a value describing distance asymmetry which is only calculated for skeletal regions with a natural corresponding contralateral skeletal region.
0. 17. The method of claim 10, comprising calculating, for each hotspot in the set of hotspots, the first likelihood value using a pre-trained machine learning technique.
0. 18. The method of claim 17, wherein the pre-trained machine learning technique is an artificial neural network (ANN).
0. 19. The method of claim 10, wherein, for each hotspot in the set of hotspots, the first likelihood value corresponds to an output of a machine learning module that implements a pre-trained machine learning technique, and the output of the machine learning module is based at least in part on one or more of the hotspot features associated with the hotspot.
0. 20. The method of claim 10, wherein for each hotspot in the set of hotspots, calculating the first likelihood value corresponding to a likelihood of the hotspot being a metastasis, based on the set of hotspot features associated with the hotspot comprises:
determining from the one or more identified anatomical structures, an anatomical structure to which the hotspot belongs based on a location of the hotspot,
selecting one of a set of artificial neural networks (ANNs), wherein each ANN in the set of ANNs is associated with a specific identified anatomical structure of the one or more identified anatomical structures, and
calculating the first likelihood value using the selected ANN, wherein the specific identified anatomical structure with which the selected ANN is associated is the anatomical structure to which the hotspot belongs.
0. 21. The method of claim 10, comprising calculating, by the computerized image processing system, a second likelihood value corresponding to an overall likelihood that the patient has one or more metastases based on the calculated first likelihood values.
0. 22. The method of claim 21, comprising:
for each hotspot in the set of hotspots, calculating the first likelihood value using a first artificial neural network (ANN), wherein:
the first likelihood value corresponds to an output of the first ANN, and
the output of the first ANN is based at least in part one or more hotspot features in the set of hotspot features associated with the hotspot; and
calculating the second likelihood value based on an output of a second ANN, wherein:
the output of the second ANN is based at least in part on one or more patient features, and
each of the one or more patient features is based at least in part on one or more of the first likelihood values calculated for each hotspot in the set of hotspots.
|
|||||||||||||||||||||||||||||||
ThisThe present application is a broadening reissue application of U.S. application Ser. No. 13/639,747, filed Jan. 2, 2013, now U.S. Pat. No. 8,855,387, issued Oct. 7, 2014, which is a nationalization under 35 U.S.C. §371 from International Application Serial No. PCT/SE2008/000746, filed Dec. 23, 2008 and published as WO 2009/084995 A1 on Jul. 9, 2009, which claims the priority benefit of U.S. Provisional Application Ser. No. 61/017,192, filed Dec. 28, 2007, the contents of which applications and publication are incorporated herein by reference in their entirety.
The present invention relates to the field of medical imaging and to the field of automated processing and interpretation of medical images. In particular, it relates to automated processing and interpretation of two-dimensional bone scan images produced via isotope imaging.
Interpreting medical images originating from different types of medical scans is a difficult, error prone, and time consuming work which often involves several manual steps. This is especially true when trying to determining contours of a human skeleton and cancer metastases in a medical scan image.
Therefore, there is a great need for a method for determining contours of a human skeleton and any cancer metastases, and being capable of extracting features for an automatic interpretation system.
With the above and following description in mind, then, an aspect of the present invention is to provide a method for determining contours of a human skeleton and being capable of extracting features for an automatic interpretation system, which seeks to mitigate or eliminate one or more of the above-identified deficiencies in the art and disadvantages singly or in any combination.
The object of the present invention is to provide a system and a method for fully automatic interpretation of bone scan images.
It is a further object to provide a method for reducing the need for manual work and to create an atlas image fully comparable with a normal reference image of the human skeleton. It is also an object of the present invention to provide a method for creating such a normal image.
An aspect of the present invention relates to a detection system for automatic detection of bone cancer metastases from a set of isotope bone scan images of a patients skeleton, the system comprising a shape identifier unit for identifying anatomical structures of the skeleton pictured in the set of bone scan images, forming an annotated set of images, a hotspot detection unit for detecting areas of high intensity in the annotated set of images based on information from the shape identifier regarding the anatomical structures corresponding to different portions of the skeleton of the images, a hotspot feature extraction unit for extracting a set of hotspot features for each hot spot detected by the hotspot detection unit ,a first artificial neural network unit arranged to calculate a likelihood for each hot spot of the hotspot set being a metastasis based on the set of hotspot features extracted by the hotspot feature extraction unit, a patient feature extraction unit arranged to extract a set of patient features based on the hotspots detected by the hotspot detection unit and on the likelihood outputs from the first artificial neural network unit, and a second artificial neural network unit arranged to calculate a likelihood that the patient has one or more cancer metastases, based on the set of patient features extracted by the patient feature extraction unit.
The detection system may also comprise a shape identifier unit comprising a predefined skeleton model of a skeleton, the skeleton model comprising one or more anatomical regions, each region representing an anatomical portion of a general skeleton.
The detection system may also comprise a predefined skeleton model adjusted to match the skeleton of the set of bone scan images of the patient, forming a working skeleton model.
The detection system may also comprise a hotspot detection unit comprising a threshold scanner unit for scanning the set of bone scan images and identifying pixels above a certain threshold level.
The detection system may also comprise a hotspot detection unit comprising different threshold levels for the different anatomical regions that are defined by the shape identifier unit.
The detection system may also comprise a hotspot feature extraction unit for extracting one or more hotspot features for each hot spot, comprises means for determining the shape and position of each hotspot.
The detection system may also comprise a first artificial neural network unit arranged to be fed with the features of each hotspot of the hotspot set produced by the hotspot feature extraction unit.
The detection system may also comprise a patient feature extraction unit provided with means to perform calculations that make use of both data from the hotspot feature extraction unit and of the outputs of the first artificial neural network unit.
The detection system may also comprise a second artificial neural network unit arranged to calculate the likelihood for the patient having one or more cancer metastases, and wherein the unit is fed with the features produced by the patient feature extraction unit.
A second aspect of the present invention relates to a method for automatically detecting bone cancer metastases from an isotope bone scan image set of a patient, the method comprising the following steps of extracting knowledge information from bone scan image set, processing extracted information to detect bone cancer metastases, wherein the processing involves the use of artificial neural networks.
The step of processing extracted information to detect bone cancer metastases may further involve feeding, to a pretrained artificial neural network, at least one of the following, a value describing the skeletal volume occupied by an extracted hotspot region, a value describing the maximum intensity calculated from all hotspots on the corresponding normalized image, a value describing the eccentricity of each hotspot, value describing the hotspot localization relative to a corresponding skeletal region, a value describing distance asymmetry which is only calculated for skeletal regions with a natural corresponding contralateral skeletal region, and a number of hotspots in one or more certain anatomical region(s).
The step of extracting information may further involve the steps of, identifying a number of anatomical structures in the bone scan image(s), detecting hotspots in each anatomical region by comparing the value of each pixel with a threshold value, different for each anatomical region, and decide, for each hotspot, which anatomical region it belongs to.
The method may further comprise the step of, for each hotspot, determining the number of pixels having an intensity above a predetermined threshold level.
The step of identifying a number of anatomical structures in the bone scan image(s) may further include the step of segmenting the bone scan image(s) by a segmentation-by-registration method.
The segmentation-by-registration method may further comprise the steps of, comparing a bone scan image set with an atlas image set, the atlas image having anatomical regions marked, adjusting a copy of the atlas image set to the bone scan image set, such that anatomical regions of the atlas image can be superimposed on the bone scan image.
A third aspect of the present invention relates to a method for creating a skeleton shape model, the method comprising the steps of providing images of a number of healthy reference skeletons, reorienting said images into a common coordinate system, using at least two landmark points corresponding to anatomical landmarks of the skeleton, making a statistical analysis of said images, and based on the statistical analysis, segmenting a skeleton shape model.
A fourth aspect of the present invention relates to a method for automatic interpretation of a two dimensional medicine image set representing a body organ where said method comprises the steps of automatically rotating the image set to adjust for accidental tilting when the images was originally taken, automatically finding the contours of the organ, automatically adjusting size, position, rotation, and shape of a predefined model shape of the type of organ in question to fit the organ of the current image, automatically, with the aid of the model shape, defining certain portions of the image, normalizing, the intensity of the image, quantifying each pixel in the image of the organ, producing a quantification result, feeding the quantification results to an interpretation system, letting the interpretation system interpret the image, producing an interpretation result, and presenting the interpretation result to a user.
The method according to the fourth aspect where the organ is the skeleton and said normalization is performed by assigning, to a certain area of the skeleton, a certain reference value.
A fifth aspect of the present invention relates to an image classification system for labeling an image into one of two or more classes where one class is normal and one class is pathological, the system comprising a pretrained artificial neural network having a plurality of inputs nodes, and a number of output nodes, a feature extractor, capable of extracting a number of features from said image, said features being suitable for feeding to the input nodes, wherein the pretrained artificial network presents a classification result on the number of output nodes when the number of features of the image is fed to the plurality of input nodes.
The classification system according to the fifth aspect wherein the image is a two dimensional skeleton image.
The classification system according to the fifth aspect wherein said number of features comprises a total number of pixels inside a contour of a skeleton of said skeleton image.
The classification system according to the fifth aspect wherein said number of features comprises number of pixels in largest cluster of pixels above a certain threshold level inside a contour of a skeleton of said skeleton image.
A sixth aspect of the present invention relates to a method for automatic normalization of bone scan images comprises the steps of, identifying image elements corresponding to the skeleton, identifying hotspot elements contained in the image, subtracting the hotspot elements from the skeleton elements, creating an image having remaining elements, calculating an average intensity of the remaining elements, calculating a suitable normalization factor, adjusting the bone scan image intensities by multiplication with the normalization factor.
The method according to the sixth aspect may also comprise the repetition of the steps of identifying hotspot elements contained in the image, subtracting the hotspot elements from the skeleton elements, creating an image having remaining elements, calculating an average intensity of the remaining elements, calculating a suitable normalization factor, adjusting the bone scan image intensities by multiplication with the normalization factor, which are repeated until no further significant change in the normalization factor occurs.
Any of the first, second, third, fourth, fifth, or sixth aspects presented above of the present invention may be combined in any way possible.
Further objects, features, and advantages of the present invention will appear from the following detailed description of some embodiments of the invention, wherein some embodiments of the invention will be described in more detail with reference to the accompanying drawings, in which:
Embodiments of the present invention relate, in general, to the field of medical imaging and to the field of automated processing and interpretation of medical images. A preferred embodiment relates to a method for automatically or semi-automatically determining contours of a human skeleton and any cancer metastases contained therein and being capable of extracting features to be used by an automatic interpretation system
An image is a digital representation wherein each pixel represents a radiation intensity, a so called “count”, as known in the art, coming from a radio active substance injected into the human body prior to taking of the image.
Embodiments of the present invention will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like reference signs refer to like elements throughout.
A hotspot detection unit 120 is connected to the annotated image memory 115 and is arranged to detect areas of high intensity in the annotated set of images stored in the memory 115 based on information from the shape identifier 110 regarding the anatomical structures corresponding to different portions of the skeleton of the set of images. In an embodiment the hotspot detection unit 120 may comprise a threshold scanner unit for scanning the set of bone scan images and identifying pixels above a certain threshold. The hotspot detection unit 120 preferably comprises different thresholds for the different anatomical regions that are defined by the shape identifier unit 110. The hotspot feature extraction unit 130 for extracting one or more hotspot features for each hot spot comprises means for determining the shape and position of each hotspot.
In another embodiment the hotspot detection unit 120 may comprises an image normalization and filtering/threshold unit for scanning the set of bone scan images and identifying pixels above a certain threshold. The hotspot feature extractions unit 130 for extracting one or more hotspot features for each hot spot comprises means for determining the shape, texture and geometry of each hotspot. A detailed enumeration and description of each extracted feature of a preferred set of extracted features is found in Annex 1.
Information thus created regarding the detected areas of high intensity, so called “hotspots”, is stored in a first hotspot memory 125. A hotspot feature extraction unit 130 is connected to the first hotspot memory 125 and arranged to extract a set of hotspot features for each hot spot detected by the hotspot detection unit 120. Extracted hotspot features are stored in a second hotspot memory 135.
A first artificial neural network (ANN) unit 140 is connected to the second hotspot memory and arranged to calculate a likelihood for each hotspot of the hotspot set being a metastasis. The first artificial neural network unit 140 are fed with the features of each hotspot of the hotspot set produced by the hotspot feature extraction unit 130. The likelihood calculation is based on the set of hotspot features extracted by the hotspot feature extraction unit 130. The results of the likelihood calculations are stored in a third hotspot memory 145.
Preferably, in the first artificial neural network unit 140 there is arranged a pretrained ANN for each anatomical region. Each hotspot in a region is processed, the one after another, by the ANN arranged to handle hotspots from that region.
A patient feature extraction unit 150 is connected to the second and third hotspot memory 135, 145, and arranged to extract a set of patient features based on the number of hotspots detected by the hotspot detection unit 120 and stored in the first hotspot memory 125, and on the likelihood output values from the first artificial neural network unit 140 being stored in the third hotspot memory 145. The patient feature extraction unit 150 are provided with means to perform calculations that make use of both data from the hotspot feature extraction unit 130 and of the outputs of the first artificial neural network unit 140. The extracted patient features are stored in a patient feature memory 155. Preferably, the extracted patient features are those listed in a second portion of Annex 1.
A second artificial neural network unit 160 is connected to the patient feature memory 155, and is arranged to calculate a metastasis likelihood that the patient has one or more cancer metastases, based on the set of patient creatures extracted by the patient feature extraction unit 150 being stored in the patient feature memory 155. The system may optionally (hence the jagged line in
Test performed with the different embodiments showed that the system according to any of the embodiments presented above performed very well. In one of the embodiments described above the sensitivity was measured to 90% and the specificity also to 90%. The test method used in this embodiment was identical to the test method described in the article A new computer-based decision-support system for the interpretation of bone scans by Sadik M. et al published in Nuclear Medicine Communication nr. 27: p. 417-423 (hereinafter referred to as Sadik et al).
In the optional embodiment described above the performance was measured at the three configurations corresponding to the thresholds used to stratify the output value into one of four diagnoses. The sensitivity and the specificity at these configurations were:
Further is provided a method for the interpretation of isotope bone scan images with the aid of the system described in conjunction with
The method, in an embodiment of the present invention, involves performing a delineation of the entire anterior and posterior view of the skeleton except for the lower parts of the arms and legs using an Active Shape Model (ASM) approach. Omitting said portions of the skeleton is not an issue since these locations are very rare locations for metastases and they are sometimes not acquired in the bone scanning routine. For the purpose of explaining the present invention, an Active Shape Model is defined as a statistical method for finding objects in images, said method being based on a statistical model built upon a set of training images. In each training image a shape of the object is defined by landmark points that are manually determined by a human operator during a preparation or training phase. Subsequently a point distribution model is calculated which is used to describe a general shape relating to said objects together with its variation. The general shape can then be used to search other images for new examples of the object type, e.g. a skeleton, as is the case with the present invention. A method for training of an Active Shape Model describing the anatomy of a human skeleton is provided. The model comprises the following steps described below.
A first step may be to divide a skeleton segmentation into eight separate training sets 205. The training sets are chosen to correspond to anatomical regions that the inventors have found to be particularly suitable for achieving consistent segmentation. The eight separate training sets in 210 are as follows:
Each training set 210 comprises a number of example images. Each image is prepared with a set of landmark points. Each landmark point is associated with a coordinate pair representing a particular anatomical or biological point of the image. The coordinate pair is determined by manually pinpointing the corresponding anatomical/biological point 215. In the anterior image the following easily identifiable anatomical landmarks are used.
Before capturing the statistics of the training set 220, each set of landmark points 215 were aligned to a common coordinate frame, different for each of the eight training sets 210. This was achieved by scaling, rotating and translating the training shapes so that they corresponded as closely as possible to each other as described in Active shape models—their training and application by T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham presented in Computer Vision and Image Understanding, Vol. 61, no. 1, pp. 38-59, 1995 (hereinafter referred to as Cootes et al). By examination of the statistics 220 of the training sets a two-dimensional statistical point distribution model is derived that contains the shape variations observed in the training set. This statistical modeling of landmark (shape) variations across skeletons is performed as described in Cootes et al and in Application of the Active Shape Model in a commercial medical device for bone densitometry by H. H. Thodberg and A. Rosholm presented in the Proceedings of the 12th British Machine Vision Conference, 43-52, 2001, (hereinafter referred to as Thodberg et al).
The resulting statistical model 220 of shape variations can be applied to patient images in order to segment the skeleton. Starting with a mean shape, new shapes within a range of an allowable variation of the shape model can be generated similar to those of the training set such that the generated skeletons resemble the structures present in the patient image. The anterior body segments that may be segmented using this method may in one embodiment be; Cranium-Face-Neck, Spine, Sternum Upper, Sternum Lower, Right Arm, Left Arm, Right Ribs, Left Ribs, Right Shoulder, Left Shoulder, Pelvic, Bladder, Right Femur and Left Femur. The posterior body segments may in one embodiment be the Cranium, Neck, Upper Spine, Lower Spine, Spine, Right Arm, Left Arm, Right Ribs, Left Ribs, Right Scapula, Left Scapula, Ossa Coxae, Lower Pelvic, Bladder, Right Femur and the Left Femur.
A first step in a search process may be to find a start position for the mean shape of the anterior image. For instance the peak of the head may be chosen because in tests it has proved to be a robust starting position and it is easy to locate by examining the intensity in the upper part of the image above a specified threshold value in each horizontal row in the image.
The ensuing search for an instance of the skeleton shape model that fits the skeleton in the patient image is carried out in accordance with the algorithm described in Cootes et al and in Thodberg et al.
In another embodiment of the present invention a second method for bone scan image segmentation is provided. The goal of the second bone scan image segmentation method is as in the previous embodiment to identify and to delineate different anatomical regions of the skeleton in a bone scan image 300. These regions will be defined by superimposed outlines 320 onto the patient images 310, as shown in
An image registration method transforms one image into the coordinate system of another image. It is assumed that the images depict instances of the same object class, here, a skeleton. The transformed image is denoted the source image, while the non-transformed image is denoted the target image. The coordinate systems of the source and target images are said to correspond when equal image coordinates correspond to equal geometrical/anatomical locations on the object(s) contained in the source and target images. Performing segmentation by registration amounts to using a manually defined segmentation of the source image, and registering the source image to a target image where no segmentation is defined. The source segmentation is thereby transferred to the target image, thus creating a segmentation of the target image.
The segmentation of the source image in this embodiment defines the anatomy of a reference healthy patient and has been manually drawn by a clinical expert as a set of polygons.
The healthy reference image 400 is always used as the source image by the system, while the patient image to be examined acts as the target image. The result is a segmentation of the target image into skeletal regions as depicted in
The healthy reference image 400 used as the source image is constructed from 10 real examples of healthy patients with representative image quality and with normal appearance and anatomy. An algorithm is used which creates anterior and posterior images of a fictitious normal healthy patient with the average intensity and anatomy calculated from the group of example images. The system performs this task as described in Average brain models: A convergence study by Guimond A. Meunier J. Thirion J.-P presented in Computer Vision and Image Understanding, 77(2):192-210, 2000 (hereinafter referred to as Guimond et al). The result is shown in
The registration method is an improvement of the Morphon method as described in Non-Rigid Registration using Morphons by A. Wrangsjö, J. Pettersson, H. Knutsson presented in Proceedings of the 14th Scandinavian conference on image analysis (SCIA'05), Joensuu June 2005 (hereinafter referred to as Wrangsjö et al) and in Morphons: Segmentation using Elastic Canvas and Paint on Priors by H. Knutsson, M. Andersson presented in ICIP 200, Genova, Italy. September 2005 (hereinafter referred to as Knutsson et al). The method is improved to increase robustness for the purpose of segmenting skeletal images where both an anterior image and a posterior image are supplied. We now turn to a detailed description of this improvement.
The improvement of the Morphon method contained in this invention consists of a system for using multiple images of the same object for determining a single image transformation. In particular, we use the anterior and posterior skeletal images simultaneously. The goal of the improvement is to increase robustness of the method. To describe the improvement, necessary parts of the original Morphon method are first described, followed by a description of the improvement.
The following description of the so-called displacement vector field generation used in the Morphon method serves to introduce notation and put the improvement into perspective. For a more thorough treatment, refer to Wrangsjö et al and Knutsson et al.
The Morphon registration method proceeds in iterations, where each iteration brings the source image into closer correspondence with the target image. This corresponds to a small displacement of each source image element (pixel or voxel). The collection of all such displacements during an iteration are collected in a vector field of the same size as the source image where each vector describes the displacement of the corresponding image element. The vector field is determined using 4 complex filters. Each filter captures lines and edges in the image in a certain direction. The directions corresponding to the 4 filters are vertical, horizontal, top left to bottom right diagonal and top right to bottom left diagonal. Filtering the image by one of these filters generates a complex response which can be divided into a phase and a magnitude. Due to the Fourier shift theorem, the phase difference at a particular point between the filtered source and target images is proportional to the spatial shift required to bring the objects into correspondence at that point in the direction of the filter. When the phase and magnitude at each image point has been calculated for all 4 filter directions, the displacement vector can be found by solving a least-squares problem at each point. The magnitude can be used to derive a measure of the certainty of each displacement estimate. The certainties can be incorporated in the least-squares problem as a set of weights. The resulting weighted least squares problem is
where v is the sought 2-by-1 displacement vector, ni is the direction of the ith filter, vi is the phase difference corresponding to the ith filter and wi is the certainty measure derived from the magnitude of to the ith filter.
The improvement of this method contained in present invention consists of using more than one image for estimating a single vector field of displacements. Each image is filtered separately as described above, resulting in 4 complex responses for each image. The weighted least squares problem is expanded to include all images yielding
where k is the number of images (2 in the case of skeletal images). The effect of this is that the number of data points are multiplied by the number of images in the estimation of the two-dimensional displacement v, making the problem better defined. A further explanation of the development is provided by the certainty measures. Using a single image as input, regions of the resulting displacement vector field corresponding to low certainty measures will be poorly defined. If more than one image is supplied, chances are that at least one image is able to provide adequate certainty to all relevant regions.
As mentioned before, the hotspot detection unit 120 uses information from the shape identifier unit 110 described in conjunction with
Skeletal scintigraphy images differ significantly in intensity levels across patients, studies and hardware configurations. The difference is assumed multiplicative and zero intensity is assumed to be a common reference level for all images. Normalizing a source image with respect to a target image therefore amounts to finding a scalar factor that brings the intensities of the source image to equal levels with the target image. The intensities of two skeletal images are here defined as equal when the average intensity of healthy regions of the skeleton in the source image is equal to the corresponding regions in the target image. The normalization method, shown in a flowchart in
The step in 510 is carried out using information on image regions belonging to the skeleton provided by the transformed anatomical regions derived by the shape identifier unit 110 of
In step 520 the hotspots are segmented using one image filtering operation and one thresholding operation. The image is filtered using a difference-of-Gaussians band-pass filter which emphasizes small regions of high intensity relative to their respective surroundings. The filtered image is then thresholded at a constant level, resulting in a binary image defining the hotspot elements.
In step 530 any of the elements calculated in 510 that coincide with the hotspot elements calculated in 520 are removed. The remaining elements are assumed to correspond to healthy skeletal regions.
In step 540 the average intensity of the healthy skeletal elements is calculated. Denote this average intensity by A.
In step 550 a suitable normalization factor is determined in relation to a predefined reference intensity level. This level may for instance be set to 1000 here. The normalization factor B is calculated as B=1000/A.
In step 560 the intensities of the source image are adjusted by multiplication by B.
The hotspot segmentation described in 520 is dependent on the overall intensity level of the image which in turn is determined by the normalization factor calculated in 550. However, the normalization factor calculated in 550 is dependent on the hotspot segmentation from 520. Since the results of 520 and 550 are interdependent, 520 to 560 may in an embodiment be repeated 570 until no further change in the normalization factor occurs. Extensive tests have shown that this process normally converges in 3 or 4 repetitions.
In the above description the second ANN may in one embodiment be the same or a part of the first ANN.
In the above description the term point may be used to denote one or more pixels in an image.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” “comprising,” “includes” and/or “including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing has described the principles, preferred embodiments and modes of operation of the present invention. However, the invention should be regarded as illustrative rather than restrictive, and not as being limited to the particular embodiments discussed above. The different features of the various embodiments of the invention can be combined in other combinations than those explicitly described. It should therefore be appreciated that variations may be made in those embodiments by those skilled in the art without departing from the scope of the present invention as defined by the following claims.
The artificial neural network system, i.e., the first ANN unit 140, is fed with the following set of 27 features measuring the size, shape, orientation, localization and intensity distribution of each hotspot. The features are:
The second ANN unit which determines a patient-level diagnosis pertaining to the existence of metastatic disease uses the 34 features listed below as input. All features used by the second ANN unit are calculated from hotspots classified as having high metastasis probability by file first ANN unit.
Nordblom, Pierre, Hamadeh, Iman, Sjöstrand, Karl
| Patent | Priority | Assignee | Title |
| 10665346, | Oct 27 2016 | Progenics Pharmaceuticals, Inc. | Network for medical image analysis, decision support system, and related graphical user interface (GUI) applications |
| 10762993, | Oct 27 2016 | Progenics Pharmaceuticals, Inc. | Network for medical image analysis, decision support system, and related graphical user interface (GUI) applications |
| 10973486, | Jan 08 2018 | EXINI Diagnostics AB | Systems and methods for rapid neural network-based image segmentation and radiopharmaceutical uptake determination |
| 11321844, | Apr 23 2020 | EXINI Diagnostics AB | Systems and methods for deep-learning-based segmentation of composite images |
| 11386988, | Apr 23 2020 | EXINI Diagnostics AB | Systems and methods for deep-learning-based segmentation of composite images |
| 11424035, | Oct 27 2016 | Progenics Pharmaceuticals, Inc. | Network for medical image analysis, decision support system, and related graphical user interface (GUI) applications |
| 11534125, | Apr 24 2019 | EXINI Diagnostics AB | Systems and methods for automated and interactive analysis of bone scan images for detection of metastases |
| 11544407, | Sep 27 2019 | PROGENICS PHARMACEUTICALS, INC | Systems and methods for secure cloud-based medical image upload and processing |
| 11564621, | Sep 27 2019 | PROGENICS PHARMACEUTICALS, INC | Systems and methods for artificial intelligence-based image analysis for cancer assessment |
| 11657508, | Jan 07 2019 | EXINI Diagnostics AB | Systems and methods for platform agnostic whole body image segmentation |
| 11721428, | Jul 06 2020 | EXINI Diagnostics AB | Systems and methods for artificial intelligence-based image analysis for detection and characterization of lesions |
| 11894141, | Oct 27 2016 | Progenics Pharmaceuticals, Inc. | Network for medical image analysis, decision support system, and related graphical user interface (GUI) applications |
| 11900597, | Sep 27 2019 | EXINI Diagnostics AB | Systems and methods for artificial intelligence-based image analysis for cancer assessment |
| Patent | Priority | Assignee | Title |
| 7450747, | Jul 12 2002 | GE Medical Systems Global Technology Company, LLC | System and method for efficiently customizing an imaging system |
| 7970194, | May 26 2006 | Toshiba Medical Systems Corporation | Image processing apparatus, magnetic resonance imaging apparatus and image processing method |
| 8211401, | Dec 05 2008 | MOLECULAR INSIGHT PHARMACEUTICALS, INC | Technetium- and rhenium-bis(heteroaryl) complexes and methods of use thereof for inhibiting PSMA |
| 8538166, | Nov 21 2006 | MANTISVISION LTD | 3D geometric modeling and 3D video content creation |
| 8705887, | Aug 22 2008 | Weyerhaeuser NR Company | Method and apparatus for filling in or replacing image pixel data |
| 8778305, | Aug 01 2008 | The Johns Hopkins University | PSMA-binding agents and uses thereof |
| 8855387, | Dec 28 2007 | EXINI Diagnostics AB | System for detecting bone cancer metastases |
| 8962799, | Jan 09 2008 | Molecular Insight Pharmaceuticals, Inc. | Technetium—and rhenium-bis(heteroaryl) complexes and methods of use thereof |
| 20030215120, | |||
| 20060062425, | |||
| 20070081712, | |||
| 20070081713, | |||
| 20070100225, | |||
| 20150110716, | |||
| 20160203263, | |||
| 20180144828, | |||
| EP1426903, | |||
| EP1508872, | |||
| WO2007062135, | |||
| WO2009084995, | |||
| WO2018081354, | |||
| WO9905503, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Oct 26 2012 | SJOSTRAND, KARL | EXINI Diagnostics AB | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 040231 | /0584 | |
| Nov 05 2012 | NORDBLOM, PIERRE | EXINI Diagnostics AB | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 040231 | /0584 | |
| Nov 07 2012 | HAMADEH, IMAN | EXINI Diagnostics AB | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 040231 | /0584 | |
| Sep 30 2016 | EXINI Diagnostics AB | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| May 02 2022 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
| May 02 2022 | M1555: 7.5 yr surcharge - late pmt w/in 6 mo, Large Entity. |
| Date | Maintenance Schedule |
| Sep 17 2022 | 4 years fee payment window open |
| Mar 17 2023 | 6 months grace period start (w surcharge) |
| Sep 17 2023 | patent expiry (for year 4) |
| Sep 17 2025 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Sep 17 2026 | 8 years fee payment window open |
| Mar 17 2027 | 6 months grace period start (w surcharge) |
| Sep 17 2027 | patent expiry (for year 8) |
| Sep 17 2029 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Sep 17 2030 | 12 years fee payment window open |
| Mar 17 2031 | 6 months grace period start (w surcharge) |
| Sep 17 2031 | patent expiry (for year 12) |
| Sep 17 2033 | 2 years to revive unintentionally abandoned end. (for year 12) |