A computer-implemented method for tracking one or more objects in a sequence of images includes generating a dictionary based on object locations in a first image included in the sequence of images. One or more object landmark candidates are identified in the sequence of images and a plurality of tracking hypothesis for the object landmark candidates are generated. A first tracking hypothesis is selected from the plurality of tracking hypothesis based on the dictionary.
|
18. A method of updating a dictionary to represent change in appearance of a catheter target object, the method comprising:
generating, by a computer, a non-catheter structures mask identifying structures unrelated to the target catheter object in an initial image frame;
generating, by the computer, the dictionary based on an initial appearance of the target object in the initial image frame, wherein sparse coding is used to represent non-catheter structures mask as a plurality of basis vectors in the dictionary;
receiving, by the computer, a plurality of subsequent image frames indicating a change in the initial appearance of the target catheter object;
applying, by the computer, a learning algorithm to compute labels for each of the subsequent image frames;
generating, by the computer, a voting score for each of the subsequent image frames based on a voting contribution of a plurality of image patches used to localize the catheter target object in the initial image frame, wherein the images are analyzed on a patch-by-patch basis;
selecting, by the computer, a subset of the subsequent image frames based on the computed labels and the voting score;
updating, by the computer, the dictionary based on the subset of the subsequent image frames.
11. An article of manufacture for tracking one or more catheter objects in a sequence of images, the article of manufacture comprising a computer-readable, non-transitory medium holding computer-executable instructions for performing the method comprising:
determining a foreground portion of the first image comprising portions of the first image corresponding to one or more catheter electrode locations;
determining a background portion of the first image which excludes the foreground portion;
applying a steerable filter or a pre-processing method to the background portion of the first image to create a non-catheter structures mask which excludes ridge-like structures in the background portion of the first image;
generating a dictionary based on catheter object locations in the first image, wherein sparse coding is used to represent the non-catheter structures mask as a plurality of basis vectors in the dictionary;
identifying one or more catheter object landmark candidates in the sequence of images;
generating a plurality of tracking hypothesis for the catheter object landmark candidates;
generating a voting score for the catheter object landmark candidates based on a voting contribution of each of a plurality of image patches used to localize the catheter object locations in the first image; and
selecting a first tracking hypothesis from the plurality of tracking hypothesis based on the dictionary and the voting score.
1. A computer-implemented method for tracking one or more catheter objects in a sequence of images, the method comprising:
determining, by a computer, a foreground portion of the first image comprising portions of the first image corresponding to one or more catheter electrode locations;
determining, by the computer, a background portion of the first image which excludes the foreground portion;
applying, by the computer, a steerable filter or a pre-processing method to the background portion of the first image to create a non-catheter structures mask which excludes ridge-like structures in the background portion of the first image;
generating, by the computer, a dictionary based on catheter object locations in the first image, wherein sparse coding is used to represent the non-catheter structures mask as a plurality of basis vectors in the dictionary;
identifying, by the computer, one or more catheter object landmark candidates in the sequence of images;
generating, by the computer, a plurality of tracking hypothesis for the catheter object landmark candidates;
generating, by the computer, a voting score for the catheter object landmark candidates based on a voting contribution of each of a plurality of image patches used to localize the catheter object locations in the first image; and
selecting, by the computer, a first tracking hypothesis from the plurality of tracking hypothesis based on the dictionary and the voting score.
15. A system for tracking one or more catheter objects in a sequence of images, the system comprising:
a receiver module operably coupled to an imaging device and configured to receive the sequence of images from the imaging device; and
one or more first processors configured to:
determine a foreground portion of the first image comprising portions of the first image corresponding to one or more catheter electrode locations;
determining a background portion of the first image which excludes the foreground portion;
apply a steerable filter or a pre-processing method to the background portion of the first image to create a non-catheter structures mask which excludes ridge-like structures in the background portion of the first image;
generate a dictionary based on object locations in the first image, wherein sparse coding is used to represent non-catheter structures mask as a plurality of basis vectors in the dictionary,
identify one or more catheter object landmark candidates in the sequence of images; and
one or more second processors configured to:
generate a plurality of tracking hypothesis for the catheter object landmark candidates,
generate a voting score for the catheter object landmark candidates based on a voting contribution of a plurality of image patches used to localize the catheter object locations in the first image, and
select a first tracking hypothesis from the plurality of tracking hypothesis based on the dictionary and the voting score.
2. The method of
determining, by the computer, a confidence score for each of tracking hypothesis included in the plurality of tracking hypothesis; and
selecting, by the computer, a respective tracking hypothesis with the highest confidence score as the first tracking hypothesis.
3. The method of
applying, by the computer, a learning algorithm to compute labels for each image in the sequence of images following the first image;
selecting, by the computer, a plurality of images based on the computed labels; and
updating, by the computer, the dictionary based on selected images.
5. The method of
identifying, by the computer, a first set of candidate samples included in the sequence of images;
determining, by the computer, a first stage probability score for each candidate samples in the first set;
identifying, by the computer, a second set of candidate samples from the first set based on the first stage probability scores;
determining, by the computer, a second stage probability score for each of the candidate samples in the second set; and
identifying, by the computer, the catheter object landmark candidates from the second set based on the second stage probability scores.
6. The method of
identifying, by the computer, a first object landmark candidate corresponding to a first object type using one or more first classifiers trained for the first object type; and
identifying, by the computer, a second object landmark candidate corresponding to a second object type using one or more second classifiers trained for the second object type.
7. The method of
8. The method of
determining, by the computer, a set of catheter object landmarks in a previous image;
calculating, by the computer, a plurality of translation vectors, each translation vector corresponding to a translation between one of the catheter object landmark candidates and an catheter object landmark included in the set of catheter object landmarks;
generating, by the computer, a plurality of seed hypothesis by applying each of the translation vectors to the set of object landmarks in the previous image; and
applying, by the computer, a geometric transformation to each seed hypothesis to generate the plurality of tracking hypothesis.
9. The method of
10. The method of
12. The article of manufacture
determining a confidence score for each of tracking hypothesis included in the plurality of tracking hypothesis; and
selecting the tracking hypothesis with the highest confidence score as the first tracking hypothesis.
13. The article of manufacture of
identifying a first set of candidate samples included in the sequence of images;
determining a first stage probability score for each candidate samples in the first set;
identifying a second set of candidate samples from the first set based on the first stage probability scores;
determining a second stage probability score for each of the candidate samples in the second set; and
identifying the object landmark candidates from the second set based on the second stage probability scores.
14. The article of manufacture of
determining a set of catheter object landmarks in a previous image;
calculating a plurality of translation vectors, each translation vector corresponding to a translation between one of the catheter object landmark candidates and an catheter object landmark included in the set of catheter object landmarks;
generating a plurality of seed hypothesis by applying each of the translation vectors to the set of catheter object landmarks in the previous image; and
applying a geometric transformation to each seed hypothesis to generate the plurality of tracking hypothesis.
16. The system of
17. The system of
a transfer module for transferring object landmark candidates corresponding to the first image to the central processing units for object tracking while the graphical processing units identity object landmark candidates in a second image.
19. The method of
receiving, by the computer, one or more additional image frames; and
applying, by the computer, the updated dictionary to track the catheter target object in the additional image frames.
|
This application claims priority to U.S. provisional application Ser. No. 61/604,000 filed Feb. 28, 2012, which is incorporated herein by reference in its entirety.
The present invention relates generally to methods, systems, and apparatuses which utilize sparse appearance representation and online sparse appearance dictionary updating techniques for tracking objects presented in a sequence of images.
Atrial Fibrillation (“AF”) is a rapid, highly irregular heartbeat caused by abnormalities in the electrical signals generated by the atria of the heart. AF is the most common cardiac arrhythmia and involves the two upper chambers of the heart. Surgical and catheter-based electrophysiology therapies have become common AF treatments throughout the world. Catheter ablation modifies the electrical pathways of the heart in order to treat the disease.
To measure electrical signals in the heart and assist the ablation operation, three catheters are inserted and guided to the left atrium. These three catheters include an ablation catheter, a circumferential mapping catheter, and a coronary sinus catheter. The operation is monitored with live fluoroscopic images for navigation guidance. Tracking three catheters with such different characteristics presents several challenges. Catheters have non-uniform appearance and shapes. In general, catheter characteristics include items such as tip electrode, size, spacing, and insertion length. Ablation catheters often have four electrodes with the tip electrode as a solid tube appearance in the fluoroscopic images, but may have electrode configuration different from each other. The circumferential mapping catheter has large intra-class variations because of differences in catheter diameter, electrode size, and number (i.e., number of poles and spacing). Coronary sinus catheters also vary from each other in terms of catheter length and electrode configuration. In addition, the three catheters may freely move within a large range and often occlude each other or other structures in the 2-D fluoroscopic images. During an electrophysiology operation such as an AF treatment, catheters may move into and out of an image. In addition, catheters are not rigid structures and may deform during the operation. Moreover, the use of fluoroscopic images presents additional challenges to tracking catheters in fluoroscopic images during the operation. Fluoroscopic images constantly change due to cardiac and respiratory motion and device movement. Additionally, structures in a fluoroscopic image often cause the background to be cluttered. The level of radiation may also affect the image quality and the signal to noise ratio.
Embodiments of the present invention address and overcome one or more of the above shortcomings and drawbacks, by providing methods, systems, and apparatuses which utilize sparse appearance representation and online sparse appearance dictionary update techniques for tracking objects presented in a sequence of images. This technology is particularly well-suited for, but by no means limited to, tracking catheters in fluoroscopic images during AF ablation procedures and tracking objects in dynamic environments where the object appearance constantly changes due to change of the lighting conditions and/or shadows, for example. For the example of catheter tracking, using the techniques described herein, medical personnel may accurately track the location and motion of catheters in real-time during such procedures and this information may be stored in the system. In turn, the increased accuracy of such tracking may allow medical personnel to increase the effectiveness and minimize the risks of AF ablation procedures, as well as allow the medical personnel to review in-treatment catheter parameters such ablation locations, temperature, and force after the procedure is done.
Embodiments of the present invention are directed to a computer-implemented method for tracking one or more objects in a sequence of images. The method includes generating a dictionary based on object locations in a first image included in the sequence of images, identifying one or more object landmark candidates in the sequence of images, generating a plurality of tracking hypothesis for the object landmark candidates, and selecting a first tracking hypothesis from the plurality of tracking hypothesis based on the dictionary. In some embodiments, the sequence of images corresponds to a plurality of fluoroscopic images and at least some of the objects in the image correspond to at least one of a catheter tip and catheter electrode. In some embodiments, the first tracking hypothesis is selected from the plurality of tacking hypothesis by determining a confidence score for each tracking hypothesis and selecting the tracking hypothesis with the highest confidence score.
According to one aspect of the invention, foreground and background portions of the first image are determined. Then, a steerable filter or a pre-processing method is applied to the background portion to create a filtered image. The dictionary is then generated based on the filtered image. In other embodiments, the dictionary may be generated based on the background portion of the first image.
In some embodiments of the invention, a learning algorithm is applied to computed labels for each image in the sequence of images following a first image. Next, a plurality of images are selected based on the computed labels and used to update the dictionary. In one embodiment, the learning algorithm is a semi-supervised learning algorithm.
In another embodiment of the invention, one or more object landmark candidates in the sequence of images are identified by a two-step process. First, a first set of candidate samples included in the sequence of images is identified and a first stage probability score for each candidate samples in the first set is determined. Then, a second set of candidate samples from the first set is identified based on the first stage probability scores and a second stage probability score for each of the candidate samples in the second set is determined. The landmark candidates are then identified from the second set based on the second set probability scores.
According to one aspect of the invention, a first object landmark candidate corresponding to a first object type is identified using one or more first classifiers trained for the first object type and a second object landmark candidate corresponding to a second object type is identified using one or more second classifiers trained for the second object type. In some embodiments, the first object type corresponds to a catheter tip and the second object type corresponds to a catheter electrode. In some embodiments, each classifier is a probabilistic boosting tree.
According to another aspect of the invention, generating tracking hypothesis for object landmarks includes determining a set of landmarks in a previous image; calculating a plurality of translation vectors, each translation vector corresponding to a translation between one of the landmark candidates and one of the landmarks included in catheter model; generating a plurality of seed hypothesis by applying each of the translation vectors to the set of landmarks in the previous image; and applying a geometric transformation to each seed hypothesis to generate the plurality of tracking hypothesis. In some embodiments, the geometric transformation is an affine transformation.
Embodiments of the present invention are also directed to systems for tracking one or more objects in a sequence of images. The systems include a receiver module operably coupled to an imaging device and configured to receive a sequence of images from the imaging device. The system also include one or more first processors configured to generate a dictionary based on object locations in a first image included in the sequence of images and identify one or more object landmark candidates in the sequence of images. In some embodiments, these first processors are computational processor units (CPUs). The system also includes one or more second processors configured to generate a plurality of tracking hypothesis for the object landmark candidates and select a first tracking hypothesis from the plurality of tracking hypothesis based on the dictionary. In some embodiments, the second processors are graphical processing units.
Embodiments of the present invention are also directed at methods of updating a dictionary to represent change in appearance of a target object. First, the dictionary is generated based on an initial appearance of the target object in an initial image frame. Next, a plurality of subsequent image frames indicating a change in the initial appearance of the target object are received. Then a learning algorithm is used to compute labels for each of the subsequent image frames. A subset of the subsequent image frames are selected based on the computed labels. Finally, the dictionary is updated based on the subset of the subsequent image frames. In the some embodiments, the updated dictionary is then applied to track the target object in later image frames.
Additional features and advantages of the invention will be made apparent from the following detailed description of illustrative embodiments that proceeds with reference to the accompanying drawings.
The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there is shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:
The following disclosure describes the present invention according to several embodiments directed at the tracking multiple catheters during surgical procedures. However, one skilled in the art would recognize that the techniques described herein may also be applicable to other domains, allowing various types of objects to be tracked. Thus, the techniques described herein have applications both in surgical and non-surgical domains.
Continuing with reference to
Continuing with reference to
Continuing with reference to
At 425, a dictionary Φ is generated using the non-catheter mask with the background image. In some embodiments, sparse coding is used to represent the non-catheter structures mask as several basis vectors in the dictionary. As would be understood by one skilled in the art, sparse coding allows a signal x to be represented as a linear combination of a one or more basis vectors in the dictionary Φ=[φ1 . . . φk]εRn×k. More generally, the signal may be represented by the equation:
x=Φα+ε,
where α are the coefficients of the bases and ε represents the noise. Given Φ and a dataset X={xi}i=1N, the solution of a may be formulated as a sparse coding product with the l0 regularization:
α*=argαmin∥α∥0,s.t.Σi=1N∥xi−Φαi∥2≦ε,
where ∥·∥0 denotes the l0-norm, which is the number on non-zero entries in the vector. Thus, given a dictionary Φ for each image patch x of an object, a sparse solution can be obtained by solving this optimization problem. However, the lo regularization presented above is non-convex and may be challenging to solve. Thus, in some embodiments, the l0 regulation for α* is reformulated as a convex optimization problem with the l1 regulation:
α*=argαmin∥α∥1,s.t.Σi=1N∥xi−Φαi∥2≦ε,
To learn the dictionary, an objective function is used. In some embodiments, where locality is more essential than sparsity, techniques such as Linear Locality Coding (LLC) may be used and the objective function may include one or more distance terms. For example, in some embodiments the objective function is defined as
Φ=argφ,αΣi=1N∥xi−Σi=1N∥2+∥di⊙αi∥2,s.t.∀i,1Tαi=1,
where ⊙ denotes element-wise multiplication and di is the Euclidean distance vector between xi and the basis vectors in Φ. To minimize the search required to find a solution for Φ, methods such as K-selection may be used to perform basis selection.
In some embodiments, multiple dictionaries may be learned and used for object tracking. For example, the portions of the image corresponding to the objects are used to learn a positive dictionary, while the remaining portions (i.e., the background) may be used to learn a negative dictionary. Learning of the positive dictionary and negative dictionary may be performed simultaneously, in parallel, or sequentially.
As illustrated in the example of
At 520, clustered detections are removed from the set of candidate samples to keep high-confident detections using a technique such as non-maximal suppression (NMS). In each image frame, a number of electrodes and tip candidates are selected and denoted as a catheter landmark candidate. Then, at 525, any detection located greater than a threshold number of pixels from the initial catheter location (e.g., as identified by the process 300 illustrated in
Any detector known in the art may be used in the landmark detection process 500 illustrated in
For example, according to an embodiment of the present invention, each classifier is a Probabilistic Boosting Tree (PBT) that uses approximately 100,000 Haar features in a centered window of size Hc×Hc. Classifiers in this embodiment output a probability P(e=(x,y)|D). The detected candidate positions may then be augmented with a set of discrete orientations and fed to a trained oriented point detector. The oriented point detectors may use a richer feature pool including steerable feature responses and image intensity differences relative to the query position and orientation. Probabilistic Boosting Trees are described in greater detail in U.S. Pat. No. 7,702,596, issued Apr. 20, 2010, and entitled “Probabilistic Boosting Tree Framework for Learning Discriminative Models”, which is incorporated herein by reference in its entirety.
To make the landmark detection process 500 more computationally efficient, techniques such as Marginal Space Learning (“MSL”) may be used to first detect just the tip and electrode positions and then, at promising positions, search for all orientations. MSL is a fast object detection method that searches for objects in a sequence of increasing dimensions. Promising candidates from one subspace may be augmented with more parameters and a trained detector may be used to prune the new candidates. MSL is described in greater detail in U.S. Pat. No. 7,916,919, issued Mar. 29, 2011, and entitled “System and Method for Segmenting Chambers of a Heart in a Three Dimensional Image”, which is incorporated herein by reference in its entirety.
The example process 600 illustrated in
Y*t=argαmaxP(Ytα|Z0 . . . t)
Markovian representation of catheter motion leads to:
Y*t=argY
The formula for Y*t combines two parts: a likelihood term, P(Zt|Ytα) and a prediction term P(Ytα|Y*t-1).
Continuing with reference to
P(Zt|Ytα)=(1−λ·δo)·P(L*t|Ytα)P(
where P(L*t|Ytα) is the estimated detection probability measure about catheter landmarks at the t—the frame that assists estimation of Yt. Tt-1Y is the template for the catheter Y, while λ is a weight factor computed by the normalized cross-correlation (“NCC”) score. The P(
Some embodiments of the present invention include an occlusion factor δO in the calculation of the likelihood term at 710. In AF ablation fluoroscopic images, catheters freely move inside the heart chamber and often occlude with each other or other structures. When occlusion occurs, integration of intensity-based normalized cross-correlation (“NCC”) matching in the MAP estimation may introduce noise. Therefore, the framework described herein may reason an occlusion map using the scene sparse representation and catheter landmark detection candidates. Assume two or more objects occlude each other in the image, and denote the interacting region as St; the goal is to assign a label, oi, from the set {occlusion as 1, no occlusion as 0} to each pixel, xi, in St to obtain a label set Ot. The occlusion factor δO in the equation for P(Zt|Ytα) may be computed as
where v is the occlusion threshold and ∥Ytα| is the model size. The occlusion inference is using the catheter landmark detection probability maps and fluoroscopic scene probability map. The methods described herein may be used to track all three catheters used for atrial fibrillation ablation procedures. Therefore, four maps are used to compute Ot(xi). More specifically, Ot(xi) may be defined as:
where Ptk represents each probability map. Using the scene representation and landmark detection probability, the likelihood term is dynamically estimated via occlusion reasoning. The catheter landmark detectors are trained using a large amount of data covering various object-context scenarios including occlusion and catheter foreshortening. As one skilled in the art would understand, occlusion reasoning integrates NCC matching score for non-occlusion hypothesis evaluation and utilizes the landmark detection probability and scene sparse representation in case of occlusion.
Returning to
where g0(·) is updated by the tracking result of the previous frame Yt-1. The values for g0(·) ∀k, k≠0 are learned from the training database to represent the most probable catheter locations in the fluoroscopic image. Finally, at 720, the likelihood term and the prediction term are used to calculate the catheter's location and appearance Y*t.
In some embodiments of the present invention a voting map comprised of image patches is used to localize the target location. For each landmark candidate, a voting score is calculated by considering the voting contribution of each of the patches. The image patch with the largest voting score is then used to select the targets and may also be used to update the dictionary.
Since the model-based hypotheses are generated in a discrete space, small location errors may be present even with the best candidate. In order to refine the results, in some embodiments of the invention, the tracking estimation is refined by searching for a local maximum in the parameter space. Any search technique known in the art may be used to perform the searching including, for example, Powell's conjugate gradient descent.
Foreground and background structures in a fluoroscopic image sequence change and move from image to image. Using the template learned at catheter initialization 400 may not be sufficient to overcome the catheter appearance change due to device movement and heart motion. Thus, in some embodiments, the catheter template is dynamically updated and to MAP estimation of Y*t. The catheter model may be updated online as:
TtY=(1−φY)·Tt-1Y+φY·l(Y*t),
where TtY represents the catheter template and l(Y*t) is the image patch of Y*t. Thus, the high-confidence localized catheter appearance in the current image may be fused with the learned template.
Continuing with reference to
Continuing with reference to
Φnew=(Φ\Pr)∪Ur,
where Pr and Ur represent the first r basis in the sorted basis of the dictionary and candidate data, respectively.
As demonstrated in the example of
As shown in
The computer system 1210 also includes a system memory 1230 coupled to the bus 1221 for storing information and instructions to be executed by processors 1220. The system memory 1230 may include computer readable storage media in the form of volatile and/or nonvolatile memory, such as read only memory (ROM) 1231 and/or random access memory (RAM) 1232. The system memory RAM 1232 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The system memory ROM 1231 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 1230 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 1220. A basic input/output system (233 (BIOS) containing the basic routines that help to transfer information between elements within computer system 1210, such as during start-up, may be stored in ROM 1231. RAM 1232 may contain data and/or program modules that are immediately accessible to and/or presently being operated on by the processors 1220. System memory 1230 may additionally include, for example, operating system 1234, application programs 1235, other program modules 1236 and program data 1237.
The computer system 1210 also includes a disk controller 1240 coupled to the bus 1221 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 1241 and a removable media drive 1242 (e.g., floppy disk drive, compact disc drive, tape drive, and/or solid state drive). The storage devices may be added to the computer system 1210 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire).
The computer system 1210 may also include a display controller 1265 coupled to the bus 1221 to control a display or monitor 1265, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The computer system includes an input interface 1260 and one or more input devices, such as a keyboard 1262 and a pointing device 1261, for interacting with a computer user and providing information to the processor 1220. The pointing device 1261, for example, may be a mouse, a trackball, or a pointing stick for communicating direction information and command selections to the processor 1220 and for controlling cursor movement on the display 1266. The display 1266 may provide a touch screen interface which allows input to supplement or replace the communication of direction information and command selections by the pointing device 1261.
The computer system 1210 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 1220 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 1230. Such instructions may be read into the system memory 1230 from another computer readable medium, such as a hard disk 1241 or a removable media drive 1242. The hard disk 1241 may contain one or more datastores and data files used by embodiments of the present invention. Datastore contents and data files may be encrypted to improve security. The processors 1220 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 1230. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
As stated above, the computer system 1210 may include at least one computer readable medium or memory for holding instructions programmed according embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processor 1220 for execution. A computer readable medium may take many forms including, but not limited to, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as hard disk 1241 or removable media drive 1242. Non-limiting examples of volatile media include dynamic memory, such as system memory 1230. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the bus 1221. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
The computing environment 1200 may further include the computer system 1220 operating in a networked environment using logical connections to one or more remote computers, such as remote computer 1280. Remote computer 1280 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer 1210. When used in a networking environment, computer 1210 may include modem 1272 for establishing communications over a network 1271, such as the Internet. Modem 1272 may be connected to system bus 1221 via user network interface 1270, or via another appropriate mechanism.
Network 1271 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 1210 and other computers (e.g., remote computing system 1280). The network 1271 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-12 or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 1271.
The embodiments of the present disclosure may be implemented with any combination of hardware and software. In addition, the embodiments of the present disclosure may be included in an article of manufacture (e.g., one or more computer program products) having, for example, computer-readable, non-transitory media. The media has embodied therein, for instance, computer readable program code for providing and facilitating the mechanisms of the embodiments of the present disclosure. The article of manufacture can be included as part of a computer system or sold separately.
While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
Zhang, Lei, Comaniciu, Dorin, Strobel, Norbert, Chen, Terrence, Wu, Wen
Patent | Priority | Assignee | Title |
10366490, | Mar 27 2017 | SIEMENS HEALTHINEERS AG | Highly integrated annotation and segmentation system for medical imaging |
Patent | Priority | Assignee | Title |
5267328, | Jan 22 1990 | Method for selecting distinctive pattern information from a pixel generated image | |
6405072, | Jan 28 1991 | INTEGRA BURLINGTON MA, INC | Apparatus and method for determining a location of an anatomical target with reference to a medical apparatus |
6484049, | Apr 28 2000 | STRYKER EUROPEAN HOLDINGS I, LLC | Fluoroscopic tracking and visualization system |
6567949, | May 31 1999 | Qualcomm Incorporated | Method and configuration for error masking |
6856827, | Apr 28 2000 | STRYKER EUROPEAN HOLDINGS I, LLC | Fluoroscopic tracking and visualization system |
7756567, | Aug 29 2003 | MIDCAP FUNDING IV TRUST, AS SUCCESSOR TO EXISTING ADMINISTRATIVE AGENT | Image guided radiosurgery method and apparatus using registration of 2D radiographic images with digitally reconstructed radiographs of 3D scan data |
7844320, | Jun 28 1996 | CICAS IP LLC | Method and apparatus for volumetric image navigation |
7920732, | Feb 13 2008 | FUJIFILM Corporation | Capsule endoscope system and endoscopic image filing method |
8340437, | May 29 2007 | University of Iowa Research Foundation | Methods and systems for determining optimal features for classifying patterns or objects in images |
8599266, | Jul 01 2002 | Regents of the University of California, The | Digital processing of video images |
8649606, | Feb 10 2010 | California Institute of Technology | Methods and systems for generating saliency models through linear and/or nonlinear integration |
8675021, | Nov 24 1999 | Dartfish SA | Coordination and combination of video sequences with spatial and temporal normalization |
20060188139, | |||
20080025586, | |||
20080051648, | |||
20080219466, | |||
20100061597, | |||
20100183204, | |||
20100220906, | |||
20110058026, | |||
20110313285, | |||
20120114215, | |||
20120230565, | |||
20130108131, | |||
20130113791, | |||
20140051992, | |||
20140089000, | |||
20140112566, | |||
20150018671, | |||
20150051480, | |||
20150164329, | |||
20150282890, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Feb 28 2013 | Siemens Healthcare GmbH | (assignment on the face of the patent) | / | |||
Mar 11 2013 | WU, WEN | Siemens Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030566 | /0990 | |
Mar 11 2013 | COMANICIU, DORIN | Siemens Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030566 | /0990 | |
Mar 11 2013 | STROBEL, NORBERT | Siemens Aktiengesellschaft | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030567 | /0033 | |
Mar 12 2013 | ZHANG, LEI | Siemens Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030566 | /0990 | |
Jun 05 2013 | CHEN, TERRENCE | Siemens Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 030566 | /0990 | |
Aug 15 2013 | Siemens Corporation | Siemens Aktiengesellschaft | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 031059 | /0435 | |
May 24 2017 | Siemens Aktiengesellschaft | Siemens Healthcare GmbH | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 042540 | /0044 | |
Dec 19 2023 | Siemens Healthcare GmbH | SIEMENS HEALTHINEERS AG | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 066088 | /0256 |
Date | Maintenance Fee Events |
Sep 24 2020 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Date | Maintenance Schedule |
Jul 11 2020 | 4 years fee payment window open |
Jan 11 2021 | 6 months grace period start (w surcharge) |
Jul 11 2021 | patent expiry (for year 4) |
Jul 11 2023 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jul 11 2024 | 8 years fee payment window open |
Jan 11 2025 | 6 months grace period start (w surcharge) |
Jul 11 2025 | patent expiry (for year 8) |
Jul 11 2027 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jul 11 2028 | 12 years fee payment window open |
Jan 11 2029 | 6 months grace period start (w surcharge) |
Jul 11 2029 | patent expiry (for year 12) |
Jul 11 2031 | 2 years to revive unintentionally abandoned end. (for year 12) |