In one aspect, a method to reduce false alarms in identifying whether a candidate image is from an object class includes projecting the candidate image onto an object class subspace and projecting the candidate image onto a non-object class subspace. The method also includes determining whether the candidate image is from the object class using a bayesian decision function based on the projections on the object class subspace and the non-object class subspace.
In another aspect, a method to reduce false alarms in identifying whether a candidate mine image is from a mine class includes projecting the candidate mine image onto a mine subspace and projecting the candidate mine image onto a non-mine subspace. The method also includes determining whether the candidate mine image represents a mine using a bayesian decision function based on the projections on the mine class subspace and the non-mine class subspace.
|
17. A method to reduce false alarms in identifying whether a candidate mine image is from a mine class comprising:
generating a mine subspace using principal component analysis on images of mines;
generating a non-mine subspace using principal component analysis on images determined to be false positives of mines;
projecting the candidate mine image onto the mine subspace;
projecting the candidate mine image onto the non-mine subspace;
determining whether the candidate mine image represents a mine using a bayesian decision function based on projecting of the candidate image onto the mine class subspace and projecting of the candidate image onto the non-mine subspace; and
removing similar principal components from the mine subspace and the non-mine class subspace so that each subspace is disjoint from each other before projecting the candidate image onto the mine subspace and projecting the candidate image onto the non-mine subspace.
12. An apparatus to reduce false alarms in identifying whether a candidate mine image is from a mine class, comprising:
circuitry to:
generate a mine subspace using principal component analysis on images of mines;
generate a non-mine subspace using principal component analysis on images determined to be false positives of mines;
project the candidate mine image onto the mine subspace;
project the candidate mine image onto the non-mine subspace;
determine whether the candidate mine image represents a mine using a bayesian decision function based on projecting of the candidate image onto the mine subspace and projecting of the candidate image onto the non-mine subspace space; and
remove similar principal components from the mine subspace and the non-mine subspace so that each subspace is disjoint from each other before projecting the candidate image onto the mine subspace and projecting the candidate image onto the non-mine subspace.
4. A method to reduce false alarms in identifying whether a candidate mine image is from a mine class comprising:
generating a mine subspace using principal component analysis on images of mines;
generating a non-mine subspace using principal component analysis on images determined to be false positives of mines;
projecting the candidate mine image onto the mine subspace;
projecting the candidate mine image onto the non-mine subspace;
removing similar principal components from each of the mine class subspace and the non-mine class subspace so that each subspace is disjoint from each other before projecting the candidate image onto the mine subspace and projecting the candidate image onto the non-mine subspace; and
determining whether the candidate mine image represents a mine using a bayesian decision function based on the projecting of the candidate image onto the mine subspace and the projecting of the candidate image onto the non-mine subspace.
1. A method to reduce false alarms in identifying whether a candidate image is from an object class comprising:
generating an object class subspace using principal component analysis on images of objects from the object class;
generating a non-object class subspace using principal component analysis on images determined to be false positives of objects from the object class;
projecting the candidate image onto the object class subspace;
projecting the candidate image onto the non-object class subspace;
removing similar principal components from each subspace so that each subspace is disjoint from each other before projecting the candidate image onto the object class subspace and projecting the candidate image onto the non-object class subspace; and
determining whether the candidate image is from the object class using a bayesian decision function based on the projecting of the candidate image onto the object class subspace and the projecting of the candidate image onto the non-object class subspace.
8. An article comprising a non-transitory machine-readable medium that stores executable instructions to reduce false alarms in identifying whether a candidate mine image is from a mine class, the instructions causing a machine to:
generate a mine subspace using principal component analysis on images of mines;
generate a non-mine subspace using principal component analysis on images determined to be false positives of mines;
project the candidate mine image onto the mine subspace;
project the candidate mine image onto the non-mine subspace;
remove similar principal components from each of the mine class subspace and the non-mine class subspace so that each subspace is disjoint from each other before projecting the candidate image onto the mine subspace and projecting the candidate image onto the non-mine subspace; and
determine whether the candidate mine image represents a mine using a bayesian decision function based on projecting of the candidate image onto the mine subspace and projecting of the candidate image onto the non-mine subspace.
2. The method of
receiving the candidate image; and
performing shape normalization on the received candidate image.
3. The method of
5. The method of
wherein generating the non-mine subspace using principal component analysis comprises generating the mine subspace based on twenty principal components.
6. The method of
7. The method of
Pr(x)=(2π)−d/2|Σ|−1/2exp(−1/2(x−μ)TΣ−1(x−μ)) where xε2 is a stochastic feature vector of a mine, Σ is a covariance matrix of data x, μ is a mean of data x, d is dimensions of vector x, and T is a transpose operator.
9. The article of
Wherein the instructions causing a machine to generate the non-mine subspace using principal component analysis comprise instructions causing a machine to generate the mine subspace based on twenty principal components.
10. The article of
11. The article of
Pr(x)=(2π)−d/2|Σ|−1/2exp(−1/2(x−μ)TΣ−1(x−μ)) where xε32 is a stochastic feature vector of a mine, Σ is a covariance matrix of data x, μ is a mean of data x, d is dimensions of vector x, and T is a transpose operator.
13. The apparatus of
14. The apparatus of
wherein circuitry to generate the non-mine subspace using principal component analysis comprises circuitry to generate the mine subspace based on twenty principal components.
15. The apparatus of
16. The apparatus of
Pr(x)=(2π)−d/2|Σ|−1/2exp(−1/2(x−μ)TΣ−1(x−μ)) where xε32 is a stochastic feature vector of a mine, Σ is a covariance matrix of data x, μ is a mean of data x, d is dimensions of vector x, and T is a transpose operator.
18. The method of
wherein generating the non-mine subspace using principal component analysis comprises generating the mine subspace based on twenty principal components.
19. The method of
|
Traditionally, mine detection techniques using sonar imagery have used image processing and detection methods that often rely on preset segmentation and predetermined feature extraction methods followed by classification methods (e.g., neural networks, Bayesian networks and so forth). These classification methods have the disadvantage of not providing good detection or classification accuracy.
In one aspect, a method to reduce false alarms in identifying whether a candidate image is from an object class includes projecting the candidate image onto an object class subspace and projecting the candidate image onto a non-object class subspace. The method also includes determining whether the candidate image is from the object class using a Bayesian decision function based on the projections on the object class subspace and the non-object class subspace.
In another aspect, a method to reduce false alarms in identifying whether a candidate mine image is from a mine class includes projecting the candidate mine image onto a mine subspace and projecting the candidate mine image onto a non-mine subspace. The method also includes determining whether the candidate mine image represents a mine using a Bayesian decision function based on the projections on the mine class subspace and the non-mine class subspace.
In a further aspect, an article includes a machine-readable medium that stores executable instructions to reduce false alarms in identifying whether a candidate mine image is from a mine class. The instructions causing a machine to generate a mine subspace using principal component analysis on images of mines, generate a non-mine subspace using principal component analysis on images determined to be false positives of mines, project the candidate mine image onto the mine subspace, project the candidate mine image onto the non-mine subspace and determine whether the candidate mine image represents a mine using a Bayesian decision function based on the projections on the mine class subspace and the non-mine class subspace.
In a still further aspect, an apparatus to reduce false alarms in identifying whether a candidate mine image is from a mine class includes circuitry to project the candidate mine image onto the mine subspace, project the candidate mine image onto the non-mine subspace and determine whether the candidate mine image represents a mine using a Bayesian decision function based on the projections on the mine class subspace and the non-mine class subspace. The apparatus further includes circuitry to remove similar components from the mine class subspace and the non-mine class subspace so that each subspace is disjoint from each other.
In contrast to prior art techniques, described herein is an approach to identifying mines; however, this approach may be applied to identifying objects other than mines.
Referring to
In one example, the sonar sensor system 14 may include one or more sonar sensors, such as sonar buoys. In another example, the sonar sensor system 14 is a sonar sensor located aboard a submarine. In one example, the network 16 is a wireless network.
The IPS 12 may be located on a ground-based platform (e.g., in a building, in a vehicle and so forth), a space-based platform (e.g., a satellite, a space-vehicle and so forth), a sea-based platform (e.g., a ship, a submarine, a buoy, an anchored sea structure, a torpedo, an undersea robot vehicle and so forth) or on an air-based platform (e.g., an aircraft, a helicopter, a missile and so forth).
In one example, the IPS 12 may be co-located (i.e., on the same platform) with the sonar sensor system 14. In other examples, the IPS 12 is not co-located with the sonar sensor system 14.
As will be shown below, a received image of a candidate object is converted to a candidate mine image patch which is used to determine whether the candidate object is a mine or not based on a mine image subspace formed from mine image patches of known mines. In particular, a distance, d, is determined between a candidate mine image patch 100 (
Referring to
The following is mathematical support that a mine image subspace, in particular an eigen-subspace (based on eigenvalues and eigenvectors), may be used to model mines rather than using an entire space of images. For example, let an image, M(x,y), be a two-dimensional N by N array of intensity values, that is, a vector of dimension N2. Mine image patches are treated as squares for the sake of simplicity here, although generally mine images are “shape-normalized” to rectangular grid dimensions of 32 by 128, describing a vector of dimension 4096. Of note is that the shape-normalized images of mines, being similar in overall configuration, will not be randomly distributed in this high dimensional space but instead form a smaller subspace. Principal Components Analysis (PCA) may be used to identify a small number of basis vectors that best capture the distribution of mine images within the larger image space.
Let {Mi}i=1 . . . T with MiεN
where T is the number of data points. Each mine image deviates from the average by mi=Mi−ψ. PCA is applied to the mean subtracted set described by {mi}i=1 . . . T in search for a set of orthonormal basis vectors {ui}i=1 . . . T and associated eigenvalues {λi}i=1 . . . T that optimally, in a least squares sense, describe the distribution of the random variable m εN
where A=[m1 m2 m3 . . . mn]/√{square root over (T)} is a block compositional matrix whose column i corresponds to vector mi and A* is the conjugate transpose of the matrix A. The covariance matrix C is an N2×N2 matrix. Note that eigenvector computation for this size matrix is not a computationally feasible task. However, assuming that the number of data points T in the sample space is far less than N2, the covariance matrix will have a maximum of T−1 distinct and meaningful eigenvectors.
Now consider the eigenvectors vi of A*A such that A*Avi=μivi. Pre-multiplying both sides by A results in AA*Avi=μiAvi, from which ui=Avi are the eigenvectors of the original Covariance matrix C=AA*. Thus, an alternative matrix L=A*A may be constructed and the T eigenvectors vi of L may be determined. This transposition analysis reduces the computation from an intractable task to a feasible task. In reality the number of samples is far smaller than the dimension of the image space, so this is a useful step in image-based eigenspace analysis.
For example, if the eigen-subspace is represented as Un where n is the number of dimensions and the total space is represented by n where m is the number of dimensions, then Un ⊂n and n<<m. Thus, eigenvectors of L span a basis set which describe the normalized mine images. For example, of this set, picking the top 8 eigenvectors (n=8) associated with the largest 8 eigenvalues forms a lower dimensional subspace U8 to model the mine image subspace than the total space.
Based on the preceding mathematical support for using a mine image subspace rather than the entire higher dimensional space, the mine image subspace may be constructed using known images of mines.
Referring back to
Process 60 performs shape-normalization on the received image (72). For example, IPS 12 normalizes the shape of the received image to fit within the boundaries of a mine image patch (see
Process 60 projects the candidate mine image patch onto the mine subspace (76). In one example, the candidate mine image patch 100 (
wj=u*j(y−ψ), for j=1, . . . , 8
Process 60 determines if the candidate mine image patch includes a mine (82). In one example, the candidate mine image patch 100 and the mine subspace image 96 are used to determine if the received image includes a mine (
Referring to
In one example, the set of known mines is generated using side-scan sonar images of actual mines. Side-scan refers to scan configuration where the sonar sensor is along a horizontal strip on the side of the under water vehicle. In one particular example, over two hundred mine images are identified and segmented out of a set of sonar imagery. Each mine image is shape-normalized and hand-annotated with the six vertices 94a-94f describing a convex polygonal shape of the mine contained within the mine patch 90. The shape-normalized mine patches are used to construct the mine image subspace. One example of a mine image subspace representation is a mine image subspace 96 having mine image patches 90a-90i shown in
Referring to
In another example, each mine is shaped as a polygon. In one example, the mine has six vertices 94a-94f (
Process 200 performs subspace identification (208). For example, a distribution is determined from a collection of known mine image patches. By determining the distribution, it is possible to determine what subspace from the entire image space mine image patches occupy. This subspace becomes the mine image subspace.
Process 200 performs subspace distance statistical analysis (212). For example, a distance threshold is determined by measuring statistics between image patches that are known to be mines and image patches that are known not to be mines to determine a distance threshold. As will be shown below, a distance between a candidate mine image patch 100 and the mine image subspace determines the likelihood that the candidate mine image patch contains a mine. For example, moving closer towards the mine image subspace from a far distance, what distance do objects identified as non-mines become identified as mines. In one example, a probability density is determined that the object is a mine and a probability distance is determined that the object is a non-mine, each probability density is a function of distance from the mine image subspace 96.
Referring to
For example, referring to
The candidate mine image 100 is separated from the mine subspace 96 by a distance, d. The distance, d=∥y−yproj∥, provides a good measure of determining whether the candidate mine image patch 100 contains a mine or not. For example, if the distance is small then there is a higher likelihood that the candidate mine image patch 100 contains a mine. Likewise, a very large distance indicates a less likelihood that the candidate mine image patch 100 contains a mine.
Referring
One difference between the approach described herein and the prior art is that the features, such as the shapes of polygons and image pixel content (colors) corresponding to within those polygonal regions, and model for a mine is determined directly from the known mine image data. Therefore, the selected features and the model are optimal in the sense that they best represent a given data set (e.g., a particular mine); whereas pre-fixed features and models may be optimal in a general sense but not for a particular data set (e.g., a particular mine). The approach previously described, which extends the process of modeling with the addition of shape parameterization, has the advantage of better modeling appearance variations due to changing mine geometry and thereby improving the detection/classification accuracy.
Referring to
In other embodiments, it is possible to reduce the number of false alarms (e.g., the number of false positives). For example, a method that extracts discriminative subspaces that describe two distinct classes (e.g., a mine image patch class and a non-mine image patch class (extracted from false alarms)) and uses higher order statistical information contained in the image patches for each class to determine whether a candidate mine image patch is a true mine or a false alarm (non-mine).
Referring to
A sonar image of a candidate object is received (308) and shape normalized (312) to form a candidate mine image patch. In one example, processing blocks 308 and 312 are similar as processing blocks 68 and 72 in
Using the probability models for the mine and non-mine subspaces, an object is determined to be either a mine or a non-mine based on the projection (322). For example, a Bayesian decision function is used to determine whether a candidate mine image patch is a true mine or a false alarm. In one particular example, the Bayesian decision function is logarithmic likelihood Of Vcandidate corresponding to a mine and is represented as:
Log Lik(Vcandidate)=Log((Pr(Vcandidate|mine)/Pr(Vcandidate|Non-mine))
where Pr(Vcandidate|mine) is the probability that a candidate mine image patch includes a mine feature given that it belongs to a mine class and Pr(Vcandidate|Non-mine) is the probability that a candidate mine image patch includes a mine feature given that it belongs to a non-mine class. If LogLik (Vcandidate) is positive then the candidate image patch is likely to be a mine and if LogLik (Vcandidate) is negative then the candidate image patch is likely to be a non-mine.
Referring to
Principal component analysis (PCA) is used extract discriminate subspaces from each of the two classes (408). For example, each image subspace (mine or non-mine) may be represented using twenty-dimensional vectors (components), for example. Each subspace may be formed similar to mine subspace 96.
Similar components between each of the mine subspace and the non-mine subspace are determined (412) and removed to form disjoint subspaces (418). Disjoint subspaces are obtained by removing, from each subspace, components with similar counterparts in the opposite class thereby leaving behind subspaces with components that are descriptive of their own class, while at the same time being distinctly non-descriptive of the opposite class. For example, consider an analogy of a face detector that detects glasses on a person. One class is a group of people with glasses and another class is of people with no glasses (previously detected false positives). Therefore, components of a person's face above the eyebrows or below the nose would be eliminated because none of those components would be descriptive of either class.
In one example, similar components are removed if the mutual information between the subspace classes are greater than a threshold. For example, if xi represents components in the mine subspace and yk represents a components in the non-mine subspace and I( ) is the mutual information function, then I(xi;yk)>threshold where,
and p(x) is a probability density of the mine subspace, p(y) is the probability density of the non-mine subspace and p(x, y) is probability density of x and y jointly (i.e., a joint probability distribution for any two random variables x and y).
Mutual information takes into account the actual probability distributions of the underlying signal and does not assume a Gaussian distribution, and is, therefore, able to capture higher order statistical information content beyond second order mean and variances. Mutual information is also invariant to a range of image transformations and therefore more conducive to capturing subtle structure that might exist in geometrically incoherent images.
Each mine image patch is projected on to the disjoint mine subspace and the disjoint mine subspace (422). For example, the projections are performed similar to
Pr(x)=(2π)−d/2|Σ|−1/2exp(−½(x−μ)TΣ−1(x−μ))
where
xε32 is a stochaistic (feature vector of a mine)
and where Σ is a covariance matrix of data x, μ is a mean of data x, d is [dimensions of vector x and T is a transpose operator. In one example, Pr(x) is derived from Pr(x|mine) by integrating over the variable “mine.” In another example, Pr(x) is derived from Pr(x|mine) collecting data about x for cases where the condition or hypothesis “mine” is known to be true.
Each non-mine image patch is projected on to the disjoint mine subspace and the disjoint mine subspace (432). For example, the projections form two sixteen dimensional vectors. A probability model is generated for the non-mine subclass (438). For example, the two sixteen dimensional vectors are combined to form a thirty-two dimensional probability distribution for a non-mine class. From the 32-dimensional probability distribution for a non-mine class, the conditional probability, Pr(y|non-mine), is determined from:
Pr(y)=(2π)−d/2|Σ|−1/2exp(−½(y−μ)TΣ−1(y−μ))
where
y ε32 is a stochastic (feature vector of non-mine).
With the two conditional probability models for each class, Pr(x|Mine) and Pr(y|Non-mine), determined, the likelihood ratios for a candidate mine image may be also determined in order to form a decision theoretic measure of whether a feature representation of a candidate image patch is likely to have come from a true mine contact or a false-alarm.
Using the process 300 in mine detection, experimental results indicate a significant reduction in false alarms. For example, a benchmarking data set was taken from 69 true mine contacts, where 65 were correctly identified as mines while 4 were incorrectly identified as non-mines, and the benchmarking data set was taken from 80 non-mine contacts identified as false alarms where 77 were identified as false alarms and 3 were incorrectly identified a mines. 81 image patches that were previously identified as false alarms received (and not the same false alarm image patches in the benchmarking data set) were used as candidate image patches in process 300. In the results, 56 of the 81 image patches (69.1%) were correctly identified as non-mines.
The processes described herein (e.g., process 60 and process 300) are not limited to use with the hardware and software of
The system may be implemented, at least in part, via a computer program product, (i.e., a computer program tangibly embodied in an information carrier (e.g., in a machine-readable storage device)), for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers)). Each such program may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the programs may be implemented in assembly or machine language. The language may be a compiled or an interpreted language and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network. A computer program may be stored on a storage medium or device (e.g., CD-ROM, hard disk, or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform a process (e.g., a process 60 and a process 300). The processes may also be implemented as a machine-readable storage medium, configured with a computer program, where upon execution, instructions in the computer program cause the computer to operate in accordance with the respective process.
The processes described herein are not limited to the specific embodiments described herein. For example, the processes are not limited to the specific processing order of
Even though
Elements of different embodiments described herein may be combined to form other embodiments not specifically set forth above. Other embodiments not specifically described herein are also within the scope of the following claims.
Patent | Priority | Assignee | Title |
10832414, | Nov 21 2017 | Zoox, Inc. | Sensor data segmentation |
11157527, | Feb 20 2018 | ZOOX, INC | Creating clean maps including semantic information |
11475573, | Nov 21 2017 | Zoox, Inc. | Sensor data segmentation |
11798169, | Nov 21 2017 | Zoox, Inc. | Sensor data segmentation |
Patent | Priority | Assignee | Title |
4951268, | May 10 1988 | Thomson-CSF | Method for the sonar classification of underwater objects, notably moored mines |
5181254, | Dec 14 1990 | Northrop Grumman Systems Corporation | Method for automatically identifying targets in sonar images |
5937078, | Apr 10 1996 | The United States of America as represented by the Secretary of the Navy | Target detection method from partial image of target |
6700833, | Sep 17 2001 | Frantorf Investments GmbH, LLC | Acoustical imaging interferometer for detection of buried underwater objects |
6842401, | Apr 06 2000 | TeraTech Corporation | Sonar beamforming system |
6854410, | Nov 24 2003 | The United States of America as represented by the Secretary of the Navy | Underwater investigation system using multiple unmanned vehicles |
6868041, | May 01 2002 | Quester Tangent Corporation | Compensation of sonar image data primarily for seabed classification |
7343362, | Oct 07 2003 | ARMY, UNITED STATES OF AMERICA AS REPRESENTED BY THE SECRETARY OF THE | Low complexity classification from a single unattended ground sensor node |
20030046068, | |||
20030095140, | |||
20040066966, | |||
20050058021, | |||
20050220191, | |||
20060067573, | |||
20060110046, | |||
20080107311, | |||
20080260239, | |||
20090060353, | |||
20090232406, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Mar 17 2008 | Raytheon Company | (assignment on the face of the patent) | / | |||
Oct 22 2008 | SAISAN, PAYAM | Raytheon Company | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 021756 | /0183 |
Date | Maintenance Fee Events |
Jan 14 2014 | ASPN: Payor Number Assigned. |
Aug 03 2017 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Jul 21 2021 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Date | Maintenance Schedule |
Feb 18 2017 | 4 years fee payment window open |
Aug 18 2017 | 6 months grace period start (w surcharge) |
Feb 18 2018 | patent expiry (for year 4) |
Feb 18 2020 | 2 years to revive unintentionally abandoned end. (for year 4) |
Feb 18 2021 | 8 years fee payment window open |
Aug 18 2021 | 6 months grace period start (w surcharge) |
Feb 18 2022 | patent expiry (for year 8) |
Feb 18 2024 | 2 years to revive unintentionally abandoned end. (for year 8) |
Feb 18 2025 | 12 years fee payment window open |
Aug 18 2025 | 6 months grace period start (w surcharge) |
Feb 18 2026 | patent expiry (for year 12) |
Feb 18 2028 | 2 years to revive unintentionally abandoned end. (for year 12) |