In one embodiment, a system includes: a processor; and logic in and/or executable by the processor to cause the processor to: generate a first feature vector based on a digital image captured by a mobile device; compare the first feature vector to a plurality of reference feature matrices; classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison; determine one or more object features of the object based at least in part on the particular object class; and detect one or more additional objects belonging to the particular object class based on the determined object feature(s). The one or more additional objects are depicted either in the digital image or another digital image received by the mobile device. Corresponding computer program products are also disclosed.

Patent
   10127441
Priority
Mar 13 2013
Filed
Dec 20 2016
Issued
Nov 13 2018
Expiry
Mar 18 2033

TERM.DISCL.
Extension
5 days
Assg.orig
Entity
Large
7
750
currently ok
1. A computer program product comprising: a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to:
generate a first feature vector based on a digital image captured by a mobile device;
compare the first feature vector to a plurality of reference feature matrices;
classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison;
predict an identity of text depicted on the object using optical character recognition (OCR), wherein the prediction is based at least in part on the particular object class;
modify at least one identity predicted using OCR based on an expected format of text depicted on the object, wherein the expected format is determined based at least in part on the particular object class;
determine one or more object features of the object based at least in part on the particular object class; and
detect one or more additional objects belonging to the particular object class based on the determined object feature(s);
wherein the one or more additional objects are depicted either in the digital image or another digital image received by the mobile device.
13. A computer program product comprising: a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to:
generate a first feature vector based on a digital image captured by a mobile device;
compare the first feature vector to a plurality of reference feature matrices;
classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison;
determine one or more object features of the object based at least in part on the particular object class; and
detect one or more additional objects belonging to the particular object class based on the determined object feature(s);
wherein the one or more additional objects are depicted either in the digital image or another digital image received by the mobile device;
wherein the one or more object features comprise an object class mask, and
wherein the computer program product further comprises program code readable/executable by the processor to binarize the digital image based at least in part on:
determining the object class mask;
applying the object class mask to the digital image; and
thresholding a subregion of the digital image based on the object class mask.
7. A computer program product comprising: a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to:
generate a first feature vector based on a digital image captured by a mobile device;
compare the first feature vector to a plurality of reference feature matrices;
classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison;
determine one or more object features of the object based at least in part on the particular object class; and
detect one or more additional objects belonging to the particular object class based on the determined object feature(s);
wherein the one or more additional objects are depicted either in the digital image or another digital image received by the mobile device;
wherein the one or more object features comprise an object color scheme; and
wherein the computer program product further comprises program code readable/executable by the processor to binarize the digital image based at least in part on:
determining the object color scheme;
adjusting one or more binarization parameters based on the object color scheme; and
thresholding the digital image using the one or more adjusted binarization parameters.
2. The computer program product as recited in claim 1, further comprising program code readable/executable by the processor to: generate a first representation of the digital image, the first representation being characterized by a reduced resolution,
wherein generating the first representation comprises:
dividing the digital image into a plurality of sections; and
determining, for each section, at least one characteristic value, each characteristic value corresponding to one or more features descriptive of the section; and
wherein the first feature vector is generated based on the first representation of the digital image.
3. The computer program product as recited in claim 1, further comprising program code readable/executable by the processor to: define at least one hyperplane between at least some of the reference feature matrices, each hyperplane defining a boundary between first objects corresponding to one of the reference feature matrices and second objects corresponding to another of the reference feature matrices.
4. The computer program product as recited in claim 1, wherein generating the first feature vector comprises determining a color profile of some or all of the object.
5. The computer program product as recited in claim 1, further comprising program code readable/executable by the processor to: determine a geographical location associated with the mobile device, wherein classifying the object is further based at least in part on the geographical location.
6. The computer program product as recited in claim 1, further comprising program code readable/executable by the processor to:
output an indication of the particular object class to a display of the mobile device; and
receive user input via the display of the mobile device in response to outputting the indication, the user input comprising either a confirmation, a negation, or a modification of the particular object class.
8. The computer program product as recited in claim 7, wherein the one or more object features comprise an object class mask, and
wherein the computer program product further comprises program code readable/executable by the processor to binarize the digital image based at least in part on:
determining the object class mask;
applying the object class mask to the digital image; and
thresholding a subregion of the digital image based on the object class mask
wherein the one or more object features further comprise an object color scheme, and
wherein binarizing the digital image further comprises:
determining the object color scheme;
adjusting one or more binarization parameters based on the object color scheme; and
thresholding the digital image using the one or more adjusted binarization parameters.
9. The computer program product as recited in claim 7, further comprising program code readable/executable by the processor to: generate a first representation of the digital image, the first representation being characterized by a reduced resolution,
wherein generating the first representation comprises:
dividing the digital image into a plurality of sections; and
determining, for each section, at least one characteristic value, each characteristic value corresponding to one or more features descriptive of the section; and
wherein the first feature vector is generated based on the first representation of the digital image.
10. The computer program product as recited in claim 7, further comprising program code readable/executable by the processor to: define at least one hyperplane between at least some of the reference feature matrices, each hyperplane defining a boundary between first objects corresponding to one of the reference feature matrices and second objects corresponding to another of the reference feature matrices.
11. The computer program product as recited in claim 7, wherein generating the first feature vector comprises determining a color profile of some or all of the object.
12. The computer program product as recited in claim 7, further comprising program code readable/executable by the processor to perform at least one of the following functions:
predict an identity of text depicted on the object using optical character recognition (OCR), wherein the prediction is based at least in part on the particular object class; and
modify at least one identity predicted using OCR based on an expected format of text depicted on the object, wherein the expected format is determined based at least in part on the particular object class.
14. The computer program product as recited in claim 13, wherein the one or more object features further comprise an object color scheme, and
wherein binarizing the digital image further comprises:
determining the object color scheme;
adjusting one or more binarization parameters based on the object color scheme; and
thresholding the digital image using the one or more adjusted binarization parameters.
15. The computer program product as recited in claim 13, wherein the one or more object features comprise an object class mask, and
wherein the computer program product further comprises program code
readable/executable by the processor to binarize the digital image based at least in
part on:
determining the object class mask;
applying the object class mask to the digital image; and
thresholding a subregion of the digital image based on the object class mask
wherein the one or more object features further comprise an object color scheme, and
wherein binarizing the digital image further comprises:
determining the object color scheme;
adjusting one or more binarization parameters based on the object color scheme; and
thresholding the digital image using the one or more adjusted binarization parameters.
16. The computer program product as recited in claim 13, further comprising program code readable/executable by the processor to: generate a first representation of the digital image, the first representation being characterized by a reduced resolution,
wherein generating the first representation comprises:
dividing the digital image into a plurality of sections; and
determining, for each section, at least one characteristic value, each characteristic value corresponding to one or more features descriptive of the section; and
wherein the first feature vector is generated based on the first representation of the digital image.
17. The computer program product as recited in claim 13, further comprising program code readable/executable by the processor to: define at least one hyperplane between at least some of the reference feature matrices, each hyperplane defining a boundary between first objects corresponding to one of the reference feature matrices and second objects corresponding to another of the reference feature matrices.
18. The computer program product as recited in claim 13, wherein generating the first feature vector comprises determining a color profile of some or all of the object.
19. The computer program product as recited in claim 13, further comprising program code readable/executable by the processor to perform at least one of the following functions:
predict an identity of text depicted on the object using optical character recognition (OCR), wherein the prediction is based at least in part on the particular object class; and
modify at least one identity predicted using OCR based on an expected format of text depicted on the object, wherein the expected format is determined based at least in part on the particular object class.

This application is related to copending U.S. patent application Ser. No. 13/740,123, filed Jan. 11, 2013; Ser. No. 12/042,774, filed Mar. 5, 2008; and Ser. No. 12/368,685, filed Feb. 10, 2009, each of which is herein incorporated by reference in its entirety. This application is also related to U.S. Pat. No. 7,761,391, granted Jul. 20, 2010 (U.S. patent application Ser. No. 11/752,364, filed May 13, 2007) and U.S. Pat. No. 6,370,277, granted Apr. 9, 2002 (U.S. patent application Ser. No. 09/206,753, filed Dec. 7, 1998), each of which is also herein incorporated by reference in its entirety.

The present invention relates to mobile image capture and image processing, and more particularly to capturing and processing digital images using a mobile device, and classifying objects detected in such digital images.

Digital images having depicted therein an object inclusive of documents such as a letter, a check, a bill, an invoice, etc. have conventionally been captured and processed using a scanner or multifunction peripheral coupled to a computer workstation such as a laptop or desktop computer. Methods and systems capable of performing such capture and processing are well known in the art and well adapted to the tasks for which they are employed.

However, in an era where day-to-day activities, computing, and business are increasingly performed using mobile devices, it would be greatly beneficial to provide analogous document capture and processing systems and methods for deployment and use on mobile platforms, such as smart phones, digital cameras, tablet computers, etc.

A major challenge in transitioning conventional document capture and processing techniques is the limited processing power and image resolution achievable using hardware currently available in mobile devices. These limitations present a significant challenge because it is impossible or impractical to process images captured at resolutions typically much lower than achievable by a conventional scanner. As a result, conventional scanner-based processing algorithms typically perform poorly on digital images captured using a mobile device.

In addition, the limited processing and memory available on mobile devices makes conventional image processing algorithms employed for scanners prohibitively expensive in terms of computational cost. Attempting to process a conventional scanner-based image processing algorithm takes far too much time to be a practical application on modern mobile platforms.

A still further challenge is presented by the nature of mobile capture components (e.g. cameras on mobile phones, tablets, etc.). Where conventional scanners are capable of faithfully representing the physical document in a digital image, critically maintaining aspect ratio, dimensions, and shape of the physical document in the digital image, mobile capture components are frequently incapable of producing such results.

Specifically, images of documents captured by a camera present a new line of processing issues not encountered when dealing with images captured by a scanner. This is in part due to the inherent differences in the way the document image is acquired, as well as the way the devices are constructed. The way that some scanners work is to use a transport mechanism that creates a relative movement between paper and a linear array of sensors. These sensors create pixel values of the document as it moves by, and the sequence of these captured pixel values forms an image. Accordingly, there is generally a horizontal or vertical consistency up to the noise in the sensor itself, and it is the same sensor that provides all the pixels in the line.

In contrast, cameras have many more sensors in a nonlinear array, e.g., typically arranged in a rectangle. Thus, all of these individual sensors are independent, and render image data that is not typically of horizontal or vertical consistency. In addition, cameras introduce a projective effect that is a function of the angle at which the picture is taken. For example, with a linear array like in a scanner, even if the transport of the paper is not perfectly orthogonal to the alignment of sensors and some skew is introduced, there is no projective effect like in a camera. Additionally, with camera capture, nonlinear distortions may be introduced because of the camera optics.

Conventional image processing algorithms designed to detect documents in images captured using traditional flat-bed and/or paper feed scanners may also utilize information derived from page detection to attempt to classify detected documents as members of a particular document class. However, due to the unique challenges introduced by virtue of capturing digital images using cameras of mobile devices, these conventional classification algorithms perform inadequately and are incapable of robustly classifying documents in such digital images.

Moreover, even when documents can be properly classified, the hardware limitations of current mobile devices make performing classification using the mobile device prohibitively expensive from a computational efficiency standpoint.

In view of the challenges presented above, it would be beneficial to provide an image capture and processing algorithm and applications thereof that compensate for and/or correct problems associated with image capture, processing and classification using a mobile device, while maintaining a low computational cost via efficient processing methods.

Moreover, it would be a further improvement in the field to provide object classification systems, methods and computer program products capable of robustly assigning objects to a particular class of objects and utilize information known about members of the class to further address and overcome unique challenges inherent to processing images captured using a camera of a mobile device.

In one embodiment, a system includes: a processor; and logic in and/or executable by the processor to cause the processor to: generate a first feature vector based on a digital image captured by a mobile device; compare the first feature vector to a plurality of reference feature matrices; classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison; determine one or more object features of the object based at least in part on the particular object class; and detect one or more additional objects belonging to the particular object class based on the determined object feature(s). The one or more additional objects are depicted either in the digital image or another digital image received by the mobile device.

In another embodiment, a computer program product includes: a computer readable storage medium having program code embodied therewith. The program code is readable/executable by a processor to: generate a first feature vector based on a digital image captured by a mobile device; compare the first feature vector to a plurality of reference feature matrices; classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison; determine one or more object features of the object based at least in part on the particular object class; and detect one or more additional objects belonging to the particular object class based on the determined object feature(s). The one or more additional objects are depicted either in the digital image or another digital image received by the mobile device.

Other aspects and embodiments of the inventive concepts presented above will become clear from carefully reviewing the following detailed descriptions in light of the drawings.

FIG. 1 illustrates a network architecture, in accordance with one embodiment.

FIG. 2 shows a representative hardware environment that may be associated with the servers and/or clients of FIG. 1, in accordance with one embodiment.

FIG. 3A depicts a digital image of an object, according to one embodiment

FIG. 3B depicts a schematic representation of the digital image shown in FIG. 3A divided into a plurality of sections for generating a first representation of the digital image, according to one embodiment.

FIG. 3C is depicts a first representation of the digital image shown in FIG. 3A, the first representation being characterized by a reduced resolution relative to the resolution of the digital image.

FIG. 4A is a schematic representation of a plurality of subregions depicted in a digital image of a document, according to one embodiment.

FIG. 4B is a masked representation of the digital image shown in FIG. 4A, according to one embodiment.

FIG. 4C is a masked representation of the digital image shown in FIG. 4A, according to one embodiment.

FIG. 4D is a masked representation of the digital image shown in FIG. 4A, according to one embodiment.

FIG. 5 is a flowchart of a method, according to one embodiment.

FIG. 6 is a flowchart of a method, according to one embodiment.

The following description is made for the purpose of illustrating the general principles of the present invention and is not meant to limit the inventive concepts claimed herein. Further, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations.

Unless otherwise specifically defined herein, all terms are to be given their broadest possible interpretation including meanings implied from the specification as well as meanings understood by those skilled in the art and/or as defined in dictionaries, treatises, etc.

It must also be noted that, as used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless otherwise specified.

The present application refers to image processing of images (e.g. pictures, figures, graphical schematics, single frames of movies, videos, films, clips, etc.) captured by cameras, especially cameras of mobile devices. As understood herein, a mobile device is any device capable of receiving data without having power supplied via a physical connection (e.g. wire, cord, cable, etc.) and capable of receiving data without a physical data connection (e.g. wire, cord, cable, etc.). Mobile devices within the scope of the present disclosures include exemplary devices such as a mobile telephone, smartphone, tablet, personal digital assistant, iPod®, iPad®, BLACKBERRY® device, etc.

However, as it will become apparent from the descriptions of various functionalities, the presently disclosed mobile image processing algorithms can be applied, sometimes with certain modifications, to images coming from scanners and multifunction peripherals (MFPs). Similarly, images processed using the presently disclosed processing algorithms may be further processed using conventional scanner processing algorithms, in some approaches.

Of course, the various embodiments set forth herein may be implemented utilizing hardware, software, or any desired combination thereof. For that matter, any type of logic may be utilized which is capable of implementing the various functionality set forth herein.

One benefit of using a mobile device is that with a data plan, image processing and information processing based on captured images can be done in a much more convenient, streamlined and integrated way than previous methods that relied on presence of a scanner. However, the use of mobile devices as document(s) capture and/or processing devices has heretofore been considered unfeasible for a variety of reasons.

In one approach, an image may be captured by a camera of a mobile device. The term “camera” should be broadly interpreted to include any type of device capable of capturing an image of a physical object external to the device, such as a piece of paper. The term “camera” does not encompass a peripheral scanner or multifunction device. Any type of camera may be used. Preferred embodiments may use cameras having a higher resolution, e.g. 8 MP or more, ideally 12 MP or more. The image may be captured in color, grayscale, black and white, or with any other known optical effect. The term “image” as referred to herein is meant to encompass any type of data corresponding to the output of the camera, including raw data, processed data, etc.

General Embodiments

In one general embodiment a method includes: receiving a digital image captured by a mobile device; and using a processor of the mobile device: generating a first representation of the digital image, the first representation being characterized by a reduced resolution; generating a first feature vector based on the first representation; comparing the first feature vector to a plurality of reference feature matrices; and classifying an object depicted in the digital image as a member of a particular object class based at least in part on the comparing.

In another general embodiment, a method includes: generating a first feature vector based on a digital image captured by a mobile device; comparing the first feature vector to a plurality of reference feature matrices; classifying an object depicted in the digital image as a member of a particular object class based at least in part on the comparing; and determining one or more object features of the object based at least in part on the particular object class; and performing at least one processing operation using a processor of a mobile device, the at least one processing operation selected from a group consisting of: detecting the object depicted in the digital image based at least in part on the one or more object features; rectangularizing the object depicted in the digital image based at least in part on the one or more object features; cropping the digital image based at least in part on the one or more object features; and binarizing the digital image based at least in part on the one or more object features.

In still another general embodiment, a system includes a processor; and logic in and/or executable by the processor to cause the processor to: generate a first representation of a digital image captured by a mobile device; generate a first feature vector based on the first representation; compare the first feature vector to a plurality of reference feature matrices; and classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison.

In still yet another general embodiment, a computer program product includes a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to: generate a first representation of a digital image captured by a mobile device; generate a first feature vector based on the first representation; compare the first feature vector to a plurality of reference feature matrices; and classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as “logic,” “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, processor, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband, as part of a carrier wave, an electrical connection having one or more wires, an optical fiber, etc. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

FIG. 1 illustrates an architecture 100, in accordance with one embodiment. As shown in FIG. 1, a plurality of remote networks 102 are provided including a first remote network 104 and a second remote network 106. A gateway 101 may be coupled between the remote networks 102 and a proximate network 108. In the context of the present architecture 100, the networks 104, 106 may each take any form including, but not limited to a LAN, a WAN such as the Internet, public switched telephone network (PSTN), internal telephone network, etc.

In use, the gateway 101 serves as an entrance point from the remote networks 102 to the proximate network 108. As such, the gateway 101 may function as a router, which is capable of directing a given packet of data that arrives at the gateway 101, and a switch, which furnishes the actual path in and out of the gateway 101 for a given packet.

Further included is at least one data server 114 coupled to the proximate network 108, and which is accessible from the remote networks 102 via the gateway 101. It should be noted that the data server(s) 114 may include any type of computing device/groupware. Coupled to each data server 114 is a plurality of user devices 116. Such user devices 116 may include a desktop computer, lap-top computer, hand-held computer, printer or any other type of logic. It should be noted that a user device 111 may also be directly coupled to any of the networks, in one embodiment.

A peripheral 120 or series of peripherals 120, e.g., facsimile machines, printers, networked and/or local storage units or systems, etc., may be coupled to one or more of the networks 104, 106, 108. It should be noted that databases and/or additional components may be utilized with, or integrated into, any type of network element coupled to the networks 104, 106, 108. In the context of the present description, a network element may refer to any component of a network.

According to some approaches, methods and systems described herein may be implemented with and/or on virtual systems and/or systems which emulate one or more other systems, such as a UNIX system which emulates an IBM z/OS environment, a UNIX system which virtually hosts a MICROSOFT WINDOWS environment, a MICROSOFT WINDOWS system which emulates an IBM z/OS environment, etc. This virtualization and/or emulation may be enhanced through the use of VMWARE software, in some embodiments.

In more approaches, one or more networks 104, 106, 108, may represent a cluster of systems commonly referred to as a “cloud.” In cloud computing, shared resources, such as processing power, peripherals, software, data, servers, etc., are provided to any system in the cloud in an on-demand relationship, thereby allowing access and distribution of services across many computing systems. Cloud computing typically involves an Internet connection between the systems operating in the cloud, but other techniques of connecting the systems may also be used.

FIG. 2 shows a representative hardware environment associated with a user device 116 and/or server 114 of FIG. 1, in accordance with one embodiment. Such figure illustrates a typical hardware configuration of a workstation having a central processing unit 210, such as a microprocessor, and a number of other units interconnected via a system bus 212.

The workstation shown in FIG. 2 includes a Random Access Memory (RAM) 214, Read Only Memory (ROM) 216, an I/O adapter 218 for connecting peripheral devices such as disk storage units 220 to the bus 212, a user interface adapter 222 for connecting a keyboard 224, a mouse 226, a speaker 228, a microphone 232, and/or other user interface devices such as a touch screen and a digital camera (not shown) to the bus 212, communication adapter 234 for connecting the workstation to a communication network 235 (e.g., a data processing network) and a display adapter 236 for connecting the bus 212 to a display device 238.

The workstation may have resident thereon an operating system such as the Microsoft Windows® Operating System (OS), a MAC OS, a UNIX OS, etc. It will be appreciated that a preferred embodiment may also be implemented on platforms and operating systems other than those mentioned. A preferred embodiment may be written using JAVA, XML, C, and/or C++ language, or other programming languages, along with an object oriented programming methodology. Object oriented programming (OOP), which has become increasingly used to develop complex applications, may be used.

An application may be installed on the mobile device, e.g., stored in a nonvolatile memory of the device. In one approach, the application includes instructions to perform processing of an image on the mobile device. In another approach, the application includes instructions to send the image to a remote server such as a network server. In yet another approach, the application may include instructions to decide whether to perform some or all processing on the mobile device and/or send the image to the remote site.

In various embodiments, the presently disclosed methods, systems and/or computer program products may utilize and/or include any of the functionalities disclosed in related U.S. patent application Ser. No. 13/740,123, filed Jan. 11, 2013. For example, digital images suitable for processing according to the presently disclosed algorithms may be subjected to any image processing operations disclosed in the aforementioned patent application, such as page detection, rectangularization, detection of uneven illumination, illumination normalization, resolution estimation, blur detection, etc.

In more approaches, the presently disclosed methods, systems, and/or computer program products may be utilized with, implemented in, and/or include one or more user interfaces configured to facilitate performing any functionality disclosed herein and/or in the aforementioned related patent application, such as an image processing mobile application, a case management application, and/or a classification application, in multiple embodiments.

In still more approaches, the presently disclosed systems, methods and/or computer program products may be advantageously applied to one or more of the use methodologies and/or scenarios disclosed in the aforementioned related patent application, among others that would be appreciated by one having ordinary skill in the art upon reading these descriptions.

It will further be appreciated that embodiments presented herein may be provided in the form of a service deployed on behalf of a customer to offer service on demand.

Document Classification

In accordance with one inventive embodiment commensurate in scope with the present disclosures, as shown in FIG. 5, a method 500 is shown. The method 500 may be carried out in any desired environment, and may include embodiments and/or approaches described in relation to FIGS. 1-4D, among others. Of course, more or less operations than those shown in FIG. 5 may be performed in accordance method 500 as would be appreciated by one of ordinary skill in the art upon reading the present descriptions.

In operation 502, a digital image captured by a mobile device is received.

In one embodiment the digital image may be characterized by a native resolution. As understood herein, a “native resolution” may be an original, native resolution of the image as originally captured, but also may be a resolution of the digital image after performing some pre-classification processing such as any of the image processing operations described above and in copending U.S. patent application Ser. No. 13/740,123, filed Jan. 11, 2013, a virtual re-scan (VRS) processing as disclosed in related U.S. Pat. No. 6,370,277, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions. In one embodiment, the native resolution is approximately 500 pixels by 600 pixels (i.e. a 500×600 digital image) for a digital image of a driver license subjected to processing by VRS before performing classification. Moreover, the digital image may be characterized as a color image in some approaches, and in still more approaches may be a cropped-color image, i.e. a color image depicting substantially only the object to be classified, and not depicting image background.

In operation 504, a first representation of the digital image is generated using a processor of the mobile device. The first representation may be characterized by a reduced resolution, in one approach. As understood herein, a “reduced resolution” may be any resolution less than the native resolution of the digital image, and more particularly any resolution suitable for subsequent analysis of the first representation according to the principles set forth herein.

In preferred embodiments, the reduced resolution is sufficiently low to minimize processing overhead and maximize computational efficiency and robustness of performing the algorithm on the respective mobile device, host device and/or server platform. For example, in one approach the first representation is characterized by a resolution of about 25 pixels by 25 pixels, which has been experimentally determined to be a particularly efficient and robust reduced resolution for processing of relatively small documents, such as business cards, driver licenses, receipts, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.

Of course, in other embodiments, different resolutions may be employed without departing from the scope of the present disclosure. For example, classification of larger documents or objects may benefit from utilizing a higher resolution such as 50 pixels by 50 pixels, 100 pixels by 100 pixels, etc. to better represent the larger document or object for robust classification and maximum computational efficiency. The resolution utilized may or may not have the same number of pixels in each dimension. Moreover, the most desirable resolution for classifying various objects within a broad range of object classes may be determined experimentally according to a user's preferred balance between computational efficiency and classification robustness. In still more embodiments, any resolution may be employed, and preferably the resolution is characterized by comprising between 1 pixel and about 1000 pixels in a first dimension, and between 1 and about 1000 pixels in a second dimension.

One exemplary embodiment of inputs, outputs and/or results of a process flow for generating the first representation will now be presented with particular reference to FIGS. 3A-3C, which respectively depict: a digital image before being divided into sections (e.g. digital image 300 as shown in FIG. 3A); a digital image divided into sections (e.g. sections 304 as shown in FIG. 3B); and a first representation of the digital image (e.g. representation 310 as shown in FIG. 3C) characterized by a reduced resolution.

As shown in FIGS. 3A-3B, a digital image 300 captured by a mobile device may be divided into a plurality of sections 304. Each section may comprise a plurality of pixels 306, which may comprise a substantially rectangular grid of pixels such that the section has dimensions of pS(x) horizontal pixels (pS(x)=4 as shown in FIG. 3B) by pS(y) vertical pixels (pS(y)=4 as shown in FIG. 3B).

In one general embodiment, a first representation may be generated by dividing a digital image R (having a resolution of xR pixels by yR pixels) into Sx horizontal sections and Sy vertical sections and thus may be characterized by a reduced resolution r of Sx pixels by Sy pixels. Thus, generating the first representation essentially includes generating a less-granular representation of the digital image.

For example, in one approach the digital image 300 is divided into S sections, each section 304 corresponding to one portion of an s-by-s grid 302. Generating the first representation involves generating a s-pixel-by-s-pixel first representation 310, where each pixel 312 in the first representation 310 corresponds to one of the S sections 304 of the digital image, and wherein each pixel 312 is located in a position of the first representation 310 corresponding to the location of the corresponding section 304 in the digital image, i.e. the upper-leftmost pixel 312 in the first representation corresponds to the upper-leftmost section 304 in the digital image, etc.

Of course, other reduced resolutions may be employed for the first representation, ideally but not necessarily according to limitations and/or features of a mobile device, host device, and or server platform being utilized to carry out the processing, the characteristics of the digital image (resolution, illumination, presence of blur, etc.) and/or characteristics of the object which is to be detected and/or classified (contrast with background, presence of text or other symbols, closeness of fit to a general template, etc.) as would be understood by those having ordinary skill in the art upon reading the present descriptions.

In some approaches, generating the first representation may include one or more alternative and/or additional suboperations, such as dividing the digital image into a plurality of sections. The digital image may be divided into a plurality of sections in any suitable manner, and in one embodiment the digital image is divided into a plurality of rectangular sections. Of course, sections may be characterized by any shape, and in alternative approaches the plurality of sections may or may not represent the entire digital image, may represent an oversampling of some regions of the image, or may represent a single sampling of each pixel depicted in the digital image. In a preferred embodiment, as discussed above regarding FIGS. 3A-3C, the digital image is divided into S substantially square sections 304 to form an s×s grid 302.

In further approaches, generating the first representation may also include determining, for each section of the digital image, at least one characteristic value, where each characteristic value corresponds to one or more features descriptive of the section. Within the scope of the present disclosures, any feature that may be expressed as a numerical value is suitable for use in generating the first representation, e.g. an average brightness or intensity (0-255) across each pixel in the section, an average value (0-255) of each color channel of each pixel in the section, such as an average red-channel value, and average green-channel value, and an average blue-channel value for a red-green-blue (RGB) image, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.

With continuing reference to FIGS. 3A-3C, in some embodiments each pixel 312 of the first representation 310 corresponds to one of the S sections 304 not only with respect to positional correspondence, but also with respect to feature correspondence. For example, in one approach generating the first representation 310 may additionally include determining a characteristic section intensity value iS by calculating the average of the individual intensity values iP of each pixel 306 in the section 304. Then, each pixel 312 in the first representation 310 is assigned an intensity value equal to the average intensity value iS calculated for the corresponding section 304 of the digital image 300. In this manner, the first representation 310 reflects a less granular, normalized representation of the features depicted in digital image 300.

Of course, the pixels 312 comprising the first representation 310 may be represented using any characteristic value or combination of characteristic values without departing from the scope of the presently disclosed classification methods. Further, characteristic values may be computed and/or determined using any suitable means, such as by random selection of a characteristic value from a distribution of values, by a statistical means or measure, such as an average value, a spread of values, a minimum value, a maximum value, a standard deviation of values, a variance of values, or by any other means that would be known to a skilled artisan upon reading the instant descriptions.

In operation 506, a first feature vector is generated based on the first representation.

The first feature vector and/or reference feature matrices may include a plurality of feature vectors, where each feature vector corresponds to a characteristic of a corresponding object class, e.g. a characteristic minimum, maximum, average, etc. brightness in one or more color channels at a particular location (pixel or section), presence of a particular symbol or other reference object at a particular location, dimensions, aspect ratio, pixel density (especially black pixel density, but also pixel density of any other color channel), etc.

As would be understood by one having ordinary skill in the art upon reading the present descriptions, feature vectors suitable for inclusion in first feature vector and/or reference feature matrices comprise any type, number and/or length of feature vectors, such as described in U.S. patent application Ser. No. 12/042,774, filed Mar. 5, 2008; and Ser. No. 12/368,685, filed Feb. 10, 2009 and/or U.S. Pat. No. 7,761,391, granted Jul. 20, 2010 (U.S. patent application Ser. No. 11/752,364, filed May 13, 2007).

In operation 508, the first feature vector is compared to a plurality of reference feature matrices.

The comparing operation 508 may be performed according to any suitable vector matrix comparison, such as described in U.S. patent application Ser. Nos. 12/042,774, filed Mar. 5, 2008; and Ser. No. 12/368,685, filed Feb. 10, 2009 and U.S. Pat. No. 7,761,391, granted Jul. 20, 2010 (U.S. patent application Ser. No. 11/752,364, filed May 13, 2007).

Thus, in such approaches the comparing may include an N-dimensional feature space comparison. In at least one approach, N is greater than 50, but of course, N may be any value sufficiently large to ensure robust classification of objects into a single, correct object class, which those having ordinary skill in the art reading the present descriptions will appreciate to vary according to many factors, such as the complexity of the object, the similarity or distinctness between object classes, the number of object classes, etc.

As understood herein, “objects” include any tangible thing represented in an image and which may be described according to at least one unique characteristic such as color, size, dimensions, shape, texture, or representative feature(s) as would be understood by one having ordinary skill in the art upon reading the present descriptions. Additionally, objects include or classified according to at least one unique combination of such characteristics. For example, in various embodiments objects may include but are in no way limited to persons, animals, vehicles, buildings, landmarks, documents, furniture, plants, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.

For example, in one embodiment where attempting to classify an object depicted in a digital image as one of only a small number of object classes (e.g. 3-5 object classes), each object class being characterized by a significant number of starkly distinguishing features or feature vectors (e.g. each object class corresponding to an object or object(s) characterized by very different size, shape, color profile and/or color scheme and easily distinguishable reference symbols positioned in unique locations on each object class, etc.), a relatively low value of N may be sufficiently large to ensure robust classification.

On the other hand, where attempting to classify an object depicted in a digital image as one of a large number of object classes (e.g. 30 or more object classes), and each object class is characterized by a significant number of similar features or feature vectors, and only a few distinguishing features or feature vectors, a relatively high value of N may be preferable to ensure robust classification. Similarly, the value of N is preferably chosen or determined such that the classification is not only robust, but also computationally efficient; i.e. the classification process(es) introduce only minimal processing overhead to the device(s) or system(s) utilized to perform the classification algorithm.

The value of N that achieves the desired balance between classification robustness and processing overhead will depend on many factors such as described above and others that would be appreciated by one having ordinary skill in the art upon reading the present descriptions. Moreover, determining the appropriate value of N to achieve the desired balance may be accomplished using any known method or equivalent thereof as understood by a skilled artisan upon reading the instant disclosures.

In a concrete implementation, directed to classifying driver licenses according to state and distinguishing driver licenses from myriad other document types, it was determined that a 625-dimensional comparison (N=625) provided a preferably robust classification without introducing unsatisfactorily high overhead to processing performed using a variety of current-generation mobile devices.

In operation 510, an object depicted in the digital image is classified as a member of a particular object class based at least in part on the comparing operation 508. More specifically, the comparing operation 508 may involve evaluating each feature vector of each reference feature matrix, or alternatively evaluating a plurality of feature matrices for objects belonging to a particular object class, and identifying a hyper-plane in the N-dimensional feature space that separates the feature vectors of one reference feature matrix from the feature vectors of other reference feature matrices. In this manner, the classification algorithm defines concrete hyper-plane boundaries between object classes, and may assign an unknown object to a particular object class based on similarity of feature vectors to the particular object class and/or dissimilarity to other reference feature matrix profiles.

In the simplest example of such feature-space discrimination, imagining a two-dimensional feature space with one feature plotted along the ordinate axis and another feature plotted along the abscissa, objects belonging to one particular class may be characterized by feature vectors having a distribution of values clustered in the lower-right portion of the feature space, while another class of objects may be characterized by feature vectors exhibiting a distribution of values clustered in the upper-left portion of the feature space, and the classification algorithm may distinguish between the two by identifying a line between each cluster separating the feature space into two classes—“upper left” and “lower-right.” Of course, as the number of dimensions considered in the feature space increases, the complexity of the classification grows rapidly, but also provides significant improvements to classification robustness, as will be appreciated by one having ordinary skill in the art upon reading the present descriptions.

Additional Processing

In some approaches, classification according to embodiments of the presently disclosed methods may include one or more additional and/or alternative features and/or operations, such as described below.

In one embodiment, classification such as described above may additionally and/or alternatively include assigning a confidence value to a plurality of putative object classes based on the comparing operation (e.g. as performed in operation 508 of method 500) the presently disclosed classification methods, systems and/or computer program products may additionally and/or alternatively determine a location of the mobile device, receive location information indicating the location of the mobile device, etc. and based on the determined location, a confidence value of a classification result corresponding to a particular location may be adjusted. For example, if a mobile device is determined to be located in a particular state (e.g. Maryland) based on a GPS signal, then during classification, a confidence value may be adjusted for any object class corresponding to the particular state (e.g. Maryland Driver License, Maryland Department of Motor Vehicle Title/Registration Form, Maryland Traffic Violation Ticket, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions).

Confidence values may be adjusted in any suitable manner, such as increasing a confidence value for any object class corresponding to a particular location, decreasing a confidence value for any object class not corresponding to a particular location, normalizing confidence value(s) based on correspondence/non-correspondence to a particular location, etc. as would be understood by the skilled artisan reading the present disclosures.

The mobile device location may be determined using any known method, and employing hardware components of the mobile device or any other number of devices in communication with the mobile device, such as one or more satellites, wireless communication networks, servers, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.

For example, the mobile device location may be determined based in whole or in part on one or more of a global-positioning system (GPS) signal, a connection to a wireless communication network, a database of known locations (e.g. a contact database, a database associated with a navigational tool such as Google Maps, etc.), a social media tool (e.g. a “check-in” feature such as provided via Facebook, Google Plus, Yelp, etc.), an IP address, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.

In more embodiments, classification additionally and/or alternatively includes outputting an indication of the particular object class to a display of the mobile device; and receiving user input via the display of the mobile device in response to outputting the indication. While the user input may be of any known type and relate to any of the herein described features and/or operations, preferably user input relates to confirming, negating or modifying the particular object class to which the object was assigned by the classification algorithm.

The indication may be output to the display in any suitable manner, such as via a push notification, text message, display window on the display of the mobile device, email, etc. as would be understood by one having ordinary skill in the art. Moreover, the user input may take any form and be received in any known manner, such as detecting a user tapping or pressing on a portion of the mobile device display (e.g. by detecting changes in resistance, capacitance on a touch-screen device, by detecting user interaction with one or more buttons or switches of the mobile device, etc.)

In one embodiment, classification further includes determining one or more object features of a classified object based at least in part on the particular object class. Thus, classification may include determining such object features using any suitable mechanism or approach, such as receiving an object class identification code and using the object class identification code as a query and/or to perform a lookup in a database of object features organized according to object class and keyed, hashed, indexed, etc. to the object class identification code.

Object features within the scope of the present disclosures may include any feature capable of being recognized in a digital image, and preferably any feature capable of being expressed in a numerical format (whether scalar, vector, or otherwise), e.g. location of subregion containing reference object(s) (especially in one or more object orientation states, such as landscape, portrait, etc.) object color profile, or color scheme, object subregion color profile or color scheme, location of text, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.

In accordance with another inventive embodiment commensurate in scope with the present disclosures, as shown in FIG. 6, a method 600 is shown. The method 600 may be carried out in any desired environment, and may include embodiments and/or approaches described in relation to FIGS. 1-4D, among others. Of course, more or less operations than those shown in FIG. 6 may be performed in accordance method 600 as would be appreciated by one of ordinary skill in the art upon reading the present descriptions.

In operation 602, a first feature vector is generated based on a digital image captured by a mobile device.

In operation 604, the first feature vector is compared to a plurality of reference feature matrices.

In operation 606, an object depicted in the digital image is classified as a member of a particular object class based at least in part on the comparing (e.g. the comparing performed in operation 604).

In operation 608, one or more object features of the object are determined based at least in part on the particular object class.

In operation 610, a processing operation is performed. The processing operation includes performing one or more of the following subprocesses: detecting the object depicted in the digital image based at least in part on the one or more object features; rectangularizing the object depicted in the digital image based at least in part on the one or more object features; cropping the digital image based at least in part on the one or more object features; and binarizing the digital image based at least in part on the one or more object features.

As will be further appreciated by one having ordinary skill in the art upon reading the above descriptions of document classification, in various embodiments it may be advantageous to perform one or more additional processing operations, such as the subprocesses described above with reference to operation 610, on a digital image based at least in part on object features determined via document classification.

For example, after classifying an object depicted in a digital image, such as a document, it may be possible to refine other processing parameters, functions, etc. and/or utilize information known to be true for the class of objects to which the classified object belongs, such as object shape, size, dimensions, location of regions of interest on and/or in the object, such as regions depicting one or more symbols, patterns, text, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.

Regarding performing page detection based on classification, it may be advantageous in some approaches to utilize information known about an object belonging to a particular object class in order to improve object detection capabilities. For example, and as would be appreciated by one having ordinary skill in the art, it may be less computationally expensive, and/or may result in a higher-confidence or higher-quality result to narrow a set of characteristics that may potentially identify an object in a digital image to one or a few discrete, known characteristics, and simply search for those characteristic(s).

Exemplary characteristics that may be utilized to improve object detection may include characteristics such as object dimensions, object shape, object color, one or more reference features of the object class (such as reference symbols positioned in a known location of a document).

In another approach, object detection may be improved based on the one or more known characteristics by facilitating an object detection algorithm distinguishing regions of the digital image depicting the object from regions of the digital image depicting other objects, image background, artifacts, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions. For example, if objects belonging to a particular object class are known to exhibit a particular color profile or scheme, it may be simpler and/or more reliable to attempt detecting the particular color profile or scheme within the digital image rather than detecting a transition from one color profile or scheme (e.g. a background color profile or scheme) to another color profile or scheme (e.g. the object color profile or scheme), especially if the two colors profiles or schemes are not characterized by sharply contrasting features.

Regarding performing rectangularization based on classification, it may be advantageous in some approaches to utilize information known about an object belonging to a particular object class in order to improve object rectangularization capabilities. For example, and as would be appreciated by one having ordinary skill in the art, it may be less computationally expensive, and/or may result in a higher-confidence or higher-quality result to transform a digital representation of an object from a native appearance to a true configuration based on a set of known object characteristics that definitively represent the true object configuration, rather than attempting to estimate the true object configuration from the native appearance and project the native appearance onto an estimated object configuration.

In one approach, the classification may identify known dimensions of the object, and based on these known dimensions the digital image may be rectangularized to transform a distorted representation of the object in the digital image into an undistorted representation (e.g. by removing projective effects introduced in the process of capturing the image using a camera of a mobile device rather than a traditional flat-bed scanner, paper-feed scanner or other similar multifunction peripheral (MFP)).

Regarding performing cropping based on classification, and similar to the principles discussed above regarding rectangularization, it may be advantageous in some approaches to utilize information known about an object belonging to a particular object class to improve cropping of digital images depicting the object such that all or significantly all of the cropped image depicts the object and not image background (or other objects, artifacts, etc. depicted in the image).

As a simple example, it may be advantageous to determine an object's known size, dimensions, configuration, etc. according to the object classification and utilize this information to identify a region of the image depicting the object from regions of the image not depicting the object, and define crop lines surrounding the object to remove the regions of the image not depicting the object.

Regarding performing binarization based on classification, the presently disclosed classification algorithms provide several useful improvements to mobile image processing. Several exemplary embodiments of such improvements will now be described with reference to FIGS. 4A-4D.

For example, binarization algorithms generally transform a multi-tonal digital image (e.g. grayscale, color, or any other image such as image 400 exhibiting more than two tones) into a bitonal image, i.e. an image exhibiting only two tones (typically white and black). Those having ordinary skill in the art will appreciate that attempting to binarize a digital image depicting an object with regions exhibiting two or more distinct color profiles and/or color schemes (e.g. a region depicting a color photograph 402 as compared to a region depicting a black/white text region 404, a color-text region 406, a symbol 408 such as a reference object, watermark, etc. object background region 410, etc.) may produce an unsuccessful or unsatisfactory result.

As one explanation, these difficulties may be at least partially due to the differences between the color profiles, schemes, etc., which counter-influence a single binarization transform. Thus, providing an ability to distinguish each of these regions having disparate color schemes or profiles and define separate binarization parameters for each may greatly improve the quality of the resulting bitonal image as a whole and with particular respect to the quality of the transformation in each respective region.

According to one exemplary embodiment shown in FIGS. 4A-4B, improved binarization may include determining an object class color profile and/or scheme (e.g. determining a color profile and/or color scheme for object background region 410); adjusting one or more binarization parameters based on the object class color profile and/or color scheme; and thresholding the digital image using the one or more adjusted binarization parameters.

Binarization parameters may include any parameter of any suitable binarization process as would be appreciated by those having ordinary skill in the art reading the present descriptions, and binarization parameters may be adjusted according to any suitable methodology. For example, with respect to adjusting binarization parameters based on an object class color profile and/or color scheme, binarization parameters may be adjusted to over- and/or under-emphasize a contribution of one or more color channels, intensities, etc. in accordance with the object class color profile/scheme (such as under-emphasizing the red channel for an object class color profile/scheme relatively saturated by red hue(s), etc.).

Similarly, in other embodiments such as particularly shown in FIGS. 4B-4D, improved binarization may include determining an object class mask, applying the object class mask to the digital image and thresholding a subregion of the digital image based on the object class mask. The object class mask may be any type of mask, with the condition that the object class mask provides information regarding the location of particular regions of interest characteristic to objects belonging to the class (such as a region depicting a color photograph 402, a region depicting a black/white text region 404, a color-text region 406, a symbol region depicting a symbol 408 such as a reference object, watermark, etc., an object background region 410, etc.) and enabling the selective inclusion and/or exclusion of such regions from the binarization operation(s).

For example, as shown in FIG. 4B, improved binarization includes determining an object class mask 420 identifying regions such as discussed immediately above and applying the object class mask 420 to exclude from binarization all of the digital image 400 except a single region of interest, such as object background region 410. Alternatively the entire digital image may be masked-out and a region of interest such as object background region 410 subsequently masked-in to the binarization process. Moreover, in either event the masking functionality now described with reference to FIG. 4B may be combined with the exemplary color profile and/or color scheme information functionality described above, for example by obtaining both the object class mask and the object color profile and/or color scheme, applying the object class mask to exclude all of the digital image from binarization except object background region 410, adjusting one or more binarization parameters based on the object background region color profile and/or color scheme, and thresholding the object background region 410 using the adjusted binarization parameters.

Extending the principle shown in FIG. 4B, multiple regions of interest may be masked-in and/or masked-out using object class mask 420 to selectively designate regions and/or parameters for binarization in a layered approach designed to produce high-quality bitonal images. For example, as shown in FIG. 4C multiple text regions 404, 406 may be retained for binarization (potentially using adjusted parameters) after applying object class mask 420, for example to exclude all non-text regions from binarization, in some approaches.

Similarly, it may be advantageous to simply exclude only a portion of an image from binarization, whether or not adjusting any parameters. For example, with reference to FIG. 4D, it may be desirable to mask-out a unique region of a digital image 400, such as a region depicting a color photograph 402, using an object class mask 420. Then, particularly in approaches where the remaining portion of the digital image 400 is characterized by a single color profile and/or color scheme, or a small number (i.e. no more than 3) substantially similar color profile and/or color schemes, binarization may be performed to clarify the remaining portions of the digital image 400. Subsequently, the masked-out unique region may optionally be restored to the digital image 400, with the result being an improved bitonal image quality in all regions of the digital image 400 that were subjected to binarization coupled with an undisturbed color photograph 402 in the region of the image not subjected to binarization.

In still more embodiments, it may be advantageous to perform optical character recognition (OCR) based at least in part on the classification and/or result of classification. Specifically, it may be advantageous to determine information about the location, format, and/or content of text depicted in objects belonging to a particular class, and modify predictions estimated by traditional OCR methods based on an expected text location, format and/or content. For example, in one embodiment where an OCR prediction estimates text in a region corresponding to a “date” field of a document reads “Jan. 14, 201l” the presently disclosed algorithms may determine the expected format for this text follows a format such as “[Abbreviated Month][.] [##][,][####]” the algorithm may correct the erroneous OCR predictions, e.g. converting the comma after “Jan” into a period and/or converting the letter “l” at the end of 201l” into a numerical one character. Similarly, the presently disclosed algorithms may determine the expected format for the same text is instead “[##]/[##]/[####]” and convert “Jan” to “01” and convert each set of comma-space characters “,” into a slash “/” to correct the erroneous OCR predictions.

Of course, other methods of improving upon and/or correcting OCR predictions that would be appreciated by the skilled artisan upon reading these descriptions are also fully within the scope of the present disclosure.

While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of an embodiment of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Amtrup, Jan W., Macciola, Anthony, Thompson, Steve, Thrasher, Christopher W., Shustorovich, Alexander, Ma, Jiyong

Patent Priority Assignee Title
10242285, Jul 20 2015 TUNGSTEN AUTOMATION CORPORATION Iterative recognition-guided thresholding and data extraction
10635712, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
10657600, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
10699146, Oct 30 2014 TUNGSTEN AUTOMATION CORPORATION Mobile document detection and orientation based on reference object characteristics
10783613, Sep 27 2013 TUNGSTEN AUTOMATION CORPORATION Content-based detection and three dimensional geometric reconstruction of objects in image and video data
10803350, Nov 30 2017 TUNGSTEN AUTOMATION CORPORATION Object detection and image cropping using a multi-detector approach
11062176, Nov 30 2017 TUNGSTEN AUTOMATION CORPORATION Object detection and image cropping using a multi-detector approach
Patent Priority Assignee Title
1660102,
3069654,
3696599,
4558461, Jun 17 1983 Litton Systems, Inc. Text line bounding system
4651287, Jun 14 1984 Digital image processing algorithm for output devices with discrete halftone gray scale capability
4656665, Jan 15 1985 International Business Machines Corporation; INTERNATIONAL BUSINESS MACHINES CORPORATION A CORP OF NY Thresholding technique for graphics images using histogram analysis
4836026, Jun 01 1984 Science Applications International Corporation Ultrasonic imaging system
4903312, May 19 1986 Ricoh Company, Ltd. Character recognition with variable subdivisions of a character region
4992863, Dec 22 1987 Minolta Camera Kabushiki Kaisha Colored image reading apparatus
5020112, Oct 31 1989 NCR Corporation Image recognition method using two-dimensional stochastic grammars
5063604, Nov 08 1989 CAREFUSION 303, INC Method and means for recognizing patterns represented in logarithmic polar coordinates
5101448, Aug 24 1988 Hitachi, Ltd. Method and apparatus for processing a document by utilizing an image
5124810, May 26 1989 Ricoh Company, Ltd. Image reading apparatus
5151260, Dec 21 1989 TEMAV S P A , VENEZIA-PORTO MARGHERA VENEZIA ITALY Process for preparing fine powders of aluminum nitride
5159667, May 31 1989 TUNGSTEN AUTOMATION CORPORATION Document identification by characteristics matching
5181260, Oct 31 1989 Hitachi, Ltd. Method for determining the amount of skew of image, method for correcting the same, and image data processing system
5202934, Jun 22 1990 FUJIFILM Corporation Method of and apparatus for reading image
5220621, Aug 01 1990 International Business Machines Corporation Character recognition system using the generalized hough transformation and method
5268967, Jun 29 1992 CARESTREAM HEALTH, INC Method for automatic foreground and background detection in digital radiographic images
5282055, Feb 20 1991 Fuji Xerox Co., Ltd. Compressing a regular scan image using coding parameter from the prescan image
5293429, Aug 06 1991 RICOH COMPANY, LTD A CORPORATION OF JAPAN; RICOH CORPORATION A CORPORATION OF DE System and method for automatically classifying heterogeneous business forms
5313527, Jun 07 1991 Microsoft Technology Licensing, LLC Method and apparatus for recognizing cursive writing from sequential input information
5317646, Mar 24 1992 Xerox Corporation; XEROX CORPORATION A CORPORATION OF NEW YORK Automated method for creating templates in a forms recognition and processing system
5321770, Nov 19 1991 Technology Licensing Corporation Method for determining boundaries of words in text
5344132, Jan 16 1990 Digital Image Systems Image based document processing and information management system and apparatus
5353673, Sep 07 1993 Brass-wind musical instrument mouthpiece with radially asymmetric lip restrictor
5355547, Feb 27 1992 Georg Karl Geka-Brush GmbH Dental cleaning brush
5375197, Dec 31 1990 Goldstar Co., Ltd. System and method for automatically distinguishing between graphic information and text information of image data
5430810, Nov 20 1990 IMRA America, Inc. Real time implementation of the hough transform
5467407, Jun 07 1991 Microsoft Technology Licensing, LLC Method and apparatus for recognizing cursive writing from sequential input information
5473742, Feb 22 1994 Microsoft Technology Licensing, LLC Method and apparatus for representing image data using polynomial approximation method and iterative transformation-reparametrization technique
5546474, Dec 21 1993 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Detection of photo regions in digital images
5563723, Aug 31 1994 LSI Logic Corporation Method of calibration of image scanner signal processing circuits
5563966, Nov 22 1991 Hitachi, Ltd. Method and apparatus for processing coded image data
5586199, Oct 02 1991 Fujitsu Limited Method for determining orientation of contour line segment in local area and for determining straight line and corner
5594815, Oct 19 1992 Accusoft Corporation OCR image preprocessing method for image enhancement of scanned documents
5596655, Aug 18 1992 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Method for finding and classifying scanned information
5602964, May 21 1993 BARRETT HOLDING LLC Automata networks and methods for obtaining optimized dynamically reconfigurable computational architectures and controls
5629989, Apr 27 1993 Honda Giken Kogyo Kabushiki Kaisha Image line-segment extracting apparatus
5652663, Jul 29 1994 Intellectual Ventures I LLC Preview buffer for electronic scanner
5668890, Apr 06 1992 Heidelberger Druckmaschinen AG Method and apparatus for the automatic analysis of density range, color cast, and gradation of image originals on the BaSis of image values transformed from a first color space into a second color space
5680525, Aug 08 1991 Hitachi, LTD Three-dimensional graphic system with an editor for generating a textrue mapping image
5696611, Nov 08 1994 PANASONIC COMMUNICATIONS CO , LTD Color picture processing apparatus for reproducing a color picture having a smoothly changed gradation
5696805, Sep 17 1996 CARESTREAM HEALTH, INC Apparatus and method for identifying specific bone regions in digital X-ray images
5699244, Mar 07 1994 MONSANTO TECHNOLOGY LLC Hand-held GUI PDA with GPS/DGPS receiver for collecting agronomic and GPS position data
5717794, Mar 17 1993 Hitachi, Ltd. Document recognition method and system
5721940, Nov 24 1993 Canon Information Systems, Inc. Form identification and processing system using hierarchical form profiles
5757963, Sep 30 1994 Xerox Corporation; FUJI XEROX CO , LTD Method and apparatus for complex column segmentation by major white region pattern matching
5760912, Oct 19 1994 Minolta Co., Ltd. Image forming apparatus capable of providing appropriate margin
5781665, Aug 28 1995 Pitney Bowes Inc. Apparatus and method for cropping an image
5818978, Apr 15 1994 Canon Kabushiki Kaisha Image pre-processor for character image recognition system
5822454, Apr 10 1995 Rebus Technology, Inc.; REBUS TECHNOLOGY, INC System and method for automatic page registration and automatic zone detection during forms processing
5825915, Sep 12 1995 Matsushita Electric Industrial Co., Ltd. Object detecting apparatus in which the position of a planar object is estimated by using hough transform
5832138, Mar 07 1994 Nippon Telegraph and Telephone Corporation Image processing method and apparatus for extracting lines from an image by using the Hough transform
5839019, Dec 28 1993 Minolta Co., Ltd. Image forming apparatus for copying different groups of document
5848184, Mar 15 1993 Unisys Corporation Document page analyzer and method
5857029, Jun 05 1995 United Parcel Service of America, Inc. Method and apparatus for non-contact signature imaging
5867264, Oct 15 1997 Pacific Advanced Technology Apparatus for image multispectral sensing employing addressable spatial mask
5923763, Mar 21 1996 Inventor Holdings, LLC Method and apparatus for secure document timestamping
5937084, May 22 1996 CUFER ASSET LTD L L C Knowledge-based document analysis system
5953388, Aug 18 1997 George Mason University Method and apparatus for processing data from a tomographic imaging system
5956468, Jan 10 1997 Seiko Epson Corporation Document segmentation system
5987172, Dec 06 1995 Cognex Corporation Edge peak contour tracker
6002489, Apr 02 1993 PFU Limited Product catalog having image evaluation chart
6005958, Apr 23 1997 Automotive Systems Laboratory, Inc. Occupant type and position detection system
6005968, Aug 29 1997 X-Rite, Incorporated; OTP, INCORPORATED; MONACO ACQUISITION COMPANY; X-RITE GLOBAL, INCORPORATED; X-RITE HOLDINGS, INC ; GretagMacbeth LLC; PANTONE, INC Scanner calibration and correction techniques using scaled lightness values
6009191, Feb 15 1996 Intel Corporation Computer implemented method for compressing 48-bit pixels to 16-bit pixels
6009196, Nov 28 1995 Xerox Corporation Method for classifying non-running text in an image
6011595, Sep 19 1997 Apple Inc Method for segmenting a digital image into a foreground region and a key color region
6016361, Nov 05 1996 NEC Corporation Method and apparatus for compressing binary data using pattern matching encoding
6038348, Jul 24 1996 CSR TECHNOLOGY INC Pixel image enhancement system and method
6055968, Aug 04 1997 Toyota Jidosha Kabushiki Kaisha Engine
6067385, May 07 1997 Ricoh Company Limited System for aligning document images when scanned in duplex mode
6072916, Sep 18 1996 FUJIFILM Corporation High speed pipeline image processing for digital photoprinter
6073148, Sep 25 1995 Adobe Systems Incorporated Displaying electronic documents with substitute fonts
6098065, Feb 13 1997 RPX CLEARINGHOUSE LLC Associative search engine
6104830, Jun 22 1995 ESKO-GRAPHICS A S Method and apparatus for correction of color cast in a digital image
6104840, Nov 08 1996 Ricoh Company, LTD Method and system for generating a composite image from partially overlapping adjacent images taken along a plurality of axes
6118544, Dec 26 1996 Kabushiki Kaisha Toshiba Image forming apparatus for processing according to separation sheet
6118552, Sep 21 1993 Fuji Xerox Co., Ltd. Image signal encoding apparatus
6154217, Apr 15 1997 Software Architects, Inc.; SOFTWARE ARCHITECTS, INC Gamut restriction of color image
6192360, Jun 23 1998 Microsoft Technology Licensing, LLC Methods and apparatus for classifying text and for building a text classifier
6215469, Jun 25 1997 MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD Image display method
6219158, Jul 31 1998 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Method and apparatus for a dynamically variable scanner, copier or facsimile secondary reflective surface
6219773, Oct 18 1993 VIA-Cyrix, Inc System and method of retiring misaligned write operands from a write buffer
6223223, Sep 30 1998 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Network scanner contention handling method
6229625, Jul 04 1997 SCREEN HOLDINGS CO , LTD Apparatus for determining image processing parameter, method of the same, and computer program product for realizing the method
6233059, Feb 19 1997 Canon Kabushiki Kaisha Scanner device and control method thereof, and image input system
6263122, Sep 23 1998 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P System and method for manipulating regions in a scanned image
6292168, Aug 13 1998 Xerox Corporation Period-based bit conversion method and apparatus for digital image processing
6327581, Apr 06 1998 Microsoft Technology Licensing, LLC Methods and apparatus for building a support vector machine classifier
6337925, May 08 2000 Adobe Systems Incorporated Method for determining a border in a complex scene with applications to image masking
6347152, Mar 27 1997 JBF PARTNERS, INC Multiresolutional critical point filter and image matching using the invention
6347162, Oct 09 1996 FUJIFILM Corporation Image processing apparatus
6356647, Jul 19 1996 Telefonaktiebolaget LM Ericsson Hough transform based method of estimating parameters
6370277, Dec 07 1998 KOFAX, INC Virtual rescanning: a method for interactive document image quality enhancement
6385346, Aug 04 1998 Sharp Laboratories of America, Inc. Method of display and control of adjustable parameters for a digital scanner device
6393147, Apr 13 1998 Intel Corporation Color region based recognition of unidentified objects
6396599, Dec 21 1998 Monument Peak Ventures, LLC Method and apparatus for modifying a portion of an image in accordance with colorimetric parameters
6408094, Jan 17 1995 KODAK ALARIS INC Document image assessment system and method
6408105, May 12 1998 Advantest Corporation Method for detecting slope of image data utilizing hough-transform
6424742, Aug 20 1997 Kabushiki Kaisha Toshiba Image processing apparatus for discriminating image field of original document plural times and method therefor
6426806, Mar 31 1998 Canon Kabushiki Kaisha Routing scanned documents with scanned control sheets
6433896, Jun 10 1998 MINOLTA CO , LTD Image processing apparatus
6456738, Jul 16 1998 Ricoh Company, Ltd. Method of and system for extracting predetermined elements from input document based upon model which is adaptively modified according to variable amount in the input document
6463430, Jul 10 2000 TUNGSTEN AUTOMATION CORPORATION Devices and methods for generating and managing a database
6469801, May 17 2000 THE BOARD OF THE PENSION PROTECTION FUND Scanner with prepress scaling mode
6473198, Sep 19 1997 FUJIFILM Corporation Image processing apparatus
6473535, Apr 06 1998 FUJIFILM Corporation Image processing apparatus and method
6480304, Dec 09 1998 Canon Kabushiki Kaisha Scanning system and method
6480624, Sep 30 1997 MINOLTA CO , LTD Color discrimination apparatus and method
6501855, Jul 20 1999 Parascript, LLC Manual-search restriction on documents not having an ASCII index
6512848, Nov 18 1996 Canon Kabushiki Kaisha Page analysis system
6522791, Jan 23 2001 Xerox Corporation Dynamic user interface with scanned image improvement assist
6525840, Sep 09 1997 Konica Corporation Image forming apparatus
6563531, Mar 12 1998 FUJIFILM Corporation Image processing method
6601026, Sep 17 1999 Microsoft Technology Licensing, LLC Information retrieval by natural language querying
6614930, Jan 28 1999 FUNAI ELECTRIC CO , LTD Video stream classifiable symbol isolation method and system
6621595, Nov 01 2000 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P System and method for enhancing scanned document images for color printing
6628416, Oct 13 1999 Intellectual Ventures I LLC Method and user interface for performing a scan operation for a scanner coupled to a computer system
6628808, Jul 28 1999 Datacard Corporation Apparatus and method for verifying a scanned image
6633857, Sep 04 1999 Microsoft Technology Licensing, LLC Relevance vector machine
6643413, Mar 27 2000 Microsoft Technology Licensing, LLC Manifold mosaic hopping for image-based rendering
6646765, Feb 19 1999 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Selective document scanning method and apparatus
6658147, Apr 16 2001 EVERNOTE CORPORATION Reshaping freehand drawn lines and shapes in an electronic document
6665425, Dec 16 1999 Xerox Corporation Systems and methods for automated image quality based diagnostics and remediation of document processing systems
6667774, Nov 02 2001 Imatte, Inc. Method and apparatus for the automatic generation of subject to background transition area boundary lines and subject shadow retention
6675159, Jul 27 2000 Leidos, Inc Concept-based search and retrieval system
6701009, Jun 06 2000 Sharp Kabushiki Kaisha Method of separated color foreground and background pixel improvement
6704441, Feb 04 1999 KEYENCE CORPORATION Inspection system
6724916, Jan 05 2000 The United States of America as represented by the Secretary of the Navy Composite hough transform for multitarget multisensor tracking
6729733, Mar 21 2003 Mitsubishi Electric Research Laboratories, Inc Method for determining a largest inscribed rectangular image within a union of projected quadrilateral images
6732046, Oct 03 2001 HERE GLOBAL B V Application of the hough transform to modeling the horizontal component of road geometry and computing heading and curvature
6748109, Jun 16 1998 FUJIFILM Corporation Digital laboratory system for processing photographic images
6751349, Nov 30 1999 FUJIFILM Corporation Image processing system
6757081, Apr 07 2000 HEWLETT-PACKARD DEVELOPMENT COMPANY L P Methods and apparatus for analyzing and image and for controlling a scanner
6757427, Aug 24 1999 NEC Corporation Edge enhancement preprocessing with image region determining function
6763515, Jun 05 2000 National Instruments Corporation System and method for automatically generating a graphical program to perform an image processing algorithm
6765685, Jan 22 1999 Ricoh Company, Ltd. Printing electronic documents with automatically interleaved separation sheets
6778684, Jan 20 2000 Xerox Corporation Systems and methods for checking image/document quality
6781375, Aug 31 2001 Kabushiki Kaisha Toshiba Magnetic resonance imaging using preparation scan for optimizing pulse sequence
6788830, Sep 30 1999 MINOLTA CO , LTD Image processing system and image processing method
6789069, May 01 1998 Health Discovery Corporation Method for enhancing knowledge discovered from biological data using a learning machine
6801658, Mar 01 1999 HITACHI-OMRON TERMINAL SOLUTIONS CORP Business form handling method and system for carrying out the same
6816187, Jun 08 1999 Sony Corporation Camera calibration apparatus and method, image processing apparatus and method, program providing medium, and camera
6826311, Jan 04 2001 Microsoft Technology Licensing, LLC Hough transform supporting methods and arrangements
6831755, Jun 26 1998 Sony Corporation Printer having image correcting capability
6839466, Oct 04 1999 Xerox Corporation Detecting overlapping images in an automatic image segmentation device with the presence of severe bleeding
6850653, Aug 08 2000 Canon Kabushiki Kaisha Image reading system, image reading setting determination apparatus, reading setting determination method, recording medium, and program
6873721, Mar 09 1998 OL SECURITY LIMITED LIABILITY COMPANY Method and device for determining an angled structure in the surface of a precision machined cylindrical work piece
6882983, Feb 05 2001 Oracle International Corporation Method and system for processing transactions
6898601, May 23 2002 PhoChron, Inc. System and method for digital content processing and distribution
6901170, Sep 05 2000 FUJIFILM Business Innovation Corp Image processing device and recording medium
6917438, Oct 22 1999 Kabushiki Kaisha Toshiba; Toshiba Tec Kabushiki Kaisha Information input device
6917709, Jul 20 1999 Parascript LLC Automated search on cursive records not having an ASCII index
6921220, Dec 19 2000 Canon Kabushiki Kaisha Image processing system, data processing apparatus, data processing method, computer program and storage medium
6950555, Feb 16 2001 Parascript LLC Holistic-analytical recognition of handwritten text
6987534, Aug 30 1999 Subaru Corporation Brightness adjusting apparatus for stereoscopic camera
6989914, Feb 04 2000 Fuji Photo Film Co., Ltd. Image processing apparatus and photoprinter using the same
6999625, Jul 12 2002 The United States of America as represented by the Secretary of the Navy Feature-based detection and context discriminate classification for digital images
7006707, May 03 2001 Adobe Inc Projecting images onto a surface
7016549, Jun 14 1999 Nikon Corporation Image processing method for direction dependent low pass filtering
7017108, Sep 15 1998 Canon Kabushiki Kaisha Method and apparatus for reproducing a linear document having non-linear referential links
7020320, Mar 06 2002 Parascript, LLC Extracting text written on a check
7023447, May 02 2001 Eastman Kodak Company Block sampling based method and apparatus for texture synthesis
7027181, Jul 21 2000 FUJIFILM Corporation Image processing condition determining apparatus, and image processing condition determining program storage medium
7038713, Sep 09 1997 FUJIFILM Corporation Image processing apparatus
7042603, Sep 28 2000 CANON ELECTRONICS INC Information processing apparatus, image input apparatus, method of controlling image processing apparatus, method of controlling image input apparatus, image input system, and storage medium
7043080, Nov 21 2000 Sharp Kabushiki Kaisha Methods and systems for text detection in mixed-context documents using local geometric signatures
7054036, Jan 25 2002 Kabushiki Kaisha Toshiba; Toshiba Tec Kabushiki Kaisha Image processing method and image forming apparatus
7081975, Oct 22 1999 Kabushiki Kaisha Toshiba; Toshiba Tec Kabushiki Kaisha Information input device
7082426, Jun 18 1993 CBS INTERACTIVE INC Content aggregation method and apparatus for an on-line product catalog
7107285, Mar 16 2002 Celeritasworks, LLC Method, system, and program for an improved enterprise spatial system
7123292, Sep 29 1999 Xerox Corporation Mosaicing images with an offset lens
7123387, Aug 23 2002 AI-CORE TECHNOLOGIES, LLC Image scanning method
7130471, May 24 2001 Microsoft Technology Licensing, LLC Determining optimal image compression methods for reducing file size
7145699, Mar 30 2001 Sharp Kabushiki Kaisha System and method for digital document alignment
7167281, Nov 26 1999 Fujitsu Limited Image processing apparatus and method for binarizing a multilevel image
7168614, Dec 10 2004 MITEK SYSTEMS, INC System and method for check fraud detection using signature validation
7173732, Mar 30 2001 FUJIFILM Corporation Image processing method
7174043, Feb 25 2003 BENDING SPOONS S P A On-line handwriting recognizer
7177049, May 18 1999 Electronics for Imaging, Inc. Image reconstruction architecture
7181082, Dec 18 2002 Sharp Kabushiki Kaisha Blur detection system
7184929, Jan 28 2004 Microsoft Technology Licensing, LLC Exponential priors for maximum entropy models
7194471, Apr 10 1998 Ricoh Company, LTD Document classification system and method for classifying a document according to contents of the document
7197158, Jun 28 2002 Microsoft Technology Licensing, LLC Generation of metadata for acquired images
7201323, Dec 10 2004 MITEK SYSTEMS, INC System and method for check fraud detection using signature validation
7209599, Jul 12 2002 HEWLETT-PACKARD DEVELOPMENT COMPANY L P System and method for scanned image bleedthrough processing
7228314, Nov 30 2000 Canon Kabushiki Kaisha Apparatus and method for controlling user interface
7249717, Dec 10 2004 MITEK SYSTEMS, INC System and method for check fraud detection using signature validation
7251777, Apr 16 2003 ICTECT, INC Method and system for automated structuring of textual documents
7253836, Jun 30 1998 Nikon Corporation Digital camera, storage medium for image signal processing, carrier wave and electronic camera
7263221, Jun 25 1999 MINOLTA CO , LTD Image processing for image correction
7266768, Jan 09 2001 Sharp Kabushiki Kaisha Systems and methods for manipulating electronic information using a three-dimensional iconic representation
7286177, Dec 19 2001 Nokia Corporation Digital camera
7298897, Feb 11 2004 NATIONAL AERONAUTICS AND SPACE ADMINISTRATION, U S GOVERNMENT AS REPRESENTED BY THE ADMINISTRATOR OF Optimal binarization of gray-scaled digital images via fuzzy reasoning
7317828, Sep 09 2002 Minolta Co., Ltd. Color image processing apparatus with background control
7337389, Dec 07 1999 Microsoft Technology Licensing, LLC System and method for annotating an electronic document independently of its content
7339585, Jul 19 2004 Pie Medical Imaging B.V. Method and apparatus for visualization of biological structures with use of 3D position information from segmentation results
7340376, Jan 28 2004 Microsoft Technology Licensing, LLC Exponential priors for maximum entropy models
7349888, Nov 10 2003 ZXIBIX INC System and method to customize the facilitation of development of user thinking about and documenting of an arbitrary problem
7365881, Aug 19 2002 Eastman Kodak Company Halftone dot-growth technique based on morphological filtering
7366705, Apr 15 2004 Microsoft Technology Licensing, LLC Clustering based text classification
7382921, Feb 25 2003 BENDING SPOONS S P A Training an on-line handwriting recognizer
7386527, Dec 06 2002 TUNGSTEN AUTOMATION CORPORATION Effective multi-class support vector machine classification
7392426, Jun 15 2004 Honeywell International Inc.; Honeywell International Inc Redundant processing architecture for single fault tolerance
7403008, Aug 02 2004 Cornell Research Foundation, Inc. Electron spin resonance microscope for imaging with micron resolution
7403313, Sep 27 2001 Intellectual Ventures I LLC Automatic scanning parameter setting device and method
7406183, Apr 28 2003 International Business Machines Corporation System and method of sorting document images based on image quality
7409092, Jun 20 2002 HRL Laboratories, LLC Method and apparatus for the surveillance of objects in images
7409633, Mar 07 2000 Microsoft Corporation System and method for annotating web-based document
7416131, Dec 13 2006 BOTTOMLINE TECHNLOGIES, INC Electronic transaction processing server with automated transaction evaluation
7426293, Apr 06 2004 Sunplus Technology Co., Ltd. Rapid color recognition method
7430059, May 24 2004 Xerox Corporation Systems, methods and graphical user interfaces for interactively previewing a scanned document
7430066, Oct 13 1999 Intellectual Ventures I LLC Method and user interface for performing an automatic scan operation for a scanner coupled to a computer system
7430310, Dec 10 2004 MITEK SYSTEMS, INC. System and method for check fraud detection using signature validation
7447377, Oct 10 2003 FUJIFILM Corporation Image processing method and apparatus, and image processing program
7464066, Oct 26 2004 International Business Machines Corporation Multi-dimensional, expert behavior-emulation system
7478332, May 24 2006 TUNGSTEN AUTOMATION CORPORATION System for and method of providing a user interface for a computer-based software application
7487438, Mar 08 2005 Accusoft Corporation Method and apparatus for recognizing a digitized form, extracting information from a filled-in form, and generating a corrected filled-in form
7492478, Jun 26 2003 Fuji Xerox Co., Ltd. Image processing device and method
7492943, Oct 29 2004 GEORGE MASON INTELLECTUAL PROPERTIES, INC Open set recognition using transduction
7515313, Sep 20 2001 Intellectual Ventures I LLC Method and system for scanning with one-scan-and-done feature
7515772, Aug 21 2004 Xerox Corporation Document registration and skew detection system
7528883, May 08 2006 Primax Electronics Ltd. Method for blurred image judgment
7542931, Jun 01 2001 Liberty Peak Ventures, LLC System and method for global automated address verification
7545529, Mar 24 2005 TUNGSTEN AUTOMATION CORPORATION Systems and methods of accessing random access cache for rescanning
7553095, Nov 27 2003 Konica Minolta Business Technologies, Inc. Print data transmitting apparatus, image forming system, printing condition setting method and printer driver program
7562060, Mar 31 2006 R2 SOLUTIONS LLC Large scale semi-supervised linear support vector machines
7580557, Aug 31 2004 TECHINSIGHTS INC Method of design analysis of existing integrated circuits
7636479, Feb 24 2004 TRW Automotive U.S. LLC Method and apparatus for controlling classification and classification switching in a vision system
7639387, Aug 23 2005 Ricoh Co., Ltd. Authoring tools using a mixed media environment
7643665, Aug 31 2004 TECHINSIGHTS INC Method of design analysis of existing integrated circuits
7651286, Nov 19 2002 Oce Printing Systems GmbH Method arrangement and computer software for the printing of a separator sheet by means of an electrophotographic printer or copier
7655685, Nov 02 2007 JENRIN DISCOVERY, INC Cannabinoid receptor antagonists/inverse agonists useful for treating metabolic disorders, including obesity and diabetes
7657091, Mar 06 2006 MITEK SYSTEMS, INC Method for automatic removal of text from a signature area
7665061, Apr 08 2003 Microsoft Technology Licensing, LLC Code builders
7673799, Jan 26 2007 MAGTEK, INC Card reader for use with web based transactions
7702162, Nov 05 2004 INTELLECTUAL DISCOVERY CO , LTD Mixed code, and method and apparatus for generating the same
7735721, Nov 30 1999 Diebold Nixdorf, Incorporated; DIEBOLD SELF-SERVICE SYSTEMS DIVISION OF DIEBOLD NIXDORF, INCORPORATED Method of evaluating checks deposited into a cash dispensing automated banking machine
7738730, Jan 25 2006 TUNGSTEN AUTOMATION CORPORATION Method of image analysis using sparse hough transform
7739127, Sep 23 2004 HALL BENEFITS, LLC Automated system for filing prescription drug claims
7761391, Jul 12 2006 TUNGSTEN AUTOMATION CORPORATION Methods and systems for improved transductive maximum entropy discrimination classification
7778457, Jan 18 2008 MITEK SYSTEMS, INC Systems for mobile image capture and processing of checks
7782384, Nov 05 2004 Digital camera having system for digital image composition and related method
7787695, Jun 06 2006 MITEK SYSTEMS, INC Method for applying a signature simplicity analysis for improving the accuracy of signature validation
7937345, Jul 12 2006 TUNGSTEN AUTOMATION CORPORATION Data classification methods using machine learning techniques
7941744, Apr 25 2005 ADP, INC System and method for electronic document generation and delivery
7949167, Jun 12 2008 SIEMENS HEALTHINEERS AG Automatic learning of image features to predict disease
7949176, Jan 18 2008 MITEK SYSTEMS, INC Systems for mobile image capture and processing of documents
7949660, Jun 28 2006 Oracle America, Inc Method and apparatus for searching and resource discovery in a distributed enterprise system
7953268, Jan 18 2008 MITEK SYSTEMS, INC Methods for mobile image capture and processing of documents
7958067, Jul 12 2006 TUNGSTEN AUTOMATION CORPORATION Data classification methods using machine learning techniques
7978900, Jan 18 2008 MITEK SYSTEMS, INC Systems for mobile image capture and processing of checks
7999961, Sep 12 2000 Canon Kabushiki Kaisha Information processing apparatus, distributed printing controlling method, storing medium and program
8000514, Jan 18 2008 MITEK SYSTEMS, INC Methods for mobile image capture and processing of checks
8035641, Nov 28 2007 Adobe Inc Fast depth of field simulation
8064710, Dec 06 2004 Canon Kabushiki Kaisha Image processing apparatus, method of controlling thereof, and program
8073263, Jul 31 2006 RICOH CO , LTD Multi-classifier selection and monitoring for MMR-based image recognition
8078958, Jun 24 1999 Microsoft Technology Licensing, LLC Scalable computing system for managing annotations
8081227, Nov 30 2006 Adobe Inc Image quality visual indicator
8094976, Oct 03 2007 Esker, Inc. One-screen reconciliation of business document image data, optical character recognition extracted data, and enterprise resource planning data
8135656, Apr 22 2008 Xerox Corporation Online management service for identification documents which prompts a user for a category of an official document
8136114, Apr 21 2006 Sprint Communications Company L.P. Business process management system having dynamic task assignment
8184156, Jun 02 2003 FUJIFILM Corporation Image displaying system and apparatus for displaying images by changing the displayed images based on direction or direction changes of a displaying unit
8194965, Nov 19 2007 Parascript, LLC Method and system of providing a probability distribution to aid the detection of tumors in mammogram images
8213687, Apr 28 2006 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Image processing methods, image processing systems, and articles of manufacture
8238880, Oct 31 2008 ZTE Corporation Method and apparatus for processing authentication of mobile terminal
8239335, Jul 12 2006 TUNGSTEN AUTOMATION CORPORATION Data classification using machine learning techniques
8244031, Apr 13 2007 TUNGSTEN AUTOMATION CORPORATION System and method for identifying and classifying color regions from a digital image
8265422, Feb 20 2009 Adobe Inc Method and apparatus for removing general lens distortion from images
8279465, May 01 2007 TUNGSTEN AUTOMATION CORPORATION Systems and methods for routing facsimiles based on content
8295599, Aug 03 2009 Sharp Kabushiki Kaisha Image output apparatus, captured image processing system, and recording medium
8311296, Nov 21 2007 Parascript, LLC Voting in mammography processing
8326015, Jan 18 2008 MITEK SYSTEMS, INC Methods for mobile image capture and processing of documents
8345981, Feb 10 2009 TUNGSTEN AUTOMATION CORPORATION Systems, methods, and computer program products for determining document validity
8354981, Jul 02 2007 Canon Kabushiki Kaisha Active matrix type display apparatus and driving method thereof
8374977, Jul 12 2006 TUNGSTEN AUTOMATION CORPORATION Methods and systems for transductive data classification
8379914, Jan 18 2008 MITEK SYSTEMS, INC Systems and methods for mobile image capture and remittance processing
8385647, Jan 25 2006 TUNGSTEN AUTOMATION CORPORATION Method of image analysis using sparse Hough transform
8406480, Feb 17 2009 International Business Machines Corporation Visual credential verification
8433775, Mar 31 2010 Bank of America Corporation Integration of different mobile device types with a business infrastructure
8441548, Jun 15 2012 GOOGLE LLC Facial image quality assessment
8443286, Feb 27 2008 Adobe Inc Document mapped-object placement upon background change
8452098, Jun 06 2006 MITEK SYSTEMS, INC Method for applying a signature simplicity analysis for improving the accuracy of signature validation
8478052, Jul 17 2009 GOOGLE LLC Image classification
8483473, Jan 18 2008 MITEK SYSTEMS, INC Systems and methods for obtaining financial offers using mobile image capture
8503769, Dec 28 2010 Microsoft Technology Licensing, LLC Matching text to images
8503797, Sep 05 2007 THE NEAT COMPANY, INC DOING BUSINESS AS NEATRECEIPTS, INC Automatic document classification using lexical and physical features
8515163, Apr 13 2007 TUNGSTEN AUTOMATION CORPORATION System and method for identifying and classifying color regions from a digital image
8515208, Apr 05 2010 TUNGSTEN AUTOMATION CORPORATION Method for document to template alignment
8526739, Feb 10 2009 TUNGSTEN AUTOMATION CORPORATION Systems, methods and computer program products for determining document validity
8532419, Jan 13 2010 NORTHWEST IP, LLC Automatic image capture
8559766, Aug 16 2011 NORTHWEST IP, LLC Automatic image capture
8577118, Jan 18 2008 MITEK SYSTEMS, INC Systems for mobile image capture and remittance processing
8582862, May 12 2010 MITEK SYSTEMS, INC Mobile image quality assurance in mobile document image processing applications
8587818, Aug 07 2009 Sharp Kabushiki Kaisha Captured image processing system and recording medium
8620058, Jan 18 2008 MITEK SYSTEMS, INC Methods for mobile image capture and processing of documents
8639621, Apr 25 2012 WELLS FARGO BANK, N A System and method for a mobile wallet
8675953, Feb 02 2011 INTUIT INC. Calculating an object size using images
8676165, Dec 28 2004 TELEFONAKTIEBOLAGET L M ERICSSON PUBL Method and apparatus for peer-to-peer instant messaging
8677249, May 24 2006 TUNGSTEN AUTOMATION CORPORATION System for and method of providing a user interface for a computer-based software application
8693043, Dec 19 2003 TUNGSTEN AUTOMATION CORPORATION Automatic document separation
8705836, Aug 06 2012 A2iA S.A. Systems and methods for recognizing information in objects using a mobile device
8719197, Jul 12 2006 TUNGSTEN AUTOMATION CORPORATION Data classification using machine learning techniques
8724907, Mar 28 2012 Open Text Corporation Method and system for using OCR data for grouping and classifying documents
8745488, Jun 30 2010 System and a method for web-based editing of documents online with an editing interface and concurrent display to webpages and print documents
8749839, Mar 24 2005 TUNGSTEN AUTOMATION CORPORATION Systems and methods of processing scanned data
8774516, Feb 10 2009 TUNGSTEN AUTOMATION CORPORATION Systems, methods and computer program products for determining document validity
8805125, Jun 28 2013 GOOGLE LLC Comparing extracted card data using continuous scanning
8813111, Aug 22 2011 Xerox Corporation Photograph-based game
8823991, Mar 24 2005 TUNGSTEN AUTOMATION CORPORATION Systems and methods of processing scanned data
8855375, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
8855425, Feb 10 2009 TUNGSTEN AUTOMATION CORPORATION Systems, methods and computer program products for determining document validity
8879120, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
8879783, Jun 28 2013 GOOGLE LLC Comparing extracted card data with user data
8879846, Feb 10 2009 TUNGSTEN AUTOMATION CORPORATION Systems, methods and computer program products for processing financial documents
8885229, May 03 2013 TUNGSTEN AUTOMATION CORPORATION Systems and methods for detecting and classifying objects in video captured using mobile devices
8908977, Mar 28 2014 Paycasso Verify Ltd System and method for comparing images
8955743, Apr 17 1998 Diebold Nixdorf, Incorporated; DIEBOLD SELF-SERVICE SYSTEMS DIVISION OF DIEBOLD NIXDORF, INCORPORATED Automated banking machine with remote user assistance
8971587, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
8989515, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
8995012, Nov 05 2010 RESEARCH DEVELOPMENT AND MANUFACTURING CORPORATION System for mobile image capture and processing of financial documents
8995769, Oct 26 2011 INTERDIGITAL CE PATENT HOLDINGS, SAS Method and system of obtaining contact information for a person or an entity
9058327, Jan 25 2012 Veritas Technologies LLC Enhancing training of predictive coding systems through user selected text
9058515, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for identification document processing and business workflow integration
9058580, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for identification document processing and business workflow integration
9064316, Jun 28 2012 CHINA CITIC BANK CORPORATION LIMITED, GUANGZHOU BRANCH, AS COLLATERAL AGENT Methods of content-based image identification
9117117, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
9129210, Mar 24 2005 TUNGSTEN AUTOMATION CORPORATION Systems and methods of processing scanned data
9137417, Mar 24 2005 TUNGSTEN AUTOMATION CORPORATION Systems and methods for processing video data
9141926, Apr 23 2013 TUNGSTEN AUTOMATION CORPORATION Smart mobile application development platform
9158967, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
9165187, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
9165188, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
9208536, Sep 27 2013 TUNGSTEN AUTOMATION CORPORATION Systems and methods for three dimensional geometric reconstruction of captured image data
9253349, May 03 2013 TUNGSTEN AUTOMATION CORPORATION Systems and methods for detecting and classifying objects in video captured using mobile devices
9275281, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Mobile image capture, processing, and electronic form generation
9298979, Jan 18 2008 MITEK SYSTEMS, INC Systems and methods for mobile image capture and content processing of driver's licenses
9311531, Mar 13 2013 TUNGSTEN AUTOMATION CORPORATION Systems and methods for classifying objects in digital images captured using mobile devices
9342741, Feb 10 2009 TUNGSTEN AUTOMATION CORPORATION Systems, methods and computer program products for determining document validity
9342742, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
9355312, Mar 13 2013 TUNGSTEN AUTOMATION CORPORATION Systems and methods for classifying objects in digital images captured using mobile devices
9514357, Jan 12 2012 TUNGSTEN AUTOMATION CORPORATION Systems and methods for mobile image capture and processing
9576272, Feb 10 2009 TUNGSTEN AUTOMATION CORPORATION Systems, methods and computer program products for determining document validity
9584729, May 03 2013 TUNGSTEN AUTOMATION CORPORATION Systems and methods for improving video captured using mobile devices
9946954, Sep 27 2013 TUNGSTEN AUTOMATION CORPORATION Determining distance between an object and a capture device based on captured image data
20010027420,
20020030831,
20020054693,
20020069218,
20020113801,
20020122071,
20020126313,
20020165717,
20030002068,
20030007683,
20030026479,
20030030638,
20030044012,
20030046445,
20030053696,
20030063213,
20030086615,
20030095709,
20030101161,
20030117511,
20030120653,
20030142328,
20030151674,
20030156201,
20030197063,
20030210428,
20030223615,
20040019274,
20040021909,
20040022437,
20040049401,
20040083119,
20040090458,
20040093119,
20040102989,
20040111453,
20040143796,
20040169873,
20040169889,
20040175033,
20040181482,
20040190019,
20040245334,
20040261084,
20040263639,
20050021360,
20050030602,
20050046887,
20050050060,
20050054342,
20050060162,
20050063585,
20050065903,
20050080844,
20050100209,
20050131780,
20050134935,
20050141777,
20050151990,
20050160065,
20050163343,
20050180628,
20050180632,
20050193325,
20050204058,
20050206753,
20050212925,
20050216426,
20050216564,
20050228591,
20050234955,
20050246262,
20050265618,
20050271265,
20050273453,
20060013463,
20060017810,
20060023271,
20060031344,
20060033615,
20060047704,
20060048046,
20060074821,
20060089907,
20060093208,
20060095373,
20060095374,
20060098899,
20060112340,
20060114488,
20060115153,
20060120609,
20060126918,
20060147113,
20060159364,
20060161646,
20060164682,
20060195491,
20060203107,
20060206628,
20060212413,
20060215231,
20060219773,
20060222239,
20060235732,
20060235812,
20060236304,
20060242180,
20060256392,
20060257048,
20060263134,
20060265640,
20060268352,
20060268356,
20060268369,
20060279798,
20060282442,
20060282463,
20060282762,
20060288015,
20060294154,
20070002348,
20070002375,
20070003155,
20070003165,
20070005341,
20070016848,
20070030540,
20070035780,
20070036432,
20070046957,
20070046982,
20070047782,
20070065033,
20070086667,
20070109590,
20070118794,
20070128899,
20070133862,
20070165801,
20070172151,
20070177818,
20070204162,
20070239642,
20070250416,
20070252907,
20070260588,
20080004073,
20080005678,
20080068452,
20080082352,
20080086432,
20080086433,
20080095467,
20080097936,
20080130992,
20080133388,
20080137971,
20080144881,
20080147561,
20080147790,
20080166025,
20080175476,
20080177643,
20080183576,
20080199081,
20080212115,
20080215489,
20080219543,
20080225127,
20080232715,
20080235766,
20080253647,
20080294737,
20080298718,
20090015687,
20090073266,
20090089078,
20090103808,
20090132468,
20090132504,
20090141985,
20090154778,
20090159509,
20090175537,
20090185241,
20090214112,
20090225180,
20090228499,
20090285445,
20090324025,
20090324062,
20100007751,
20100045701,
20100060910,
20100060915,
20100062491,
20100169250,
20100202698,
20100202701,
20100214584,
20100232706,
20100280859,
20110013039,
20110025842,
20110025860,
20110032570,
20110055033,
20110090337,
20110091092,
20110116716,
20110129153,
20110137898,
20110145178,
20110182500,
20110196870,
20110200107,
20110246076,
20110249905,
20110279456,
20110280450,
20110285873,
20110285874,
20120008858,
20120019614,
20120038549,
20120069131,
20120077476,
20120092329,
20120105662,
20120113489,
20120114249,
20120116957,
20120131139,
20120134576,
20120162527,
20120194692,
20120215578,
20120230606,
20120236019,
20120272192,
20120284122,
20120290421,
20120293607,
20120294524,
20120300020,
20120301011,
20120308139,
20130004076,
20130022231,
20130027757,
20130060596,
20130073459,
20130088757,
20130090969,
20130097157,
20130117175,
20130121610,
20130124414,
20130142402,
20130152176,
20130182002,
20130182105,
20130182128,
20130182292,
20130182951,
20130182959,
20130182970,
20130182973,
20130185618,
20130188865,
20130198192,
20130198358,
20130223762,
20130230246,
20130251280,
20130268378,
20130268430,
20130287265,
20130287284,
20130297353,
20130308832,
20130329023,
20140003721,
20140006129,
20140006198,
20140012754,
20140047367,
20140055826,
20140079294,
20140108456,
20140153787,
20140153830,
20140164914,
20140172687,
20140181691,
20140201612,
20140207717,
20140233068,
20140254887,
20140270349,
20140270439,
20140270536,
20140316841,
20140317595,
20140327940,
20140328520,
20140333971,
20140368890,
20150040001,
20150040002,
20150086080,
20150098628,
20150170085,
20150254469,
20150317529,
20150324640,
20150339526,
20150347861,
20150355889,
20160019530,
20160028921,
20160034775,
20160055395,
20160063358,
20160112645,
20160125613,
20160147891,
20160171603,
20160320466,
20160350592,
20170024629,
CN101295305,
CN101329731,
CN101493830,
EP549329,
EP723247,
EP767578,
EP809219,
EP843277,
EP936804,
EP1128659,
EP1317133,
EP1319133,
EP1422520,
EP1422920,
EP1956518,
EP1959363,
EP1976259,
EP2472372,
JP11118444,
JP2000067065,
JP2000103628,
JP2000298702,
JP2000354144,
JP2001309128,
JP2002024258,
JP2002312385,
JP2002519766,
JP2003091521,
JP2003196357,
JP2003234888,
JP2003303315,
JP2004005624,
JP2004523022,
JP2005018678,
JP2005085173,
JP2005173730,
JP2006031379,
JP2006185367,
JP2006209588,
JP2006330863,
JP200752670,
JP2008134683,
JP2009015396,
JP2009211431,
JP2011034387,
JP2011055467,
JP2011118513,
JP2011118600,
JP2012194736,
JP2012517637,
JP2013196357,
JP5462286,
JP7260701,
JP9091341,
JP9116720,
JP962826,
TW401553,
WO2063812,
WO263812,
WO2004053630,
WO2004056360,
WO2006104627,
WO2007081147,
WO2007082534,
WO2008008142,
WO2010030056,
WO2010056368,
WO9604749,
WO97006522,
WO9847098,
WO9967731,
///////////////
Executed onAssignorAssigneeConveyanceFrameReelDoc
Mar 12 2013THRASHER, CHRISTOPHER W KOFAX, INCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0426750643 pdf
Mar 12 2013SHUSTOROVICH, ALEXANDERKOFAX, INCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0426750643 pdf
Mar 13 2013AMTRUP, JAN W KOFAX, INCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0426750643 pdf
Mar 14 2013MA, JIYONGKOFAX, INCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0426750643 pdf
Mar 14 2013THOMPSON, STEVEKOFAX, INCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0426750643 pdf
Mar 19 2013MACCIOLA, ANTHONYKOFAX, INCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0426750643 pdf
Dec 20 2016KOFAX, INC.(assignment on the face of the patent)
Jul 07 2017KOFAX, INCCREDIT SUISSESECURITY INTEREST SEE DOCUMENT FOR DETAILS 0431080207 pdf
Jul 20 2022PSIGEN SOFTWARE, INC JPMORGAN CHASE BANK, N A AS COLLATERAL AGENTFIRST LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT0607570565 pdf
Jul 20 2022KOFAX, INCJPMORGAN CHASE BANK, N A AS COLLATERAL AGENTFIRST LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT0607570565 pdf
Jul 20 2022PSIGEN SOFTWARE, INC CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENTSECURITY INTEREST SEE DOCUMENT FOR DETAILS 0607680159 pdf
Jul 20 2022KOFAX, INCCREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENTSECURITY INTEREST SEE DOCUMENT FOR DETAILS 0607680159 pdf
Jul 20 2022Credit Suisse AG, Cayman Islands BranchKAPOW TECHNOLOGIES, INC RELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS 0608050161 pdf
Jul 20 2022Credit Suisse AG, Cayman Islands BranchKOFAX, INCRELEASE BY SECURED PARTY SEE DOCUMENT FOR DETAILS 0608050161 pdf
May 07 2024KOFAX, INCTUNGSTEN AUTOMATION CORPORATIONASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0674280392 pdf
Date Maintenance Fee Events
May 02 2022M1551: Payment of Maintenance Fee, 4th Year, Large Entity.


Date Maintenance Schedule
Nov 13 20214 years fee payment window open
May 13 20226 months grace period start (w surcharge)
Nov 13 2022patent expiry (for year 4)
Nov 13 20242 years to revive unintentionally abandoned end. (for year 4)
Nov 13 20258 years fee payment window open
May 13 20266 months grace period start (w surcharge)
Nov 13 2026patent expiry (for year 8)
Nov 13 20282 years to revive unintentionally abandoned end. (for year 8)
Nov 13 202912 years fee payment window open
May 13 20306 months grace period start (w surcharge)
Nov 13 2030patent expiry (for year 12)
Nov 13 20322 years to revive unintentionally abandoned end. (for year 12)