Aspects for allowing variably controlled alteration of image processing of digital image data in a digital image capture device include forming an image processing chain with two or more image processors to process digital image data, and providing one or more parametric controls within each of the two or more image processors. The aspects further include accessing chosen controls of the one or more parametric controls to modify the two or more image processors for alteration of the image processing.
|
23. A computer readable medium containing program instructions for:
forming an image processing chain with two or more image processors, the two or more image processors stored in memory, wherein said processors are software modules and each performing a particular type of image transformation, to process digital image data;
providing one or more parametric controls that are uniquely identified and within each of the two or more image processors; and
accessing chosen controls of the one or more parametric controls within each of the two or more image processors to modify the two or more image processors for alteration of the image processing.
0. 30. A method of controlling processing of digital image data in a digital image capture device, comprising:
forming an image processing chain with a plurality of image processors, the plurality of image processors being software modules stored in memory located within the digital image capture device, the image processors each performing a particular type of image processing on the digital image data; and
providing one or more parametric controls within each of the two or more image processors, at least one of the parametric controls accessible by a user of the digital image capture device for modifying at least one of the two or more image processors.
0. 38. A system for controlling the processing of digital image data, comprising:
a digital image capture device having an image processing backplane for forming an image processing chain with a plurality of image processors, the plurality of image processors being software modules stored in memory located within the digital image capture device, the image processors each performing a particular type of image processing on the digital image data; and
a parametric control coupled to at least one of the two or more image processors, wherein the parametric control is accessible by a user of the digital capture device for modifying at least one of the two or more image processors.
0. 26. A method for allowing variable controlled alteration of image processing of digital image data in a digital image capture device, the method comprising:
forming an image processing chain with two or more image processors, the two or more image processors being stored in memory, wherein said processors are software modules and each performing a particular type of processing of digital image data;
providing one or more parametric controls that are uniquely identified and within each of the two or more image processors; and
accessing chosen controls of the one or more parametric controls within at least one of the two or more image processors to modify the image processor for alteration of the image processing.
0. 34. A computer-readable medium having stored thereon instructions which, when executed by a processor, cause the processor to perform the steps of:
forming an image processing chain with a plurality of image processors, the plurality of image processors being software modules stored in memory located within the digital image capture device, the image processors each performing a particular type of image transformation on the digital image data; and
providing one or more parametric controls that are uniquely identified and within each of the two or more image processors, at least one of the parametric controls accessible by a user of the digital image capture device for modifying one of the two or more image processors.
1. A method for allowing variably controlled alteration of image processing of digital image data in a digital image capture device, the method comprising:
forming an image processing chain with two or more image processors, the two or more image processors being stored in memory, wherein said processors are software modules and each performing a particular type of image transformation, to process digital image data;
providing one or more parametric controls that are uniquely identified and within each of the two or more image processors; and
accessing chosen controls of the one or more parametric controls within each of the two or more image processors to modify the two or more image processors for alteration of the image processing.
0. 27. A system for allowing variably controlled alteration of image processing of digital image data, the system comprising:
a digital image capture device, the digital image capture device capable of processing digital image data through two or more image processors, the two or more image processors being stored in memory, wherein said processors are software modules and each performing a particular type of image processing and having one or more parametric controls that are uniquely identified; and
a central processing unit within the digital image capture device and capable of linking the two or more image processors to form an image processing chain, wherein the central processing unit facilitates access of chosen controls of the one or more parametric controls within at least one of the two or more processors for modification of the image processor and alteration of the image processing.
15. A system for allowing variably controlled alteration of image processing of digital image data, the system comprising:
a digital image capture device, the digital image capture device capable of processing digital image data through two or more image processors, the two or more image processors being stored in memory, wherein said processors are software modules and each performing a particular type of image transformation and having one or more parametric controls that are uniquely identified; and
a central processing unit within the digital image capture device and capable of linking the two or more image processors to form an image processing chain, wherein the central processing unit facilitates access of chosen controls of the one or more parametric controls within each of the two or more image processors for modification of the two or more image processors and alteration of the image processing.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. The method of
10. The method of
11. The method of
12. The method of
13. The method of
14. The method of
16. The system of
17. The system of
18. The system of
19. The system of
20. The system of
21. The system of
22. The system of
0. 24. The system of
0. 25. The system of
0. 28. The system of
0. 29. The system of
0. 31. The method of
0. 32. The method of
0. 33. The method of
0. 35. The computer-readable medium of
0. 36. The computer-readable medium of
0. 37. The computer-readable medium of
0. 39. The system of
0. 40. The system of
0. 41. The system of
|
The present invention is related to co-pending U.S. patent application, Ser. No. 08/705,619, filed on Aug. 29, 1996, entitled MODULAR DIGITAL IMAGE PROCESSING VIA AN IMAGE PROCESSING CHAIN, and assigned to the assignee of the present invention.
The present invention is also related to co-pending U.S. patent application, Ser. No. 08/705,588 filed on Sep. Aug. 29, 1996 (which is now U.S. Pat. No. 6,157,394 issued on Dec. 5, 2000), entitled FLEXIBLE DIGITAL IMAGE PROCESSING VIA AN IMAGE PROCESSING CHAIN WITH MODULAR IMAGE PROCESSORS, and assigned to the assignee of the present invention.
The present invention relates to digital image data processing, and more particularly to modular digital image data processing with modifiable parameter control.
Modern digital cameras typically include an imaging device which is controlled by a computer system. The computer system accesses raw image data captured by the imaging device and then processes and compresses the data before storing the compressed data into an internal memory. The conventional digital camera captures image data and then remains unusable until the data is completely processed and stored into internal flash memory.
In processing image data, typical digital cameras operate with exclusive and specific image processing. Thus, all the potential manipulation on image data, such as linearization, sharpening, and compression, occur as a result of isolated preset programming and/or specifically designed hardware.
While some level of manipulation of image data is achieved with the programming or hardware, attempts to alter and improve the processing are hampered by the rigid structure of using a single file/specific components. Furthermore, camera functionality remains tied to technology available at the time of the design and is not readily replaced and updated as technology improves. Accordingly, a need exists for a more flexible, modular approach for processing digital image data that provides enhanced digital image output through an adaptable image processing system.
Accordingly, the present invention meets these needs and provides a method and system for allowing variably controlled alteration of image processing of digital image data in a digital image capture device. In a method aspect, the method includes forming an image processing chain with two or more image processors to process digital image data, and providing one or more parametric controls within each of the two or more image processors. The method further includes accessing chosen controls of the one or more parametric controls to modify the two or more image processors for alteration of the image processing.
In a system aspect, the system includes a digital image capture device, the digital image capture device capable of processing digital image data through two or more image processors, the two or more image processors have one or more parametric controls, and a central processing unit. The central processing is included within the digital image capture device and capable of linking the two or more image processors to form an image processing chain. The central processing unit further facilitates access of chosen controls of the one or more parametric controls for modification of the two or more image processors and alteration of the image processing.
With the present invention, processing of digital image data occurs with a linked series of image processors. Each of the image processors performs some level of manipulation of the digital image data. The separation of digital image processing into a series of image processors allows a more modular approach to processing digital image data. Further, the present invention uniquely allows modification of the series through deletion of an image processor, insertion of a different image processor, or replacement of an existing image processor. In addition, aspects of an image processor, including parameter control values, are alterable in accordance with a preferred embodiment to allow greater adaptability to user-specific design preferences. Enhancements and changes to the chain are therein easily achieved, allowing greater flexibility and more convenient upgrading of digital image processing.
These and other advantages of the aspects of the present invention will be more fully understood in conjunction with the following detailed description and accompanying drawings.
The present invention relates to a flexible, modular approach to processing of digital image data. The following description is presented to enable one of ordinary skill in the art to make and use the invention and is provided in the context of a patent application and its requirements. Various modifications to the preferred embodiment and the generic principles and features described herein will be readily apparent to those skilled in the art.
Although the following describes processing of digital image data captured through a digital camera device, it is meant as an illustrative embodiment of the features of the present invention. The present invention is equally capable of utilization with other devices that perform digital image data capture and processing, including, but not limited to, computer systems, including those used to capture digital images accessible from Internet sites and image scanner equipment. Further, the data structures and commands discussed with reference to a preferred embodiment are suitably included as part of high level code used directly by one or more applications that is readily achieved through the use of C, C++, or other similar programming language, and stored on a computer readable medium.
A digital camera architecture has been disclosed in co-pending U.S. patent application Ser. No. 08/666,241, entitled “A System And Method For Using A Unified Memory Architecture To Implement A Digital Camera Device,” filed on Jun. 20, 1996, and assigned to the Assignee of the present application. The Applicant hereby incorporates the co-pending application by reference, and reproduces portions of that application herein with reference to
Referring now to
Referring now to
U.S. patent application Ser. No. 08/355,031, entitled “A System and Method For Generating a Contrast Overlay as a Focus Assist for an Imaging Device,” filed on Dec. 13, 1994, is incorporated herein by reference and provides a detailed discussion of the preferred elements of imaging device 114. Briefly, imaging device 114 captures an image of object 112 via reflected light impacting image sensor 224 along optical path 236. Image sensor 224 responsively generates a set of raw image data representing the captured image 112. The raw image data is then routed through ASP 228, A/D converter 230 and interface 232. Interface 232 has outputs for controlling ASP 228, motors 234 and timing generator 226. From interface 232, the raw image data passes over system bus 116 to computer 118.
Referring now to
Power manager 342 communicates via line 366 with power supply 356 and coordinates power management operations for camera 110. CPU 344 typically includes a conventional processor device for controlling the operation of camera 110. In the preferred embodiment, CPU 344 is capable of concurrently running multiple software routines to control the various processes of camera 110 within a multi-threading environment. DRAM 346 is a contiguous block of dynamic memory which may be selectively allocated to various storage functions.
I/O 348 is an interface device allowing communications to and from computer 118. For example, I/O 348 permits an external host computer (not shown) to connect to and communicate with computer 118. I/O 348 also permits a camera 110 user to communicate with camera 110 via an external user interface and via an external display panel, referred to as a view finder.
ROM 350 typically comprises a conventional nonvolatile read-only memory which stores a set of computer-readable program instructions to control the operation of camera 110. ROM 350 is further discussed below in conjunction with FIG. 4. Removable memory 354 serves as an additional image data storage area and is preferably a non-volatile device, readily removable and replaceable by a camera 110 user via buffers/connector 352. Thus, a user who possesses several removable memories 354 may replace a full removable memory 354 with an empty removable memory 354 to effectively expand the picture-taking capacity of camera 110. In the preferred embodiment of the present invention, removable memory 354 is typically implemented using a flash disk.
Power supply 356 supplies operating power to the various components of camera 110. In the preferred embodiment, power supply 356 provides operating power to a main power bus 362 and also to a secondary power bus 364. The main power bus 362 provides power to imaging devices 114, I/O 348, ROM 350 and removable memory 354. The secondary power bus 364 provides power to power manager 342, CPU 344 and DRAM 346.
Power supply 356 is connected to main batteries 358 and also to backup batteries 360. In the preferred embodiment, a camera 110 user may also connect power supply 356 to an external power source. During normal operation of power supply 356, the main batteries 358 provide operating power to power supply 356 which then provides the operating power to camera 110 via both main power bus 362 and secondary power bus 364.
During a power failure mode in which the main batteries 358 have failed (when their output voltage has fallen below a minimum operational voltage level) the backup batteries 360 provide operating power to power supply 356 which then provides the operating power only to the secondary power bus 364 of camera 110. Selected components of camera 110 (including DRAM 346) are thus protected against a power failure in main batteries 358.
Power supply 356 preferably also includes a flywheel capacitor connected to the power line coming from the main batteries 358. If the main batteries 358 suddenly fail, the flywheel capacitor temporarily maintains the voltage from the main batteries 358 at a sufficient level, so that computer 118 can protect any image data currently being processed by camera 110 before shutdown occurs.
Referring now to
Referring now to
Removable memory spooler 1 (416) may then access the raw image data from RAM disk 532 via line 616 and store it into removable memory 354 using line 618. Alternatively, if removable memory 354 is full or is not inserted, RAM disk 532 may provide the raw image data directly to image processing/compression 420 using line 620. If removable memory spooler 1 (416) stores the raw image data into removable memory 354, then image processing/compression 420 typically accesses the stored raw image data using line 622.
LINKING IMAGE PROCESSORS FOR FORMING IMAGES
In the preferred embodiment, image processing and compression 420 occurs via an image processing chain (IPC). For purposes of this discussion, the IPC preferably refers to a software process that manipulates image data in a stage by stage fashion. As shown in
The image processors 502 suitably include a first image processor 502a for linearization of the input image data 504. By way of example, linearization refers to a straightforward conversion of the image data from an eight-bit non-linear space to sixteen-bit linear space. As a more specific example, input pixels stored as eight bit compressed Bayer pattern image data are converted through linearization image processor 502a into sixteen bit extended Bayer pattern image data.
A next suitable image processor 502b is a bad pixel replacement processor. Bad pixel replacement suitably occurs through interpolation of the neighborhood pixels around the defective CCD pixels. The processing by image processor 502b capably receives and outputs pixel data in sixteen-bit linear space Bayer format.
As a next image processor 502, white balance processor 502c performs white balance image processing. Pixel data received and output by the white balance image processor 502c are approximately stored in sixteen-bit linear space Bayer format.
A fourth image processor 502d preferably performs image color or color filter array data (CFA) reconstruction. By way of example, the CFA reconstruction image processor 502d suitably achieves an interpolation operation to convert sixteen-bit Bayer CFA pattern CCD data into a forty-eight bit extended RGB image.
Following CFA reconstruction image processor 502d, color transformation image processor 502e is included. An appropriate color transformation image processor 502e employs a color correction matrix, such as to convert from device-dependent camera color space to device-independent linear CCIR709 color space. Preferably, the input and output pixel data is stored in forty-eight bit extended RGB format.
As a next image processor, YCC color space transformation image processor 502f is included. The YCC color space conversion image processor 502f suitably uses CCIR 601-2 specification to create an eight-bit YCrCb image from an RGB image. Input pixel data to image processor 502f is suitably given in forty-eight bit extended RGB format with output pixel data in twenty-four bit YCrCb444 format.
Two additional image processors 502 in the IPC 500 include sharpening image processor 502g and JPEG compression image processor 502h. Sharpening image processor 502g suitably receives input pixel data in twenty-four bit YCrCb format and outputs pixel data in the same format after performing sharpening operations. Parameter control of the sharpening suitably occurs with a range of values for the sharpening operation.
The JPEG compression image processor 502h suitably performs JFIF base line image compression. Input pixel data in twenty-four bit YCrCb444 format is output from image processor 502h as compressed and subsampled YCC format, 48-bit YCrCb411 per 4-pixel data. Two forms of parameter controls are achieved via image processor 502h to both control the degree of compression, e.g., maximum to normal to lossless, and to identify data as color or grayscale.
Coordination of the image processors 502 to form the IPC 500 is preferably done via an image processing backplane (IPB). In a preferred embodiment, the image processing backplane provides processing support in a broad manner to allow varying algorithms to be incorporated as image processors 502. The features of the processing support by the IPB are described in more detail with reference to FIG. 8 and include performing image scan line buffer input/output (I/O), IPC construction and connection, image processor parameter control setting, single pass through image data, procedural interface to the image processors, circular data pipeline support, and ring-pixel handling, with minimal memory requirements and overheads.
For image processors 502, processing suitably occurs with a data pipeline that contains a single image scan line, i.e., an image pixel line in the fast scan direction from left to right. However, some image processors 502, such as compression image processors (e.g., 502h, FIG. 7), utilize more than one scan line during processing to take neighboring effects into account. When more than one scan line is needed by an image processor 502, a data pipeline is suitably defined for convenience at the input end of the image processor 502. For purposes of this discussion, a data pipeline refers to a minimum collection of image scan lines required by an image processor 502. Generally, a data pipeline includes an image scan line currently being processed, and some number of image scan lines prior to (‘lookback’) and/or after (‘lookahead’) the current image scan line. Suitably, access to the data pipeline occurs via a circular array of buffer pointers, so that after each processing iteration of the image processor 502, the pointers in the array are circularly rotated, as is well understood by those skilled in the art. In contrast to prior devices that typically require large amounts of memory to perform image data manipulations, the image scan line buffers provide sufficient memory to perform processing one scan line at a time, thus reducing the overall memory requirements without reducing processing capabilities.
Preferably, the data pipeline required by an image processor 502 is indicated during the installation of the image processor 502 in the IPC 500. Installation of an image processor 502 suitably occurs when the camera first starts up with an IPC 500 constructed from all of the default image processors 502 stored in the system ROM. Suitable functions to coordinate the construction and deconstruction of the IPC 500 include four functions, an initialization function, e.g., IPCInit, an installation function, e.g., IPCInstallImageProcessor, a connection function, e.g., IPCConnect, and a destruction function, e.g., IPCDestroy.
The IPC initialization function is called to create a new IPC 500. A suitable default IPC 500 converts raw CCD capture data into a JPEG compressed image. Preferably, the IPC initialization function returns a reference to a new image processing chain, identifies types of image processors included in the chain, and specifies a maximum expected width in pixels to be sent through the IPC, where the maximum width includes ring-pixels, which refer to supplementary image data at each side of the image required by an image processor to perform a particular algorithm.
The IPC installation function is called by an image processing application to the IPB 520 to install the image processor 502 into the IPC 500. Preferably, the installation function specifies an IPC reference number, as identified in the initialization function, and provides pointers to the seven functional routine entries, as discussed hereinbelow, of the image processor being installed.
The IPC connection function specifies an IPC reference number, and signals to the IPB 520 that all image processors 502 have been installed and that the IPC 500 contains all the required image processors 502 to perform image processing. The IPC destruction function specifies an IPC reference number and is called to destroy an IPC 500. Although a default camera IPC 500 is unlikely to be destroyed, other IPCs added to a camera for other purposes by functions in accordance with a preferred embodiment and discussed in more detail hereinbelow, are suitably destroyed with this function.
MODULARITY OF IMAGE PROCESSORS THROUGH FLEXIBLE UPDATING OF AN IPC
Alterations to an existing IPC 500 readily occur in a preferred embodiment through an update function, e.g., IPBUpdateDefaultPC, that specifies the IPC reference number for the IPC 500 being updated/modified. Updating of an IPC 500 includes insertion of an image processor 502 to the IPC 500, deletion of an image processor 502 from the IPC 500, or replacement of an image processor 502 with an alternate image processor 502. Preferably, the default IPC 500 is updated via an image processor module on a storage device, e.g., removable memory, RAM disk, or internal memory. The image processor module suitably contains one or more plug-in image processors that each have one additional function, e.g., IPMPlugInProc, that defines the updating strategy, the signature of the target image processor to be updated, and pointers of the seven basic functions of an image processor, as described hereinbelow. Lack of identification of valid target image processors in an IPC or lack of match between the format of the output of one image processor and input of a next image processor chained together preferably results in cancellation of the updating attempt and restoration of the default IPC 500.
Defining an image processor 502, for use in a default IPC 500 or as an updating image processor, suitably occurs through seven functional routines or procedures, as indicated by block 528 in
An initialization, e.g., IPInitProc, appropriately allows an image processor 502 to allocate any internal storage it might need when processing an image. It is suitably called by the IPB 520 only once when the image processor 502 is installed into the IPC 500. Subsequent calls to the other five functions described below then pass the internal storage space allocated by the IPInitProc as an argument. Further identified by the initialization function is the maximum width specification of an image scan line in pixels that is expected at the input, including ring-pixels at both the left and right sides. In the example of defining the color correction image processor, the initialization function capably identifies a memory location storing a pointer to needed 3×3 matrix constant value internal variables, and the maximum width.
Two functions, a control function, e.g., IPControlProc, and a status function, e.g., IPStatusProc, deal with parameter controls of an image processor 502. Preferably, parameter controls for an image processor 502 each have a unique 4-character tag that is registered to avoid conflict. Parameter control values include two types, a range type and an enumerated list type. Range types of parameter control values are appropriately confined between the minimum and maximum settings for the range. Enumerated list parameter control values assign different enumerated numbers to different settings with a 32-character null terminated string used to provide an ASCII name for each enumerated list number. Examples of parameter controls include sharpening values (range type), color specification control values (range type), and compression control values (enumerated list type).
The control function, IP ControlProc, is called by the IPB 520 to control the processing parameters one parameter control at a time and only before a reset function, e.g., IPReset, call for every image to be processed. The status function, IPStatusProc, allows an image processor 502 called by the IPB 520 to determine any parameter kind, values types, factory default parameter setting, and current parameter setting of an image processor 502. No-operation routines are provided when the image processor does not support any parameter settings, such as in the example of the color correction image processor.
The reset function, IPResetProc, suitably allows IPB 520 to signal an image processor 502 to reset any internal variables used by the image processor 502 before every image is processed. With the color correction image processor example, no operation routines are provided, since no local variables need to be reset. A process function, e.g., IPProcessProc, suitably allows an image processor 502 to process image data one scan line at a time. It is suitably called whenever a data pipeline for the image processor 502 fills up. Thus, the operations for performing the 3×3 matrix manipulation in the color correction image processor example, are specified with the process function. A destruction function, e.g., IPDestroyProc, suitably allows an image processor to deallocate any internal storage allocated at initialization. It is appropriately called when the IPC 500 containing the image processor 502 is being removed. In response to this call, the image processor 502 preferably deallocates internal storage allocated in the initialization function call.
Entry points to these seven functional routines for an image processor 502, stored in an external data structure, e.g., Functions, as well as the characteristics of the image processor 502, are suitably stored in an internal data structure, e.g., ImageProc, by the IPB 520. Once the image processor 502 are defined through the seven functional routines and connected in an IPC 500, the IPB 520 suitably facilitates image processing operations by managing image buffer I/O, and activation of each image processor 502 as soon as enough input data has been collected. The information for the image data processed is suitably stored in a data structure, e.g., ImageInfo, including raw image size captured by a camera CCD, final processed output image size, bad pixel locations, etc.
MODIFIABLE PARAMETER CONTROL OR IMAGE PROCESSOR IN AN IPC
In a preferred embodiment, the IPB 520 further provides routines to allow exchanges of parameter control settings by an external mechanism, such as a control application 400 (FIG. 4). These functions include parameter control capability and value determination functions, e.g., IPBGetParameterCapability, IPBGetDefaultParameter, and IPBGetParameter. Also included are functions for setting or restoring parameter control values, e.g., IPBSetDefaultParameter, IPBSetParameter, and IPBRestoreParameter. Preferably, for the parameter control value determination functions, an IPC 500 and the number of parameters requested are identified, as well as identification of a pointer to an array of parameter tags, a pointer to a memory location used to store a pointer for the parameter settings returned, and a pointer to a memory location where the number of bytes of parameter control values are stored. Similarly, for the capability determination function, an IPC is specified, the number of parameters requested is specified, a pointer to an array of parameter tags is specified, a pointer to a memory location used to store a pointer for the parameter capability information returned is specified, and a pointer to a memory location where the number of bytes of parameter capability information is stored is specified. Thus, access to the parameter controls managed by an IPC 500 are available, as well as current values, device dependent factory default values, and user-specified default values.
For the parameter control value setting functions, preferably identified by the functions are an IPC, a number of parameter control values to be set, and a pointer to a list of parameter tags and either a current value or a user default value that are to be set by the function. In addition, the set parameter function appropriately allows all parameters not listed in the specified parameter value list to be reset to their user default value when a Boolean variable is set. The restoration function similarly identifies an IPC and a number of parameters to be requested, provides a pointer to an array of parameter tags, and selects the type of parameter defaults, i.e., user-specified or device dependent factory, being reset through a Boolean variable. These functions therefore provide convenient accessibility to allow alteration of parameter control values in an IPC 500. Greater flexibility for adjusting an image processor 502 within an IPC 500 is advantageously provided.
Although the present invention has been described in accordance with the embodiments shown, one of ordinary skill in the art will recognize that there could be variations to the embodiment and those variations would be within the spirit and scope of the present invention. Accordingly, many modifications may be made by one of ordinary skill without departing from the spirit and scope of the present invention, the scope of which is defined by the following claims.
Patent | Priority | Assignee | Title |
11089247, | May 31 2012 | Apple Inc. | Systems and method for reducing fixed pattern noise in image data |
11689826, | May 31 2012 | Apple Inc. | Systems and method for reducing fixed pattern noise in image data |
8259198, | Oct 20 2009 | Apple Inc. | System and method for detecting and correcting defective pixels in an image sensor |
8294781, | Oct 20 2009 | Apple Inc. | System and method for sharpening image data |
8330772, | Oct 20 2009 | Apple Inc. | Image signal processor front-end image data processing system and method |
8358319, | Oct 20 2009 | Apple Inc. | System and method for processing image data using an image processing pipeline of an image signal processor |
8413123, | Dec 04 2009 | Kabushiki Kaisha Toshiba | Compiling device and compiling method |
8471932, | Sep 30 2010 | Apple Inc.; Apple Inc | Spatial filtering for image signal processing |
8472712, | Oct 20 2009 | Apple Inc. | System and method for applying lens shading correction during image processing |
8488055, | Sep 30 2010 | Apple Inc | Flash synchronization using image sensor interface timing signal |
8493482, | Aug 18 2010 | Apple Inc. | Dual image sensor image processing system and method |
8508612, | Sep 30 2010 | Apple Inc | Image signal processor line buffer configuration for processing ram image data |
8508621, | Sep 30 2010 | Apple Inc | Image sensor data formats and memory addressing techniques for image signal processing |
8525895, | Jul 29 2010 | Apple Inc. | Binning compensation filtering techniques for image signal processing |
8531542, | Sep 01 2010 | Apple Inc. | Techniques for acquiring and processing statistics data in an image signal processor |
8593483, | Oct 20 2009 | Apple Inc. | Temporal filtering techniques for image signal processing |
8605167, | Sep 01 2010 | Apple Inc. | Flexible color space selection for auto-white balance processing |
8629913, | Sep 30 2010 | Apple Inc | Overflow control techniques for image signal processing |
8638342, | Oct 20 2009 | Apple Inc. | System and method for demosaicing image data using weighted gradients |
8643770, | Sep 30 2010 | Apple Inc. | Flash synchronization using image sensor interface timing signal |
8736700, | Sep 30 2010 | Apple Inc | Techniques for synchronizing audio and video data in an image signal processing system |
8786625, | Sep 30 2010 | Apple Inc | System and method for processing image data using an image signal processor having back-end processing logic |
8817120, | May 31 2012 | Apple Inc. | Systems and methods for collecting fixed pattern noise statistics of image data |
8872946, | May 31 2012 | Apple Inc. | Systems and methods for raw image processing |
8917336, | May 31 2012 | Apple Inc. | Image signal processing involving geometric distortion correction |
8922704, | Sep 01 2010 | Apple Inc. | Techniques for collection of auto-focus statistics |
8953882, | May 31 2012 | Apple Inc. | Systems and methods for determining noise statistics of image data |
9014504, | May 31 2012 | Apple Inc. | Systems and methods for highlight recovery in an image signal processor |
9025867, | May 31 2012 | Apple Inc. | Systems and methods for YCC image processing |
9031319, | May 31 2012 | Apple Inc. | Systems and methods for luma sharpening |
9077943, | May 31 2012 | Apple Inc. | Local image statistics collection |
9105078, | May 31 2012 | Apple Inc | Systems and methods for local tone mapping |
9131196, | May 31 2012 | Apple Inc. | Systems and methods for defective pixel correction with neighboring pixels |
9142012, | May 31 2012 | Apple Inc. | Systems and methods for chroma noise reduction |
9317930, | May 31 2012 | Apple Inc. | Systems and methods for statistics collection using pixel mask |
9332239, | May 31 2012 | Apple Inc. | Systems and methods for RGB image processing |
9342858, | May 31 2012 | Apple Inc.; Apple Inc | Systems and methods for statistics collection using clipped pixel tracking |
9344613, | Sep 30 2010 | Apple Inc. | Flash synchronization using image sensor interface timing signal |
9398205, | Sep 01 2010 | Apple Inc. | Auto-focus control using image statistics data with coarse and fine auto-focus scores |
9710896, | May 31 2012 | Apple Inc. | Systems and methods for chroma noise reduction |
9741099, | May 31 2012 | Apple Inc. | Systems and methods for local tone mapping |
9743057, | May 31 2012 | Apple Inc. | Systems and methods for lens shading correction |
Patent | Priority | Assignee | Title |
4442543, | Sep 10 1979 | ERIM INTERNATIONAL, INC | Bit enable circuitry for an image analyzer system |
4794524, | Jul 03 1984 | ZiLOG, Inc. | Pipelined single chip microprocessor having on-chip cache and on-chip memory management unit |
4850027, | Jul 26 1985 | International Business Machines Corporation | Configurable parallel pipeline image processing system |
4860375, | Mar 10 1986 | WACHOVIA BANK, NATIONAL | High speed cellular processing system |
4949390, | Apr 16 1987 | LINK AUTOMATION, INC | Interconnect verification using serial neighborhood processors |
5237626, | Sep 12 1991 | INTERNATIONAL BUSINESS MACHINES CORPORATION A CORP OF NY | Universal image processing module |
5283671, | Feb 20 1991 | BARNEYSCAN CORPORATION, A CORP OF CA | Method and apparatus for converting RGB digital data to optimized CMYK digital data |
5315700, | Feb 18 1992 | TRIPATH IMAGING, INC | Method and apparatus for rapidly processing data sequences |
5365596, | Dec 17 1992 | Philip Morris Incorporated | Methods and apparatus for automatic image inspection of continuously moving objects |
5402170, | Dec 11 1991 | Eastman Kodak Company | Hand-manipulated electronic camera tethered to a personal computer |
5452375, | May 24 1993 | SAGEM S A | Digital image processing circuitry |
5477264, | Mar 29 1994 | Intellectual Ventures Fund 83 LLC | Electronic imaging system using a removable software-enhanced storage device |
5513382, | Mar 31 1993 | Apple Inc | Multi-ceramic layer switch circuit |
5563655, | Feb 28 1994 | Intellectual Ventures Fund 83 LLC | Intelligent digital image storage for an electronic camera |
5568192, | Aug 30 1995 | Intel Corporation | Method and apparatus for processing digital video camera signals |
5606365, | Mar 28 1995 | Intellectual Ventures Fund 83 LLC | Interactive camera for network processing of captured images |
5623604, | Nov 18 1992 | Canon Kabushiki Kaisha | Method and apparatus for remotely altering programmable firmware stored in an interactive network board coupled to a network peripheral |
5631974, | Aug 31 1990 | Canon Research Centre Europe, Ltd. | Image processing |
5642444, | Jul 28 1994 | University of North Carolina | Specialized image processing system architecture and method for image data arrays |
5661823, | Sep 29 1989 | Kabushiki Kaisha Toshiba | Image data processing apparatus that automatically sets a data compression rate |
5734425, | Feb 15 1994 | Intellectual Ventures Fund 83 LLC | Electronic still camera with replaceable digital processing program |
5739850, | Nov 30 1993 | Canon Kabushiki Kaisha | Apparatus for improving the image and sound processing capabilities of a camera |
5764817, | Jan 19 1994 | MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD | Image processing apparatus for effecting a series of processes |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Nov 21 2001 | Apple Computer, Inc. | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Apr 27 2007 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Apr 20 2011 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Dec 06 2008 | 4 years fee payment window open |
Jun 06 2009 | 6 months grace period start (w surcharge) |
Dec 06 2009 | patent expiry (for year 4) |
Dec 06 2011 | 2 years to revive unintentionally abandoned end. (for year 4) |
Dec 06 2012 | 8 years fee payment window open |
Jun 06 2013 | 6 months grace period start (w surcharge) |
Dec 06 2013 | patent expiry (for year 8) |
Dec 06 2015 | 2 years to revive unintentionally abandoned end. (for year 8) |
Dec 06 2016 | 12 years fee payment window open |
Jun 06 2017 | 6 months grace period start (w surcharge) |
Dec 06 2017 | patent expiry (for year 12) |
Dec 06 2019 | 2 years to revive unintentionally abandoned end. (for year 12) |