The apparatus of the present invention preferably comprises an image sensor, an orientation sensor, a memory and a processing unit. The image sensor is used for generating captured image data. The orientation sensor is coupled to the image sensor, and is used for generating signals relating to the position of the image sensor. The memory, has an auto-rotate unit comprising program instructions for transforming the captured image data into rotated image data in response to the orientation sensor signals. The processing unit, executes program instructions stored in the memory, and is coupled to the image sensor, the orientation sensor and the memory. The method of the present invention preferably comprises the steps of: generating image data representative of an object with an image sensor; identifying an orientation of the image sensor relative to the object during the generating step; and selectively transferring the image data to an image processing unit in response to the identifying step.
|
0. 36. A method for rotating image data in a digital image capture device, comprising:
capturing image data with an image capture device;
detecting one or more orientations of the image capture device by a plurality of orientation sensors, wherein a first orientation by a first orientation sensor of the plurality of the orientation sensors indicates whether the image capture device is in a landscape or a portrait orientation, and a second orientation by a second orientation sensor of the plurality of the orientation sensors indicates whether the image capture device is in a level or an off-level orientation;
selecting an orientation from the plurality of orientations of the image capture device;
rotating the captured image data according to the selected orientation of the image capture device; and
outputting the rotated image data to a display device in accordance with the selected orientation.
0. 37. A digital camera, comprising:
an image sensor for capturing an image;
a plurality of orientation sensors, coupled to the image sensor, adapted to detect one or more orientations of the captured image, wherein a first orientation by a first orientation sensor of the plurality of the orientation sensors indicates whether the digital camera is in a landscape or a portrait orientation, and a second orientation by a second orientation sensor of the plurality of the orientation sensors indicates whether the digital camera is in a level or an off-level orientation;
an auto-rotate unit, coupled to the plurality of orientation sensors, for selecting an orientation from the plurality of the orientations, and rotating the captured image according to the selected orientation;
a memory, coupled to the auto-rotate unit and the plurality of orientation sensors, for storing the rotated image and the selected orientation; and
an image processing unit, coupled to the memory, for outputting the stored rotated image in accordance with the selected orientation.
0. 1. An apparatus for rotating a display orientation of captured image data representative of an object, the apparatus comprising:
an image sensor, for generating said captured image data;
an orientation sensor coupled to said image sensor, for generating a signal corresponding to the position of the image sensor relative to said object;
a memory, having an auto-rotate unit comprising program instructions for selectively transforming said captured image data into rotated image data in response to said position signal, said memory coupled to said image sensor and to said orientation sensor; and
an image processing unit coupled to said memory for executing program instructions stored in said memory;
wherein (a) said image processing unit processes an i-by-j array of said captured image data and said image sensor generates an i+1-by-j+1 array of said image data, or (b) an image capture unit generates an additional row and column of pixels for said captured image data from said image sensor.
0. 2. The apparatus of
an image processing unit comprising program instructions for transforming one from a group consisting of captured image data and portrait image data, into processed image data.
0. 3. The apparatus of
the auto-rotate unit comprises program instructions for transforming the captured image data into portrait_left image data in response to the portrait_left signal and into portrait_right image data in response to the portrait_right signal.
0. 4. The apparatus of
the signal is a landscape signal if the image sensor is positioned in a level orientation relative to the object; and
the auto-rotate unit comprises program instructions for transforming the captured image data into landscape image data in response to the landscape signal.
0. 5. The apparatus of
the image sensor has a top, a bottom, a right side and a left side;
the auto-rotate unit program instructions transform the captured image data into the portrait_left image data by transferring a prior portrait_left line of image data which starts further toward the bottom of the image sensor and ends further toward the top of the image sensor, then transferring a subsequent portrait_left line of image data, located closer to the right side of the image sensor than the prior portrait_left line of image data, and also starting further toward the bottom of the image sensor and ending further toward the top of the image sensor; and
the auto-rotate unit program instructions transform the captured image data into the portrait_right image data by transferring a prior portrait_right line of image data which starts further toward the top of the image sensor and ends further toward the bottom of the image sensor, then transferring a subsequent portrait_right line of image data, located closer to the left side of the image sensor than the prior portrait_right line of image data, and also starting further toward the top of the image sensor and ending further toward the bottom of the image sensor.
0. 6. The apparatus of
the image sensor has a top, a bottom, a right side and a left side; and
the auto-rotate unit program instructions transform the captured image data into the landscape image data by transferring a prior landscape line of image data which starts further toward the left side of the image sensor and ends further toward the right side of the image sensor, then transferring a subsequent landscape line of image data, located closer to the bottom of the image sensor than the prior landscape line of image data, and also starting further toward the left side of the image sensor and ending further toward the right side of the image sensor.
0. 7. The apparatus of
the portrait_left signal is generated by the orientation sensor if the image sensor is rotated approximately 45° clockwise from the level orientation, and the portrait_right signal is generated by the orientation sensor if the imaging subsystem is rotated approximately 45° counter-clockwise from the level orientation.
0. 8. The apparatus of
the prior portrait_left line of image data and the prior portrait_right line of image data comprise a “green, red, green, red” pixel pattern; and
the subsequent portrait_left line of image data and the subsequent portrait_right line of image data comprise a “blue, green, blue, green” pixel pattern.
0. 9. An apparatus for rotating a display orientation of multicolor captured image data having an i-by-j pixel matrix with a pattern representative of an object, comprising:
an image sensor, for generating the multicolor captured image data;
an input device, for generating a portrait_left signal in response to a first user selection, a portrait_right signal in response to a second user selection, and a landscape signal in response to a third user selection;
a memory, having:
an auto-rotate unit comprising program instructions for selectively transforming the multicolor captured image data into portrait_left image data in response to the portrait-left signal, portrait_right image data in response to the portrait_right signal, and landscape image data in response to the landscape signal; and
an image processing unit comprising program instructions for transforming the portrait_left image data, the portrait_right image data and the landscape image data into processed image data; and
a processing unit, coupled to the image sensor, to the input device, and to the memory, for executing program instructions stored in the memory;
wherein said image processing unit changes the number of pixel rows and pixel columns of the multicolor captured image data such that, from a defined referenced viewpoint, the portrait left image data, the portrait right image data, and the landscape image data, each includes the an (i−1)-by-(j−1) pixel matrix having said pattern.
0. 10. The apparatus of
0. 11. A method for rotating a display orientation of image data representative of an object, comprising the steps of:
generating image data with an image sensor;
identifying an orientation of the image sensor relative to the object at a time substantially simultaneous with the generating step, where said identifying is performed by an orientation sensor; and
selectively transferring data to an image processing unit in response to the identifying step;
wherein said image processing unit rotates said display orientation of said image data and (a) said image processing unit processes an i-by-j array of said captured image data and said image sensor generates an i+1-by-j+1 array of said image data, or (b) an image capture unit generates an additional row and column of pixels for said captured image data from said image sensor.
0. 12. The method of
generating an additional row and column of image data.
0. 13. The method of
0. 14. The method of
identifying a portrait_left orientation, if the left side of the image sensor corresponds to the “top portion” of the object;
identifying a portrait_right orientation, if the right side of the image sensor corresponds to the “top portion” of the object; and
identifying a landscape orientation, if the top of the image sensor corresponds to the “top portion” of the object.
0. 15. The method of
identifying a portrait_left orientation, in response to a user selection of the portrait_left orientation on an input device;
identifying a portrait_right orientation, in response to a user selection of the portrait_right orientation on the input device; and
identifying a landscape orientation, in response to a user selection of the landscape orientation on the input device.
0. 16. The method of
initializing a column variable to a first column of pixel colors required by the image processing unit;
initializing a row variable to a row containing a first pixel color required by the image processing unit;
transferring pixel color at an array location defined by the row variable and the column variable to and the image processing unit;
decrementing the row variable to a row containing a next pixel color required by the image processing unit;
returning to the transferring step, if a row containing a last pixel color has not been transferred;
incrementing the column variable to a next column of pixel colors required by the image processing unit; and
returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.
0. 17. The method of
0. 18. The method of
configuring the image processing unit to accept an image data line length corresponding to the portrait_left orientation; and
performing image processing on a line of transferred image data.
0. 19. The method of
initializing a column variable to a first column of pixel colors required by the image processing unit;
initializing a row variable to a row containing a first pixel color required by the image processing unit;
transferring pixel color at an array location defined by the row variable and the column variable, to the image processing unit;
incrementing the row variable to a row containing a next pixel color required by the image processing unit;
returning to the transferring step, if a row containing a last pixel color has not been transferred;
decrementing the column variable to a next column of pixel colors required by the image processing unit; and
returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.
0. 20. The method of
initializing a row variable to a first row of pixel colors required by the image processing unit;
initializing a column variable to a column containing a first pixel color required by the image processing unit;
transferring pixel color at an array location defined by the row variable and the column variable, to the image processing unit;
incrementing the column variable to a column containing a next pixel color required by the image processing unit;
returning to the transferring step, if a column containing a last color pixel has not been transferred;
incrementing the row variable to a next row of pixel colors required by the image processing unit; and
returning to the initializing a column variable step, if a last row of pixel colors has not been transferred.
0. 21. An apparatus for rotating a display orientation of multicolor image data having an i-by-j pixel matrix with a pattern representative of an object, comprising:
means for generating multicolor image data with an image sensor;
orientation sensor means for identifying an orientation of said image sensor relative to said object at a time substantially simultaneous with said generating said multicolor image data; and
means for selectively transferring said multicolor image data to an image processing unit in response to said means for identifying;
wherein said image processing unit rotates said display orientation of said multicolor image data for providing rotated multicolor image data, and changes the number of pixel rows and pixel columns of said multicolor image data such that, from a defined referenced viewpoint, said rotated multicolor image data includes having an (i−1)-by-(j−1) pixel matrix said pattern.
0. 22. The apparatus of
0. 23. The apparatus of
0. 24. The apparatus of
means for identifying a portrait_left orientation, if the left side of the image sensor corresponds to the “top portion” of the object;
means for identifying a portrait_right orientation, if the right side of the image sensor corresponds to the “top portion” of the object; and
means for identifying a landscape orientation, if the top of the image sensor corresponds to the “top portion” of the object.
0. 25. The apparatus of
means for initializing a column variable to a first column of pixel colors required by the image processing unit;
means for initializing a row variable to a row containing a first pixel color required by the image processing unit;
means for transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;
means for decrementing the row variable to a row containing a next pixel color required by the image processing unit;
means for returning to the means for transferring, if a row containing a last pixel color has not been transferred;
means for incrementing the column variable to a next column of pixel colors required by the image processing unit; and
means for returning to the means for initializing a row variable, if a last column of pixel colors has not been transferred.
0. 26. The apparatus of
means for initializing a column variable to a first column of pixel colors required by the image processing unit;
means for initializing a row variable to a row containing a first pixel color required by the image processing unit;
means for transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;
means for incrementing the row variable to a row containing a next pixel color required by the image processing unit;
means for returning to the means for transferring, if a row containing a last pixel color has not been transferred;
means for decrementing the column variable to a next column of pixel colors required by the image processing unit; and
means for returning to the means for initializing a row variable, if a last column of pixel colors has not been transferred.
0. 27. The apparatus of
means for initializing a row variable to a first row of pixel colors required by the image processing unit;
means for initializing a column variable to a column containing a first pixel color required by the image processing unit;
means for transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;
means for incrementing the column variable to a column containing a next pixel color required by the image processing unit;
means for returning to the means for transferring, if a column containing a last color pixel has not been transferred;
means for incrementing the row variable to a next row of pixel colors required by the image processing unit; and
means for returning to the means for initializing a column variable, if a last row of pixel colors has not been transferred.
0. 28. A computer useable medium embodying computer readable program code for causing a computer to rotate a display orientation of multicolor image data having an i-by-j pixel matrix with a pattern representative of an object, by performing steps comprising:
generating said multicolor image data with an image sensor;
identifying an orientation of the image sensor relative to the object at a time substantially simultaneous with the generating step, wherein said identifying of said orientation is performed with an orientation sensor; and
selectively transferring image data to an image processing unit in response to the identifying step,
wherein said image processing unit rotates said display orientation of said multicolor image data for providing rotated multicolor image data, and changes the number of pixel rows and pixel columns of said multicolor image data such that, from a defined referenced viewpoint, said rotated multicolor image data includes having an (i−1)-by-(j−1) pixel matrix said pattern.
0. 29. The computer useable medium of
0. 30. The computer useable medium of
0. 31. The computer useable medium of
identifying a portrait_left orientation, if the left side of the image sensor corresponds to the “top portion” of the object;
identifying a portrait_right orientation, if the right side of the image sensor corresponds to the “top portion” of the object; and
identifying a landscape orientation, if the top of the image sensor corresponds to the “top portion” of the object.
0. 32. The computer useable medium of
initializing a column variable to a first column of pixel colors required by the image processing unit;
initializing a row variable to a row containing a first pixel color required by the image processing unit;
transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;
decrementing the row variable to a row containing a next pixel color required by the image processing unit;
returning to the transferring step, if a row containing a last pixel color has not been transferred;
incrementing the column variable to a next column of pixel colors required by the image processing unit; and
returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.
0. 33. The computer useable medium of
initializing a column variable to a first column of pixel colors required by the image processing unit;
initializing a row variable to a row containing a first pixel color required by the image processing unit;
transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;
incrementing the row variable to a row containing a next pixel color required by the image processing unit;
returning to the transferring step, if a row containing a last pixel color has not been transferred;
decrementing the column variable to a next column of pixel colors required by the image processing unit; and
returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.
0. 34. The computer useable medium of
initializing a row variable to a first row of pixel colors required by the image processing unit;
initializing a column variable to a column containing a first pixel color required by the image processing unit;
transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;
incrementing the column variable to a column containing a next pixel color required by the image processing unit;
returning to the transferring step, if a column containing a last color pixel has not been transferred;
incrementing the row variable to a next row of pixel colors required by the image processing unit; and
returning to the initializing a column variable step, if a last row of pixel colors has not been transferred.
0. 35. An apparatus for rotating a display orientation of multicolor captured image data having an i-by-j pixel matrix with a pattern representative of an object, comprising:
an image sensor, for generating said multicolor captured image data;
an orientation sensor coupled to said image sensor, for generating a signal corresponding to the position of said image sensor relative to said object; and
a hardware device, having an auto-rotate unit comprising circuits for selectively transforming said multicolor captured image data into rotated multicolor image data in response to said position signal, said hardware device coupled to said image sensor and to said orientation sensor;
wherein, from a defined referenced viewpoint, said rotated multicolor image data includes an (i−1)-by-(j−1) pixel matrix having said pattern.
|
Notice: More than one reissue application has been filed for the reissue of U.S. Pat. No. 6,011,585. This reissue application is a continuation of U.S. Pat. No. RE41,088 (application Ser. No. 11/206,279), which is a reissue of U.S. Pat. No. RE38,896 (application Ser. No. 10/040,249).
This application relates to co-pending U.S. patent application Ser. No. 08/355,031, entitled A System and Method For Generating a Contrast Overlay as a Focus Assist for An Imaging Device, filed on Dec. 13, 1994, by inventor Eric C. Anderson; and U.S. patent application Ser. No. 08/384,012, entitled Apparatus and Method for Camera Image and Orientation Capture, filed on Feb. 6, 1995, by inventor Scott Fullam. The subject matter of the two applications described above is hereby incorporated by reference. These related applications are commonly assigned to Apple Computer, Inc.
1. Field of the Invention
The present invention relates generally to an apparatus and method for orienting an image. More particularly, the present invention is an apparatus and method for rotating a captured image to an orientation corresponding to an imaging subsystem's orientation at the time in which the image was captured.
2. Description of the Background Art
When a digital camera captures an image of an object, the camera's frame of reference with respect to the object produces a desired image orientation. Two conventional image orientations exist, namely, a landscape orientation and a portrait orientation. Referring now to
In a digital camera, an image sensor is comprised of light-sensitive devices, such as charge-coupled devices (CCD), that convert an optical image into a set of electrical signals. Referring now to
Once an image is captured by the digital camera, a set of pixel signals corresponding to the image received by the pixels is processed by an image processing algorithm. Image processing routines are conventionally designed to process pixel signals line-by-line, conforming to a specific and unchanging pixel pattern format. Thus, image sensors manufactured with the Bayer pixel pattern format will be coupled to image processing routines specifically designed to accept pixel signals in alternating sequences of “GRGRGR” and “BGBGBG.” Due to possible imperfections in the outer rows and columns of pixels that make up the image sensor, conventional digital cameras sometimes have image sensors large enough so that one or more lines of pixels at the sides of the image sensor can be ignored.
Referring now to
As previously described, the image processing routines within digital cameras are conventionally designed to process pixel signals on a line-by-line basis according to only one pixel pattern format. Thus, conventional digital cameras process images as if they were always in a landscape format. In the presentation of the landscape image of
What is needed is an apparatus and method that efficiently and automatically rotates a stored photographic image to correspond to the orientation in which the photographic image was captured.
The present invention is an apparatus and method for rotating the display orientation of a captured image. The apparatus of the present invention preferably comprises an image sensor, an orientation sensor, a memory and a processing unit. The image sensor is used for capturing image data. The orientation sensor is coupled to the image sensor, and is used for generating a portrait signal if the image sensor is positioned in a portrait orientation relative to the object. The memory, has an auto-rotate unit comprising program instructions for transforming the captured image data into rotated image data in response to the portrait signal. The processing unit, is used for executing program instructions stored in the memory, and is coupled to the image sensor, the orientation sensor and the memory.
The method of the present invention preferably comprises the steps of: generating image data representative of an object with an image sensor; identifying an orientation of the image sensor relative to the object during the generating step; and selectively transferring the image data to an image processing unit in response to the identifying step.
The present invention is an apparatus and method for rotating the display orientation of a captured image.
Referring now to
Referring now to
Referring now to
Referring now to
As stated above, the RLOS 324 (i.e. the right/left orientation sensor) and the UDOS 326 (i.e. the up/down orientation sensor) are disposed within the imaging subsystem 204. Both the RLOS 324 and the UDOS 326 are physically coupled to the image sensor 304 and electrically coupled to the DSP 320. The RLOS 324 monitors whether the camera 200 is in a landscape orientation or a portrait orientation. If the camera 200 is in a portrait orientation, the RLOS 324 identifies whether the camera 200 has been rotated clockwise or counterclockwise, as viewed from the rear of the camera 200. In response to such camera 200 rotation, the RLOS 324 generates and transmits either a landscape signal, a portrait_left signal or a portrait_right signal to the DSP 320. More specifically, if the camera 200 is held in the landscape orientation shown in
The UDOS 326 monitors whether the camera 200 is in either a level orientation or an off-level orientation. In response to a movement of the camera 200, the UDOS 324 generates and transmits either a level signal or an off-level signal to the DSP 320. Specifically, if the camera 200 is held in the landscape orientation shown in
Both the RLOS 324 and UDOS 326 sensors are preferably identical in their structure and function. Their placement relative to the image sensor 304 determines whether the signal generated by the orientation sensor is one of the RLOS 324 signals or the UDOS 326 signals. The DSP 320 receives both the RLOS 324 and the UDOS 326 signals and transmits them to the processing and storage subsystem 208 over the bus 206. While the preferred embodiment of the present invention incorporates the orientation sensors 324, 326, those skilled in the art will recognize that the orientation sensors 324, 326 are not required to practice the present invention, if the processing and storage subsystem 208 prompts the photographer for the camera's 200 orientation at the time an image is captured. Alternatively, an embodiment using only the RLOS 324 enables practicing of the present invention, but this alternative embodiment results in loss of the off-level signal. However, such an embodiment using only the RLOS 324 is still useful for most applications requiring only landscape and portrait orientation using an approximately level camera 200. A further alternative embodiment includes an RLOS 324 which only generates the landscape, portrait_left, or portrait_right signals if the camera 200 is within a predetermined level range.
Referring now to
The buffer 400 is preferably a first-in, first-out buffer for temporarily holding the captured image data and the imaging subsystem 204 control signals passed between the DSP 320 and the processing and storage subsystem 208. The buffer 400 has data lines that are coupled to both the bus 206 and the internal bus 402. The processing unit 404 executes programming instructions stored in the non-volatile memory 406 and the volatile memory 410 to perform various operations. The non-volatile memory 406 stores a set of computer readable program instructions for controlling how the processing unit 404 accesses, transforms and outputs the image data, as described in detail below with reference to
The input device 408 is preferably a series of control buttons, responsive to user inputs, which generates signals that are translated by the processing unit 404 into control signals to adjust the focus and other operating parameters of the imaging subsystem 204. As mentioned above, while the preferred embodiment of the camera 200 incorporates the RLOS 324 and UDOS 326, an alternative embodiment of the present invention prompts the user to manually input, via the input device 408, the imaging subsystem's 204 orientation at the time the captured image data was generated. Other alternative embodiments incorporating the automatic RLOS 324 and UDOS 326 sensors could provide for an override function that instructs the processing and storage subsystem 208 to ignore the orientation signals generated by the RLOS 324 and UDOS 326 sensors, enabling the user to retain full manual control over the final orientation of the captured image data on the image display 100. The I/O Interface 414 is coupled to the internal bus 402 and has an external port connector for coupling the processing and storage subsystem 208 with a host computer (not shown) for downloading image data stored in the volatile memory 410 and thus freeing up room in the volatile memory 410 for additional sets of processed image data. In the second and alternative embodiment discussed above where the processing and storage subsystem 208 is a desktop type computer, the input device 408 will also include a keyboard and mouse type controller.
Referring now to
The frame buffer 416 is preferably comprised of a dedicated space of contiguous memory comprising an i-row by j-column area for storing image data. There is preferably a constant one-to-one correspondence between the pixels comprising the image sensor 304 array and the storage locations within the frame buffer 416 area. Alternatively, the pixel signals from the image sensor 304 array could be stored within the frame buffer 416 in a non-contiguous (i.e. sectored) memory format, as is commonly known in the art. In the preferred embodiment, i and j are set equivalent to the size of the image sensor 304. Since the preferred image sensor 304 is a 481 by 641 array, i is set to 481 and j is set to 641. In alternative embodiments where the image sensor 304 is only a 480 by 640 array, the frame buffer 416 will still preferably be a 481 by 641 array since, as is discussed below, an extra row and column is generated by the processing and storage subsystem 208.
The working memory 418 preferably provides a temporary space for storing data received from the input device 408 and for data generated by the processing unit 404. The working memory 418 also contains additional memory for storing the RLOS 324 and UDOS 326 signals at the time the image data was captured by the image sensor 304; and additional information conventionally associated with the capture process, such as exposure and focus settings, time and date, statistical data, and the presence of any defective pixels, rows or columns within the image sensor 304.
The storage memory 420 preferably provides space for storing multiple sets of image data, until the images are off-loaded from the camera 200 via the I/O interface 414. The storage memory 420 sets a maximum image data holding capacity of the camera 200. Alternatively, additional memory cards could also be coupled to the internal bus 402, providing even more image data storage capability. Such an additional memory card could be a non-volatile flash memory card.
Referring now to
Referring now to
The image capture unit 428 controls the imaging sub-system's 204 focus and aperture settings; captures the image data; routes the image data into the frame buffer 416; routes the exposure and focus settings, time and date, statistical data, and the presence of any defective image sensor 304 pixels, rows or columns into the miscellaneous data field 425; generates additional rows and columns of image data within the frame buffer 416 if required; monitors the RLOS 324 and UDOS 326; and sets the RLOS signal 421 data field and the UDOS signal 423 data field based on the RLOS 324 and UDOS 326 outputs. In the preferred embodiment the image capture unit 428 corrects any pixel defects within the image sensor 304 using the information stored in the miscellaneous data field 425 before passing the image data to the auto-rotate unit 422. In an alternative embodiment, image processing unit 424 corrects any pixel defects within the image sensor 304 using the information stored in the miscellaneous data field 425 after receiving the image data from the auto-rotate unit 422. In this alternative embodiment, the defective image sensor 304 pixel information must also be rotated by the auto-rotate unit 422 in the same manner as for the image data, described below.
The image capture unit 428 preferably is designed in two embodiments, a hardware embodiment and a software embodiment; however, the hardware embodiment is preferred.
Hardware Embodiment
In the hardware embodiment, the image sensor 304 is comprised of at least one more extra row and column than the image processing unit 424 is designed to process. Thus, if the image processing unit 424 is designed to process a 480 by 640 pixel signal array, then the image sensor 304 must be at least a 481 by 641 pixel array (i.e. “i” is 481 and “j” is 641 in
Software Embodiment
The software embodiment is required when the image sensor 304 lacks any additional rows and columns beyond those that the image processing unit 424 is designed to process. For Bayer pattern image processing, the image capture unit 428 must generate an additional row and column of image data. (For non-Bayer pattern image data, a straightforward rotation is performed as is described below and an additional row and column is not required.) Thus, for example, if the image sensor 304 is comprised of a 480 by 640 pixel array and the image processing unit 424 is designed to process a 480 by 640 array, then the image capture unit 428 must generate an additional row and column of pixels. In the preferred software embodiment, the additional row generated corresponds to row ri (i.e. GRGRGR) in
The auto-rotate unit 422 transforms the captured image data into the rotated image data for further processing by the image processing unit 424. The image processing unit 424 receives the rotated image data from the auto-rotate unit 422 and transforms it into processed image data, ready for saving in the storage memory 420. The operating system unit 426 is coupled to the internal bus 402 and preferably comprises a conventional set of program instructions for controlling the configuration and usage of the computer's 208 hardware resources. The operations performed by the auto-rotate unit 422 and the image processing unit 424 are described in more detail below.
Referring now to
The auto-rotate unit 422 preferably comprises program instructions stored in the non-volatile memory 406 that are executed by the processing unit 404. The auto-rotate unit 422 controls the flow of data between the frame buffer 416 and the image processing unit 424. First, the auto-rotation unit 422 accesses the RLOS signal 421 data field and UDOS signal 423 data field from the frame buffer data structure 417; and, based on this orientation data, selects either a portrait_left spooling routine, a portrait_right spooling routine or a landscape spooling routine. The spooling routines select-out and feed the captured image data stored in the image data field 419 to the image processing unit 424. “Spooling” is here defined as a routine for transferring data from one unit or device to a second unit or device. It is through these spooling routines that the auto-rotate unit 422 transforms the captured image data into the rotated image data. While the effect of the auto-rotate unit 422 is only described below with reference to the captured image data, some of the additional information stored in the working memory 418 (such as the location of any defective pixels, rows or columns within the image sensor 304) will also be rotated, as described above. This additional information is then used by the image processing unit 424. Alternatively, a defective pixel correction algorithm can be applied to the image data prior to rotation of the image data. Each of the different spooling routines are described in detail below.
First, the auto-rotate unit 422 determines the image sensor's 304 orientation at the time the image was captured, by accessing the RLOS signal 421 and UDOS signal 423 data fields. Next, if the RLOS signal 421 data field contains a portrait_left signal and the UDOS signal 423 data field contains a level signal, the auto-rotate unit 422 selects the portrait_left spooling routine. However, if the RLOS signal 421 data field contains a portrait_right signal and the UDOS signal 423 data field contains a level signal, the auto-rotate unit 422 selects the portrait_right spooling routine. Lastly, if the RLOS signal 421 data field contains a landscape signal and the UDOS signal 423 data field contains a level signal, or whenever the UDOS signal 423 data field contains an off-level signal, regardless of what the RLOS signal 421 data field contains, the auto-rotate unit 422 selects the landscape spooling routine. Each of the three spooling routines within the auto-rotate unit 422 are now discussed below.
If the portrait_left spooling routine is selected, the auto-rotate unit 422 interprets the captured image data within the image data field 419 as shown in
If the portrait_right spooling routine is selected, the auto-rotate unit 422 interprets the captured image data within the image data field 419 as shown in
If the landscape spooling routine is selected, the auto-rotate unit 422 interprets the captured image data within the image data field 419 as shown in
Also as introduced above, the image processing unit 424 preferably comprises program instructions stored in the non-volatile memory 406 that are executed using the processing unit 404. The image processing unit 424 receives the rotated image data from the auto-rotate unit 422, performs conventional image processing operations (such as white balancing, reconstruction, color correction, gamma correction, sharpening and compression) on the rotated image data; generates processed image data; and either saves the processed image data to the storage memory 420 or outputs it to a host computer (not shown) via the I/O interface 414. As was discussed above, the image processing unit 424 is preferably implemented such that it must receive all the image data spooled from the frame buffer 416 starting with a “GRGRGR” line followed by a “BGBGBG” line in an alternating line-by-line pattern. This is the image processing unit's 424 design limitation which drives the auto-rotate unit's 422 particular pixel selection routines described above. Alternatively, a different image processing unit 424 could be implemented which requires that the image data be spooled from the frame buffer 416 starting with a “GBGBGB” line followed by a “RGRGRG” line or any combination of the Bayer pattern.
Those skilled in the art will also recognize that alternate embodiments of the present invention that do not use the Bayer pattern may still use the teachings of present invention to rotate their image data. In such alternative embodiments the auto-rotate unit 422 performs a straightforward rotation of the image data since such alternative image processing units 424 may accept pixel signals in any orientation. Thus for the straightforward rotation where the shaded gray area of
Referring now to
In step 612 of
In step 628 of
In step 644 of
While the present invention has been described with reference to certain preferred embodiments, those skilled in the art will recognize that various modifications may be provided. For instance, while the preferred embodiment of the present invention selects and processes the captured image data as a rectangular array, alternate embodiments could select and process the image data as an array of tiles rather than lines. Furthermore, by transferring the pixels signals to the image processing unit 424 in a different order, a mirror image or an upside-down image of the captured image data can be presented on the image display 100. Lastly, while the preferred embodiment is implemented in software, those skilled in the art would readily recognize that a hardware equivalent implementation would also be acceptable. These and other variations upon and modifications to the preferred embodiment are provided for by the present invention which is limited only by the following claims.
Patent | Priority | Assignee | Title |
10088866, | Mar 18 2015 | Motorola Mobility LLC | Controlling the orientation of a device display based on usage context |
10686992, | Jul 31 2018 | Qualcomm Incorporated | Image orientation notification and adjustment |
8451344, | Mar 24 2011 | Amazon Technologies, Inc.; Amazon Technologies, Inc | Electronic devices with side viewing capability |
9007403, | Apr 17 2012 | Delta Electronics, Inc. | Projector, projecting system comprising the same and automatic image adjusting method thereof |
9282222, | Mar 24 2011 | Amazon Technologies, Inc | Electronic devices with side viewing capability |
9489927, | Apr 10 2013 | Canon Kabushiki Kaisha | Information processing device for controlling direction of display image and control method thereof |
Patent | Priority | Assignee | Title |
3814227, | |||
3971065, | Mar 05 1975 | Eastman Kodak Company | Color imaging array |
4364650, | Oct 05 1979 | Fuji Photo Film Co., Ltd. | Exposure control method and device |
5218459, | Sep 16 1991 | Eastman Kodak Company | Print scanner with soft key variable magnification |
5227889, | May 11 1990 | Matsushita Electric Industrial Co., Ltd. | Image slant compensatable video camera |
5521639, | Apr 30 1992 | Sony Corporation | Solid-state imaging apparatus including a reference pixel in the optically-black region |
5576759, | Dec 07 1992 | Nikon Corporation | Image processing system for classifying reduced image data |
5710572, | Apr 08 1994 | FUJIFILM Corporation | Image display apparatus and method |
5900909, | Apr 13 1995 | Intellectual Ventures Fund 83 LLC | Electronic still camera having automatic orientation sensing and image correction |
JP4120889, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Dec 28 2009 | Apple Inc. | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Jul 14 2011 | ASPN: Payor Number Assigned. |
Date | Maintenance Schedule |
Aug 23 2014 | 4 years fee payment window open |
Feb 23 2015 | 6 months grace period start (w surcharge) |
Aug 23 2015 | patent expiry (for year 4) |
Aug 23 2017 | 2 years to revive unintentionally abandoned end. (for year 4) |
Aug 23 2018 | 8 years fee payment window open |
Feb 23 2019 | 6 months grace period start (w surcharge) |
Aug 23 2019 | patent expiry (for year 8) |
Aug 23 2021 | 2 years to revive unintentionally abandoned end. (for year 8) |
Aug 23 2022 | 12 years fee payment window open |
Feb 23 2023 | 6 months grace period start (w surcharge) |
Aug 23 2023 | patent expiry (for year 12) |
Aug 23 2025 | 2 years to revive unintentionally abandoned end. (for year 12) |