A method of processing a plurality of still images. The method includes the steps of: detecting an object photographed for each of the still images; arranging the object detected by the step of detecting an object with respect to the plurality of still images and detecting an object photographed by the plurality of still images; relating objects having a strong relationship out of the plurality of objects detected by the step of arranging the object; selecting a still image including at least one of the objects detected by the step of relating objects from the plurality of still images; and outputting the still image selected by the step of selecting a still-image.
|
9. A non-transitory computer-readable medium encoded with computer-executable instructions, the computer-executable instructions, when executed, operable to cause one or more processors to:
detect at least one object in the plurality of still images;
arrange objects detected in the detecting step to determine a plurality of objects within the plurality of still images;
relate objects having a relationship out of the plurality of objects determined in the arranging step;
select a still image including at least one of the objects related in the relating step from the plurality of still images; and
output the selected still image.
1. A method, implemented on a machine comprising a system controller having at least one processor for processing a plurality of still images, comprising the steps of:
detecting, with the system controller, at least one object in the plurality of still images;
arranging, with the system controller, objects detected in the detecting step to determine a plurality of objects within the plurality of still images;
relating, with the system controller, objects having a relationship out of the plurality of objects determined in the arranging step;
selecting, with the system controller, a still image including at least one of the objects related in the relating step from the plurality of still images; and
outputting the selected still image.
10. An image processing apparatus for processing a plurality of still images, comprising:
an object detection section configured for detecting at least one object in the plurality of still images;
an object arrangement section configured for arranging objects detected by the object detection section to determine a plurality of objects within the plurality of still images;
a relating section configured for relating objects having a relationship out of the plurality of objects determined by the object arrangement section;
a still-image selection section configured for selecting a still image including at least one of the objects related by the relating section from the plurality of still images; and
a still-image outputting section configured for outputting the still image selected by the still-image selection section.
2. The method of
displaying a list of the objects determined in the arranging step;
accepting a plurality of object selections from the displayed list; and
selecting objects to set the plurality of objects to the objects having the relationship.
3. The method of
the object includes a person, and
outputting the still image includes printing the still image onto a paper.
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
selecting a still image including the object from the plurality of still images; and
outputting the still image for each object.
|
The present invention contains subject matter related to Japanese Patent Application JP 2006-229289 filed in the Japanese Patent Office on Aug. 25, 2006, the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to an image processing method, an image processing method program, a recording medium recording the image processing method program, and an image processing apparatus, and can be applied to a terminal apparatus for printing photographs, for example. The present invention detects objects having a strong relationship among the objects which have been detected from a plurality of still images, and selects and outputs the still images including at least one of the objects. Thereby, the present invention enables the user to figure out the number of copies for distribution easily to process still images.
2. Description of the Related Art
In recent years, terminal apparatuses for printing photographs taken by electronic still cameras have been installed in large-scale DPE stores, convenience stores, etc. Such a terminal apparatus prints out photographs by the operation of a user himself/herself. When the user attaches a recording medium, such as a memory card, etc., holding still images, which are the result of taking photographs, to the terminal apparatus, a menu including various kinds of selection options is displayed. When the user makes a selection from this menu, for example the still images recorded on this recording medium are displayed as thumbnail images in a list. Also, when the user sets the number of copies of each still image to print, etc., in this list display, and instructs printing, the user is allowed to print the set number of copies of and the set size of the photographs.
On such still image processing, for example Japanese Unexamined Patent Application Publication No. 2002-215643 has proposed a method in which human faces are recognized in still images, and the still images are classified on the basis of the face recognition result.
Photographs taken by an electronic still camera sometimes include one person, and sometimes include many persons. When a photograph including only one person is printed and distributed to that person, only one copy of photograph should be printed. On the other hand, when there are many persons taken in a photograph, there are cases where the many persons are family members, or cases where the many persons are unrelated persons, namely group members who are friends. Accordingly, when a photograph including many persons is printed and distributed to those persons, if this photograph is of a family, only one photograph should be printed. On the other hand, if this photograph is of group members, who are friends and unrelated persons, it becomes necessary to print out the same number of copies of the photograph as the number of persons in the photograph.
Accordingly, to date, when the user prints photographs taken by an electronic still camera using a printer, it has been necessary for the user to check and set the number of copies for distribution of each photograph while checking relationships among the persons in the photographs after all. Thus, there has been a problem in that the setting for the printing is troublesome.
The present invention has been made in view of the above point. It is desirable to propose an image processing method, an image processing method program, a recording medium recording the image processing method program, and an image processing apparatus capable of figuring out the number of copies for distribution easily to process still images.
According to an embodiment of the present invention, there is provided a method of processing a plurality of still images, including the steps of: detecting an object photographed for each of the still images; arranging the object detected by the step of detecting an object with respect to the plurality of still images and detecting an object photographed by the plurality of still images; relating objects having a strong relationship out of the plurality of objects detected by the step of arranging the object; selecting a still image including at least one of the objects detected by the step of relating objects from the plurality of still images; and outputting the still image selected by the step of selecting a still-image.
Also, according to another embodiment of the present invention, there is provided a program of a method of processing a plurality of still images, the program comprising the steps of: detecting an object photographed for each of the still images; arranging the object detected by the step of detecting an object with respect to the plurality of still images and detecting an object photographed by the plurality of still images; relating objects having a strong relationship out of the plurality of objects detected by the step of arranging the object; selecting a still image including at least one of the objects detected by the step of relating objects from the plurality of still images; and outputting the still image selected by the step of selecting a still-image.
Also, according to another embodiment of the present invention, there is provided a recording medium for recording a program of a method of processing a plurality of still images, including the steps of: detecting an object photographed for each still image; arranging the object detected by the step of detecting an object with respect to the plurality of still images and detecting the object photographed by the plurality of still images; relating objects having a strong relationship out of the plurality of objects detected by the object arrangement section; selecting a still image including at least one of the objects, detected by the step of relating, from the plurality of still images; and outputting the still image selected by the step of selecting a still-image.
Also, according to another embodiment of the present invention, there is provided an image processing apparatus for processing a plurality of still images, including: an object detection section detecting an object photographed for each of the still images; an object arrangement section arranging the object detected by the object detection section with respect to the plurality of still images, and detecting an object photographed by the plurality of still images; a relating section detecting objects having a strong relationship out of the plurality of objects detected by the object arrangement section; a still-image selection section selecting a still image including at least one of the objects detected by the relating section from the plurality of still images; and a still-image outputting section outputting the still image selected by the still-image selection section.
By the configuration of the above-described embodiments, even if any number of objects having strong relationships are included in each still image, it is possible to select and output only the still images including the objects having strong relationships from a plurality of still images. Accordingly, when distributing still images including a specific object, even if the specific objects are a plurality of objects, it is possible for the user to select and output still images without duplication of the plurality of the objects, and thus to figure out the number of copies for distribution easily to process still images.
By the present invention, it is possible for the user to figure out the number of copies for distribution easily to process still images.
In the following, a detailed description will be given of embodiments of the present invention with reference to the drawings appropriately.
Here, the image printer 1 is formed to be longitudinally rectangular in shape on the whole, and is provided with an outlet 2 for taking out photographs, which are the print result of the still images at the front. Also, a recording medium insertion slot 3 for loading various kinds of recording media, such as a memory card, a magnetic disk, an optical disc, etc., and a card slot for inserting a card for accounting processing, etc., are provided at the upper side of the outlet 2. Also, the image printer 1 is provided with an image display section 4 on which a touch panel is provided at the upper side of the insertion slot 3. The selection of various kinds of menu options displayed in the image display section 4 is detected by the touch panel in order to accept the user's operation.
The system controller 11 is control means for controlling the operation of the image printer 1, reserves a work area in a random access memory (RAM) 16 in accordance with the record of a read only memory (ROM) 15, and executes the programs recorded in a hard disk 17 by the central processing unit (CPU) 18 to control the operation of each section. In this regard, in this embodiment, the programs executed by the central processing unit 18 are provided by being installed in the image printer 1 in advance. However, the programs may be provided by being recorded in a recording medium, such as an optical disc, a magnetic disk, a memory card, etc., instead. Also, the programs may be provided by being downloaded through a network, such as the Internet, etc.
Thus, the hard disk 17 stores and holds the programs to be executed by the central processing unit 18, and the intermediate processing data obtained by executing the programs. Here, in this embodiment, a person-information data table described below is allocated to one of the intermediate processing data.
In step SP2, the central processing unit 18 displays a print menu screen shown in
When the central processing unit 18 displays the print menu screen, if the user selects the “print for each image”, the processing proceeds from step SP2 to step SP3, the central processing unit 18 displays the still images recorded on the recording medium attached by the user in a list by the thumbnail images. Here,
In the subsequent step SP4, when any one of the thumbnail images is selected, the central processing unit 18 displays a sub-menu for entering the number of print copies and the print size. The central processing unit 18 accepts the change of the print size and the number of print copies in accordance with the user's operation. Also, when the page-scroll button BP is operated at this time, the central processing unit 18 scrolls the list display of the thumbnail images.
Also, after the user has instructed the number of print copies and the print size, in step SP5, when the user operates the OK button BOK, the processing proceeds from step SP5 to step SP6, and the central processing unit 18 controls the printer 14 to print out the still images with the user specified number of print copies and print size. After that, the processing proceeds to step SP7, and the processing procedure is terminated. In this regard, when the user starts the printing, if the user instructs to print the list, the central processing unit 18 prints the list of the thumbnail images.
On the other hand, when the user selects “print for each person” in the print menu screen, the processing of the central processing unit 18 proceeds from step SP2 to step SP8.
In step SP8, the central processing unit 18 creates a person-information data table. Here, the person-information data table is a table in which what objects are included in each of the still images recorded on the attached recording medium. In this embodiment, the objects shown in the person-information data table is a person.
The central processing unit 18 reads the still images recorded on the recording medium in sequence, and extracts individual faces from each image file by applying, for example a method of pattern matching. Also, for example the central processing unit 18 sorts, for each person, each face image extracted by the face determination using, for example the characteristic points. In this regard, this processing is not limited to a method of pattern matching and the face determination using the characteristic points, and various methods can be applied. Thus, the central processing unit 18 detects the objects taken in the photographs for each still image.
Also, the central processing unit 18 arranges the objects detected in each of the still images with respect to a plurality of still images recorded on the recording medium, and detects the objects included in the plurality of the still images. Also, the central processing unit 18 determines what objects are included in each still image, and generates person-information data. Also, the central processing unit 18 summarizes the person-information data with respect to a plurality of the still images recorded on the recording medium to form a person-information data table.
That is to say, for example as shown in
As shown in
Next, the processing of the central processing unit 18 proceeds to step SP9, and the central processing unit 18 changes the display of the image display section 4 to the sub-menu screen of “print for each person”. Here, as shown in
In this sub-menu screen, when the user selects the menu option “print for each individual”, the processing of the central processing unit 18 proceeds from step SP9 to step SP10, and the central processing unit 18 displays the still images including each person by thumbnail images in a list for each person in order from the lower bit of the person-information data.
Here,
Also, the central processing unit 18 displays the page scroll button BP and the OK button BOK at the lower part on the display screen 4. In this regard, in this case, the central processing unit 18 displays by default, for example the print size indicated by the alphabetic character L and one copy of each of the still images for the print size and the number of copies for each thumbnail image, respectively.
In the subsequent step SP11, when any one of the thumbnail images is selected, the central processing unit 18 displays a sub-menu for changing the number of print copies and the print size. The central processing unit 18 accepts the change of the print size and the number of print copies in accordance with the user's operation. Also, when the page-scroll button BP is operated at this time, the central processing unit 18 scrolls the list display of the thumbnail images.
Also, after the user has instructed the number of print copies and the print size, in step SP12, when the user operates the OK button BOK, the processing proceeds from step SP12 to step SP13, and the central processing unit 18 prints out the still images in sequence in accordance with the specified number of print copies and print size. After the printing, the processing of the central processing unit 18 proceeds to step SP7, and the processing procedure is terminated. Accordingly, in this case, the central processing unit 18 prints the photographs including each person for each photographed person in order from the lower bit set in the person-information data. In this regard, when the user starts the printing, if the user instructs to print the list display, the central processing unit 18 also prints the list display of the thumbnail images for each person displayed in step SP10 in this case.
On the other hand, in step SP9, when the menu selection of “print for each group” is detected, the processing of the central processing unit 18 proceeds from step SP9 to step SP15, and the central processing unit 18 accepts the setting of groups. Here, the setting of groups is the processing for detecting the objects having strong relationships out of the plurality of objects detected in the plurality of the still images recorded on the recording medium. In this embodiment, the setting operation by the user is accepted, and the processing is started.
More specifically, the central processing unit 18 displays the relationship setting screen shown in
When the selection of the face image is detected through the touch panel 12, the central processing unit 18 switches the display of the check box W of the selected face image between with check (shown by a check mark in
When the relationship has been set in this manner, and then if the user operates the OK button BOK, the processing of the central processing unit 18 proceeds from step SP15 to step SP16. In step SP16, the central processing unit 18 displays the still images including the persons related in step SP 15 in a list by the thumbnail images.
Here, the central processing unit 18 searches the person-information data table, and detects the still images including any one of the persons related in step SP16, and displays the detected still images in a list by the thumbnail images. Also, the central processing unit 18 summarizes and displays the face images of the persons related in step SP16 at the left end of the list. Accordingly, an example in
Also, the central processing unit 18 displays the page scroll button BP and the OK button BOK at the lower part on the display screen. In this regard, in this case, the central processing unit 18 displays by default, for example the print size indicated by the alphabetic character L and one copy of each of the still images for the print size and the number of copies of each thumbnail image, respectively.
In the subsequent step SP17, when any one of the thumbnail images is selected, the central processing unit 18 displays a sub-menu for changing the number of print copies and the print size. The central processing unit 18 accepts the change of the print size and the number of print copies in accordance with the user's operation. Also, when the page-scroll button BP is operated at this time, the central processing unit 18 scrolls the list display of the thumbnail images.
Also, after the user has instructed the number of print copies and the print size, in step SP18, when the user operates the OK button BOK, the processing proceeds from step SP18 to step SP19, and the central processing unit 18 prints out the still images whose thumbnail images are displayed in accordance with the specified number of print copies and print size. In this regard, when the user starts the printing, if the user instructs to print the list display, the central processing unit 18 also prints the list display of the thumbnail images displayed in step SP16 in this case.
In the subsequent step SP20, the central processing unit 18 changes the display of the screen display section 4 to a menu screen showing menu options “print end” and “print continue”. Here, if the user selects the “print continue” option, the processing of the central processing unit 18 proceeds from step SP20 to step SP15, accepts the setting of relationships for another group once again, and starts the print processing. On the other hand, if the user selects the “print end” option, the processing of the central processing unit 18 proceeds from step SP20 to step SP7, and terminates this processing procedure.
With the above configuration, in the image printer 1 (
However, the still images to be printed in this manner sometimes include one person, and sometimes include many persons. For a still image including only one person, only the copies for that person should be printed. On the other hand, when there are many persons taken in a photograph, there are cases where the many persons are family members, or cases where the many persons are unrelated persons, namely group members who are friends. Accordingly, when a still image including many persons is of a family, only one still image should be printed. On the other hand, if this still image is of group members, who are friends but unrelated persons, it becomes necessary to print out the same number of copies of the still images as the number of persons.
Accordingly, when the user sets the number of print copies by simply viewing the list display of the thumbnail images of the still images recorded on a recording medium, it is necessary for the user to set the number of copies for distribution of each photograph while checking relationships among the persons included in the still images. Thus, the setting for the printing becomes troublesome.
Thus, in the image printer 1, when the user selects the menu option “print for each person” (
Also, when the user instructs to carry out “print for each group” next, the objects included in the plurality of still images are displayed in a list (
In the image printer 1, the still images including at least one object having a relationship set in this manner are selected from the plurality of still images recorded on the recording medium and displayed by the thumbnail images in order to accept the print size and the number of print copies. Accordingly, for example when the user prints a large number of still images taken at a party hall, if the user relates the objects for families, couples, brothers and sisters in advance, the user can print still images for these families, couples, brothers and sisters. Thus, the user can simplify the setting work of the number of print copies at printing time compared with before. Also, the user can prevent from printing redundant copies mistakenly, etc. On the contrary, the user can prevent from falling shortage of print copies mistakenly. Thus, it is possible to reduce various kinds of waste.
Thus, in the image printer 1, by setting the print size and the number of print copies in this list, and instructing to print, the user can print the still images of the desired group members by the set print size and number of print copies.
Also, in the image printer 1, at the time of printing the images, if the user instructs to print a list display, a list display by the thumbnail images of the still images including the group members is printed. Here, the print of the list display by such thumbnail images can be used for checking the number of print copies additionally later, and so on. In the case of this embodiment, the printing of the list of thumbnail images are on the still images including the group members, and thus it is possible to print the list display of the thumbnail images without allowing the group members to be aware of the still images of the complete strangers recorded on the same recording medium. Thus, it is possible to improve user friendliness of the print result.
On the other hand, when the user instructs to carry out “print for each individual” (
Accordingly, for example when the user takes photographs at a party including a plurality of persons, and distributes the copies of the captured still images to each person, it is possible for the user to obtain the print result by sorting for each person, and thus the distribution work of the print result can be simplified. Also, it is possible for the user to reliably print the necessary number of print copies for each person, and to prevent a shortage of copies or a surplus of print copies.
By the above configuration, it is possible to detect objects having a strong relationship among objects which have been detected from a plurality of still images, and to select and output the still images including at least one of the objects. Thereby, the present invention enables the user to figure out the number of copies for distribution easily to process still images.
Also, by displaying the still images including at least one of the objects in a list, and allowing the user to set a strong relationships among the objects, it is possible to process the still images easily, reliably, and furthermore by reflecting various intentions of the user.
Also, since the objects are persons, and the output of the still images is printing the still images onto paper, it is possible for the user to figure out the number of copies for distribution easily to perform print processing of the still images.
Also, by printing a list display of thumbnail images of the still images onto paper, it is possible to additionally print the still images conveniently without showing the still images including the other persons.
Also, by the user selecting the menu and sorting the a plurality of still images for each object detected in the plurality of still images, it is possible for the user to figure out the number of copies for distribution easily to process still images, and to improve user friendliness furthermore.
In this regard, in the above-described embodiment, a description has been given of the case of outputting still images by printing. However, the present invention is not limited to this, and can be widely applied to the case of outputting still images by recording the images onto a recording medium, such as an optical disc, a magnetic disk, a memory card, etc., and further to the case of outputting still images to a network, etc., which is communication means, by attaching the images to a mail, PtoP, etc.
Also, in the above-described embodiment, a description has been given of the case where the objects are persons. However, the present invention is not limited to this, and can be widely applied to the case where for example, the objects are plants and animals, scenes, various kinds of transportation means, etc. That is to say, in this case, it is possible to perform processing, such as selectively printing the still images including either of a specific type of vehicle and a specific person from the recording medium, such as an optical disc, etc., recording a large number of still images.
Also, in the above-described embodiment, a description has been given of the case where still images are simply printed on the basis of the person-information data table. However, the present invention is not limited to this. The created person-information data table may be recorded onto a recording medium, and may be used for the subsequent still image processing.
Also, in the above-described embodiment, a description has been given of the case where the present invention is applied to an image printer. However, the present invention is not limited to this, and can be widely applied to the case of an image processing program in a personal computer, a personal-use printer to which a recording medium is directly attached for printing, a preview apparatus of still image files, etc.
It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
6556775, | Feb 03 1998 | MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD | Image and sound reproduction system |
7340079, | Sep 13 2002 | Sony Corporation | Image recognition apparatus, image recognition processing method, and image recognition program |
7526193, | Jul 15 2003 | Omron Corporation | Object determining device and imaging apparatus |
7548936, | Jan 12 2005 | Microsoft Technology Licensing, LLC | Systems and methods to present web image search results for effective image browsing |
7580587, | Jun 29 2004 | Canon Kabushiki Kaisha | Device and method for correcting image including person area |
7652709, | Jul 31 2003 | ADVANCED INTERCONNECT SYSTEMS LIMITED | Image forming device, image output device, image processing system, image retrieving method, image quality determining method and recording medium |
7720258, | Jan 26 2006 | Adobe Inc | Structured comparison of objects from similar images |
7822233, | Nov 14 2003 | FUJIFILM Business Innovation Corp | Method and apparatus for organizing digital media based on face recognition |
20050105806, | |||
20060204134, | |||
JP2002024299, | |||
JP2002215643, | |||
JP2006067512, | |||
JP2006081021, | |||
JP2006163527, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Aug 14 2007 | KATO, SHINJI | Sony Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 019736 | /0871 | |
Aug 23 2007 | Sony Corporation | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Nov 05 2012 | ASPN: Payor Number Assigned. |
Jul 16 2015 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Jul 16 2019 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Sep 11 2023 | REM: Maintenance Fee Reminder Mailed. |
Feb 26 2024 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
Date | Maintenance Schedule |
Jan 24 2015 | 4 years fee payment window open |
Jul 24 2015 | 6 months grace period start (w surcharge) |
Jan 24 2016 | patent expiry (for year 4) |
Jan 24 2018 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jan 24 2019 | 8 years fee payment window open |
Jul 24 2019 | 6 months grace period start (w surcharge) |
Jan 24 2020 | patent expiry (for year 8) |
Jan 24 2022 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jan 24 2023 | 12 years fee payment window open |
Jul 24 2023 | 6 months grace period start (w surcharge) |
Jan 24 2024 | patent expiry (for year 12) |
Jan 24 2026 | 2 years to revive unintentionally abandoned end. (for year 12) |