robot-related information held at a robot user and data held at a robot manufacturer are used to enhance the user's sense of attachment to the robot. Each terminal device is operable to send information about the robot to a telecommunication line and to receive answer information sent from a server to the robot user via the telecommunication line. The server generates answer information on the basis of robot-related information sent from the personal terminal device and previously stored reference information corresponding to the robot-related information; and sends the answer information to the personal terminal device--via the telecommunication line. The answer information is for example a diagnostic report on the robot.
|
16. A robot device which acts autonomously based on an external and/or an internal factor and comprising:
means for transmitting to a remote location via a telecommunication channel robotic related information indicative of the robot device; and means for receiving from the remote location via a telecommunication channel answer information based on said transmitted robotic related information and reference information stored at said remote location.
10. A robot management method comprising the steps of:
transmitting via a telecommunication line robotic related information relating to a robot which acts autonomously based on an external and/or an internal factor; storing reference information in accordance with the robot; generating answer information based on the robotic related information and the reference information; and transmitting via a telecommunication line the answer information to the owner of the robot.
1. A robot management system comprising:
means for transmitting via a telecommunication line robotic related information relating to a robot which acts autonomously based on an external and/or an internal factor; means for storing reference information in accordance with the robot; means for generating answer information based on the robotic related information and the reference information; and means for transmitting via a telecommunication line the answer information to the owner of the robot.
22. A server for communicating with at least one robot device located remotely from said server, said robot device acting autonomously based on an external and/or an internal factor, said server comprising:
means for receiving from a remote location via a telecommunication channel robotic related information indicative of the robot device; means for storing reference information associated with a respective robot device; and means for transmitting to the remote location via a telecommunication channel answer information based on said received robotic related information and said stored reference information.
2. The system according to
wherein the robotic related information is indicative of the changed growth level of the robot; and wherein the answer information is indicative of a growth transition of the robot which is generated based on the robotic related information and the reference information.
3. The system according to
the robotic related information is indicative of the character of the robot; and the answer information is indicative of a character correction of the robot which is generated based on the robotic related information and the reference information.
4. The system according to
the answer information is indicative of a result of a diagnosis of the robot based on the state information.
5. The system according to
6. The system according to
7. The system according to
the generated answer information is based on a result of differentiation between the plurality of the robotic information.
8. The system according to
9. The system according to
11. The method according to
wherein the robotic related information is indicative of the changed growth level of the robot; and wherein the answer information is indicative of a growth transition of the robot which is generated based on the robotic related information and the reference information.
12. The method according to
the robotic related information is indicative of the character of the robot; and the answer information is indicative of a character correction of the robot which is generated based on the robotic related information and the reference information.
13. The method according to
the answer information is indicative of a result of a diagnosis of the robot based on the state information.
14. The method according to
15. The method according to
17. The robot device of
wherein the robotic related information is indicative of the changed growth level of the robot device; and wherein the answer information is indicative of a growth transition of the robot device which is generated based on the robotic related information and the reference information.
18. The robot device of
wherein the robotic related information is indicative of the character of the robot device; and wherein the answer information is indicative of a character correction of the robot device which is generated based on the robotic related information and the reference information.
19. The robot device of
the answer information is indicative of a result of a diagnosis of the robot device based on the state information.
20. The robot device of
21. The robot device of
23. The server of
wherein the robotic related information is indicative of the changed growth level of the robot device; and wherein the answer information is indicative of a growth transition of the robot device which is generated based on the robotic related information and the reference information.
24. The server of
wherein the robotic related information is indicative of the character of the robot device; and wherein the answer information is indicative of a character correction of the robot device which is generated based on the robotic related information and the reference information.
25. The server of
the answer information is indicative of a result of a diagnosis of the robot device based on the state information.
26. The server of
27. The server of
|
This is a Continuation of application Ser. No. 09/958,363, filed Feb. 19, 2002, U.S. Pat. No. 6,577,924 which is a National Phase Application filed under 35 U.S.C. 371, based on International Application PCT/JP01/00953, filed on Feb. 9, 2001.
The present invention relates to a robotic device management system and method, and an information management apparatus, intended to manage an autonomous type robot which acts autonomously.
Recently, there have been proposed robotic devices each capable of autonomously deciding its own behavior correspondingly to its surrounding environment and internal state. Such robots include for example a type which can have a pseudo-emotion and -instinct (will be referred to as "emotion" and "instinct" hereunder) which are changed according to its environment and internal state, and behave correspondingly to such changes in emotion and instinct, that is, can reflect changes of its emotion and instinct in its action.
A robotic device is produced by a robotic device manufacturer, and sold to a user. Therefore, information on the robotic device is reserved at the manufacturer side. For example, the information stored at the robotic device manufacturer includes a program etc. by which an action of the robotic device is decided according to a slate such as an emotion as in the above.
Therefore, if the data stored at the manufacturer can effectively be utilized, it will have a higher value as a useful information resource. Also, if the data reserved at the manufacturer side can be served to the user for linkage or correlation with data reserved at the user side, it will be more useful. Service to the user of the information thus correlated with the data reserved at the user side will enhance the sense of attachment to his or her robot.
Accordingly, the present invention has an object to overcome the above-mentioned drawbacks of the prior art by providing a robotic device management system and method and an information management apparatus, capable of an effective utilization of information on a robotic device, reserved at a user of the robotic device, and data reserved at the manufacturer of the robotic device and also enhancing the user's sense of attachment to his robotic device.
The above object can be attained by providing a robotic device management system including:
a transmitter to send, to a telecommunication line, information on an autonomous type robotic device which acts autonomously according to an external factor and/or internal factor;
an answer generator to generate answer information based on the robotic device-related information sent from the transmitter via the telecommunication line and reference information previously stored in an information storage means and corresponding to the robotic device-related information and send the answer information to a user of the robotic device via the telecommunication line; and
a receiver to receive the answer information sent from the answer generator to the robotic device user via the telecommunication line.
In the robotic device management system constructed as in the above, the information on the autonomous type robotic device which acts autonomously according to an external factor and/or internal factor is sent from the transmitter to the telecommunication line, the answer information is generated by the answer generator based on the robotic device-related information sent from the transmitter via the telecommunication line and reference information previously stored in the information storage means and corresponding to the robotic device-related information, and it is sent from the answer generator to the robotic device user via the telecommunication line, and the answer information sent from the answer generator via the telecommunication line to the robotic device user is received by the receiver.
Thus, the robotic device user can know the state of his or her robotic device from the answer information.
Also, the above object can be attained by providing a robotic device management method including steps of:
sending, to a telecommunication line, information on an autonomous type robotic device which acts autonomously according to an external factor and/or internal factor;
preparing answer information based on the robotic device-related information sent in the sending step via the telecommunication line and reference information previously stored in an information storage means and corresponding to the robotic device-related information and sending the answer information to a user of the robotic device via the telecommunication line; and
receiving the answer information sent in the answer preparing step to the robotic device user via the telecommunication line.
The above robotic device management method permits the robotic device user to know the state of his or her robotic device from the answer information.
Also, the above object can be attained by providing an information management apparatus including:
means for communicating, via a telecommunication line, with a user of an autonomous type robotic device which acts autonomously according to an external factor and/or internal factor; and
means for receiving information on the robotic device via the communicating means, preparing answer information based on the robotic device-related information and reference information previously stored in an information storage means and corresponding to the robotic device-related information, and sending the answer information to the robotic device user via the telecommunication line.
The information management apparatus constructed as in the above receives, by means of the answer generating means via the communicating means, the robotic device-related information sent from the user of the autonomous type robotic device which acts autonomously according to an external factor and/or internal factor the answer generating means receives, generates, by the answer generating means, answer information based on the robotic device-related information and reference information previously stored in the information storage means and corresponding to the robotic device-related information, and sends, from the answer generating means, the answer information to the robotic device user.
This information management apparatus permits the robotic device user to know the state of his robotic device from the answer information.
Also, the above object can be attained by providing a robotic device management method including steps of:
sending, by a user of an autonomous type robotic device which acts autonomously according to an external factor and/or internal factor, information on the robotic device, and receiving, by means of an information management apparatus, the robotic device-related information sent from the robotic device user;
managing the robotic device by the information management apparatus based on the robotic device-related information received in the receiving step; and
sending, by means of the information management apparatus, information on the state of the robotic device, generated in the managing step to the robotic device user.
The above robotic device management method permits the robotic device user to know the state of his robotic device from the answer information.
The present invention will be further discussed below concerning an embodiment thereof with reference to the accompanying drawings. The embodiment is a robotic device management system for a robotic device as shown in FIG. 1.
In
This robot 1 is of an autonomous type which autonomously decides its own action according to modeled information such as an emotion model and instinct model. More particularly, the emotion model and instinct model are changed in state correspondingly to the environment (external factor) and internal state (internal factor) to allow the robot to act correspondingly to the changes of the states, namely, allow the robot to express its emotion and instinct by an action. The instinct model and emotion model for the robot 1 will further be described later. The robot management system according to the present invention manages such a robot 1.
(1) Construction of the Robot Management System
The robot management system is constructed from a network system generally indicated with a reference 30 in FIG. 2.
The network system 30 includes personal terminal devices 31A to 31D and a registration server 38 connected to the personal terminal devices 31A to 31D by various telecommunication lines 32, 33, 34 and 39, respectively.
In the network system 30, each of the personal terminal devices 31A to 31D has both a function to send information on the robot 1 to the telecommunication line and a function to receive answer information sent from the server 38 to the user of the robot 1 via the telecommunication line, and the server 38 forms an answer generator which generates answer information on the basis of the robot-related information sent from each of the personal terminal devices 31A to 31D via the telecommunication lines 32, 33, 34 and 39 and reference information previously stored in an information storage means and corresponding to the robot-related information and sends the answer information to the personal terminal devices 31A to 31D of the robot user via the telecommunication lines. The network system 30 will further be illustrated and described in the following.
In this network system 30, the personal terminal devices 31A to 31C are connected to an Internet service provider 35 by a satellite communication line 32, cable television line 33 or telephone line 34 as shown. Also, the Internet service provider 35 is connected to the server 38 via an Internet 36, and the personal terminal device 31D is connected directly to the server 38 by the ordinary public network 39.
Each of the personal terminal devices 31A to 31D is an ordinary personal computer installed in an ordinary home, and communicates with the server 38 via the Internet 36 or the ordinary public network 39 to transmit and receive necessary data to and from the server 38 and communicates with the robot 1.
The robot 1 may be connected to the server 38 by a so-called station 50 as shown in FIG. 3. The station 50 incorporates a modem function, and connects with the ordinary public network 39 to send information on the robot 1 placed thereon to the server 38. Note that the station 50 may be provided with a power unit. In this case, the station 50 can send the information on the robot 1 to the server 38 while charging the battery of the robot 1.
The server 38 sends a variety of information to personal terminal devices 31A to 31D having accessed the server 38 via the Internet 36 or ordinary public network 39. It may be a Web server at a manufacturer or distributor 37 of the robot, for example.
The server 38 is constructed as shown by way of example in FIG. 4. As seen, the server 38 includes a LAN (local area network) card 40 incorporating an interface circuit via which the server 38 interfaces with the Internet 36, modem 41 via which the server 38 interfaces with the ordinary public network 39, CPU 42 which controls the entire server 38, temporary storage medium 43 formed from a semiconductor memory which serves as a work memory of the CPU 42, and a storage medium 44 such as a hard disc drive having stored therein various data which are used for the server 38 to perform processing operations as will further be described later.
In the server 38, data and command supplied from the personal terminal devices 31A to 31D having accessed the server 38 via the Internet 36 or ordinary public network 39 are taken into the CPU 42 through the LAN card 40 or modem 41, and a predetermined processing operation is performed based on the data and command and control information stored in the storage medium 44.
The CPU 42 generates on-screen data to be displayed on an appropriate screen for a result of the above processing operation as necessary, and sends it to an appropriate one of the personal terminal devices 31A to 31D through the LAN card 40 or modem 41.
Note that the server 38 is constructed as in the above but of course the present invention is not limited to such a construction.
The network system 30 is constructed as in the above. Next, a procedure effected over this network 30 will be described.
(2) Procedures Effected Over the Network System 30
The procedures effected over the network system 30 include mainly a procedure for registration of a user and a procedure for provision of actual service. The user is registered as in the following.
(2-1) User Registration Procedure
As shown in
Generally, the application for a request includes information required for confirmation of the user and for data distribution such as a user's name, address, e-mail address, ID of the robot and the like.
The server (at the robot manufacturer) 38 operates for such a request application procedure. For example, the server 38 will register a user having applied a request for service. More specifically, the server 38 will register an ID of the robot owned by the user and information on the user in combination.
After completion of the registration, the server 38 sends acceptance confirmation information that the user has been registered to the user 200 by means of a post card, electronic mail (will be referred to as "e-mail" hereunder wherever appropriate), Web site, etc. For example, when the acceptance confirmation information is send by the Web site, the robot user 200 accesses the Web site for making a confirmation procedure. Also, the acceptance confirmation information contains information required for subsequent service such as a user ID, password and registered data (user's name, address, etc.) for example.
With this procedure for registration of the user, the user is registered as the robot user in the server 38, and can subsequently be provided as a so-called registered user with various services as to his robot.
(2-2) Requesting Procedure for a Service
As shown in
Before making a request for a service via Web site, the user accesses the Web site and makes the request by checking any of services, which can be requested as to the robot, including "Diagnosis" 51a, "Greeting card" 51b, "Repair" 51c", "Character correction" 51d, etc. displayed on a screen 51 of the monitor of the personal terminal device, as shown in FIG. 7.
The "Diagnosis" service is to diagnose the robot. The "Greeting card" service is to send a greeting card to the robot user or to the robot itself. The "Repair" service is to repair the robot, and the "Character correction" service is to correct the character of the robot.
The user enters details of a time at, and means, by which his robot is subjected to such a service, correspondingly to the requested service.
The requesting procedure for the "Diagnosis" service is as follows; Namely, the user sends information on his desired period, treatment and means for the "Diagnosis," service to the server 38, and the server 38 will record the information.
For authentication of the user, the user enters information about "User ID" 51e, "Password" 51f and "Billing method" 51g displayed on the screen 51 as shown in FIG. 8. The person having made a request for the above service can be confirmed to be a registered used by entering a user ID and password having been acquired by the registration procedure, for the "User ID" 51e and "Password" 51f. For the "Billing method" 51g, a method of payment for a provided service is to be entered. For this item, the user selects a credit service company and enters his credit card number. Note that the billing method should be a one implemented as an ordinary method in the network.
After registering the information on the above service request, the server 38 will send back the acceptance confirmation information to the user 200 (personal terminal devices 31A to 31D) as shown in FIG. 6.
Then, the server 38 provides a service corresponding to the content of the service request to the user 200. The service is provided according to the content, time and means having been entered by the user 200.
(3) Details of Procedure for Provision of a Service Over the Network System 30
(3-1) Connection Between the Robot and Server
The robot of a user and the server 38 are connected to each other via the network system 30.
As having been described in the above, the robot user can be provided with various services for his robot from the server 38. To request for any of the "Diagnosis", "Repair" and "Character correction" services, the information on the robot of a user should be managed by the server 38. That is, for the "Diagnosis" service, the server 38 refers to the robot state in order to diagnose the robot, and for the "Repair" service, the server 38 refers to the driven state of the robot to detect a portion which has to be repaired. For the "Character correction" service, the server 38 refers to the character state of the robot in order to generate character corrective information for the robot.
Thus, the robot and server 38 are connected to each other in the network as shown in FIG. 9. The server 38 collects information on the robot and stores the robot information therein, and will provide various services as in the above on the basis of the robot information stored therein.
The robot and server 38 are connected to each other via the personal terminal devices 31A to 31D as shown in
The information online uploaded to the server 38 via the personal terminal devices 31A to 31D and station 50 is informative data on the internal state of the robot, such as growth level, character, cumulative time of use, etc. of the robot.
The server 38 stores the uploaded information, and generates information (answer information) for a requested service based on such stored information (robot-related information). More particularly, the server 38 generates answer information on the basis of the robot-related information and reference information previously stored in an information storage means and corresponding to the robot-related information. Then the server 38 sends back the answer information to he user 200. The reference information refers herein to information on the robot when shipped from factory, for example.
Thus, the server 38 collects data on the internal state of the robot via the personal terminal devices 31A to 31D and station 50, and will provide a requested service based on the informative data on the internal state of the robot.
The server 38 may collect the information on the internal state of the robot periodically or in response to each request for such a service. For example, in case the server 38 collects such information periodically, it generates answer information and so on based on latest internal-state information collected periodically and stored as updated in the information storage means in response to each request for a service, and provides the requested service.
The preparation of answer information by the sever 38 and operation effected in the robot (user) correspondingly to the answer information sent from the server 38, will further be described below.
(3-2) Diagnostic Service
As having been described above with reference to
Based on the internal-state data and reference information, the server 38 will generate a diagnostic report indicative of a result of diagnosis as the answer information for the request for the "Diagnosis".
The robot incorporates, as software, an emotion object as a model indicating a modeled emotion for expressing an emotion by an action thereof, and an instinct object indicating a modeled instinct for expressing an instinct by an action thereof, which will further be described later. Such an emotion object and instinct object are changed under the influence of external and internal factors, and the robot decides an action thereof correspondingly to the changes of the objects to express its irritable or gentle character by an action thereof.
Also, the robot includes, as software, a growth object which decides an action correspondingly to each growth level. Owing to this growth object, the robot expresses a growth level by an action thereof.
As in the above, the server 38 collects internal-state data such as robot character information (information on emotion and instinct states), growth level information and time of use (start time), etc. from the robot, and generates a diagnostic report based on the internal-state informative data and reference information. The reference information includes information on a character the robot used to have when shipped from factory. The server 38 compares the reference information with the internal-state informative data collected from the robot as in the above for generation of a diagnostic report. For generation of a diagnostic report, the internal state of the robot is checked in step ST1 as shown in FIG. 10. In next step ST2, the server 38 will generate a diagnostic report based on the result of checking the internal state of the robot. The generated diagnostic report is stored in step ST3.
Then, the server 38 will send the diagnostic report by a post card, e-mail or Web site to the user 200. Thus, the user 200 can know the current character, growth level, etc. of his robot 1.
By registering information as to a desired cycle of periodic "Diagnosis" in the server 38, such "Diagnosis" can be effected periodically.
(3-3) Character Correction Service
The user 200 operates the personal terminal devices 31A to 31D to make a request for "Character-correction" service as having been described in the above with reference to FIG. 7. At this time, the user 200 sends information on his desired character for example along with the request to the server 38.
Based on the character information and reference information, the server 38 prepares a character corrective vaccine as an answer information to the request for "Character correction" service. For example, the character corrective vaccine includes parameter data or software program, which will decide the robot character of the robot.
As having previously been described, the robot has the emotion and instinct objects thereof changed under the influence of an external factor and internal factor, and decides its action based on the changes of the emotion and instinct objects, thereby expressing its irritable or gentle character by an action thereof.
The server 38 collects, from the robot, internal-state information including the states of emotion and instinct objects having been changed under the influence of external and internal factors, and prepares the character corrective vaccine based on the internal-state information and reference information. The reference information includes information on a character the robot used to have when shipped from factory. The server 38 compares the reference information with the information uploaded from the robot to determine the current character of the robot, and prepares a character corrective vaccine corresponding to the current character. More specifically, the server 38 has stored therein a data base in which a difference between a character the user 200 desires and a current character of his robot is correlated with and information intended to compensate the difference. Referring to the data base, the server 38 generates or prepares information (character corrective vaccine) to correct the current character of the robot to a one the user desires.
Then, the server 38 sends the character corrective vaccine to the user 200 (robot 1) via the personal terminal devices 31A to 31D or station 50.
At the user 200, the robot 1 has the character thereof corrected with the character corrective vaccine. The user 200 having received the character corrective vaccine via the personal terminal devices 31A to 31D corrects the character of his robot 1 by administering the character corrective vaccine to update the information on the character of his robot 1. That is, in case the character corrective is effected by the user 200, the character corrective vaccine will work as a prescription for the character correction.
As in the above, for character correction of the robot 1 at the user 200, the character corrective vaccine sent to the station 50 is used to update information on the character of the robot 1. For example, the character corrective vaccine includes information intended to change the emotion object (state of the emotion) and instinct object (state of the instinct). In further detail, the vaccine includes parameter data or software program.
When the character information is updated with the character corrective vaccine, the robot 1 will have the character thereof corrected to a one the user 200 desired.
In the foregoing, there has been described a series of operations for preparation at the server 38 of the character corrective vaccine upon request for "Character correction" service from the user 200 and for correction of the character of the robot 1 with the character corrective vaccine.
The robot 1 can also have the growth level thereof improved with a similar procedure. In the robot 1, is growth state (growth level) is also changed under the influence of external and internal factors. Thus, the robot 1 will act in a manner depending upon the growth state. For this growth level improvement, the server 38 generates, as answer information, growth transition information indicative of a transition of growth based on information on the growth level uploaded from the robot 1 via the personal terminal devices 31A to 31D or station 50 and reference information. The reference information include growth level information indicative of a growth level the robot used to have when shipped from factory. The server 38 compares the reference information with the current growth level of the robot, collected from the robot as in the above, to generate the growth level transition information.
Then, the server 38 sends the growth level transition information to the user 200 (robot 1) via the personal terminal devices 31A to 31D or station 50.
The user 200 having thus received the growth level transition information sent from the server 38 will give an action instruction based on the growth level transition information to his robot 1 in order to improve the growth level of the robot 1.
(3-4) Repair Service
Operating the personal terminal devices 31A to 31D, the user 200 makes a request for diagnosis of his robot to the server 38 as shown in FIG. 11. At this time, the user 200 sends information on the faulty state and current state of the robot 1 and a desired treatment along with the diagnosis request to the server 38. The user 200 sends the state data on the robot 1 to the server 38 via the station 50 for example.
Having received the diagnosis request sent from the user 200, the server 38 will prepare a necessary diagnostic report for repair, as answer information, based on the state data sent along with the diagnosis request and reference information.
The data on the state of the robot 1 includes information on the current states of moving parts of the robot 1, for example, control information for controlling the moving parts. The server 38 has stored therein reference information intended for preparation of a diagnostic report based on such state data. Particularly, the reference information includes information on information on the state of the robot in normal operation, information on an improved (repaired) part corresponding to the state data. Such reference information is stored as a state data base DB2 in the server 38.
The server 38 will send, to the user 200, the diagnostic report prepared based on the state data and state data base DB2 as the reference information, and the user 200 having received the diagnostic report will make a request to the server 38 for repair of his robot based on the diagnostic report.
Upon request for repair, the server 38 will determine how to deal with the request for repair, and select an appropriate one of options of treatment and also an appropriate one of servicemen for the requested repair.
How to deal with the request for repair is determined by checking the current internal state, purchase date, faulty location, history of troubles, etc. of the robot in question, and an appropriate serviceman is selected based on a servicemen data base DB1 stored in the server 38. For example, a serviceman is selected with consideration to whether a serviceman has an appropriate repairing skill and how near a serviceman is to the user having made the request for repair.
Then, the server 38 will make a request to a serviceman 201 thus selected for the repair. For example, the server 38 uses e-mail or the like to send a request to the serviceman 201. At this time, the server 38 will send, to the servicemen 201, necessary data for the repair such as state data on the robot, diagnostic data (how to treat), etc. On the other hand, the server 38 informs the user 200 of data such as the serviceman's name and so forth.
The serviceman 201 having received the request for repair will visit the user 200 to make the requested repair and receive a fee for the repair. Then, the serviceman 201 will report the result of repair, and pay a repair royalty, to the server (robot manufacturer) 38.
As in the above, a requested "Repair" service is provided by the server 38 to the user 200. For example, the serviceman is registered in the servicemen data base DB1.
For example, the server (manufacturer) 38 holds a course or school for the knowledge of the structure and repair of the robots. The course is held periodically by Web site or e-mail for example.
The people having participated in the course requests the server (manufacturer) 38 for authorization of them as qualified servicemen, and the server 38 will authorize those of the people who satisfy predetermined requirements as qualified. This will be described in further detail below.
The server 38 will give points to each of the participants in such a course held periodically, and each participant will send the points gained him or her by the participation to the course to the server 38 for requesting the authorization. The server 38 will authorize the participant as a qualified serviceman based on his points sent from the participant. For example, the server 38 authorizes the course participant by holding a plurality of kinds (for skill levels) of courses and giving a number of points for each course to the participant, or by holding a course consisting of a series of lectures and giving a number of points for participation in each lecture. Thus, a serviceman can be authorized with his skill and level taken in consideration. Also, confirmation of the participation in a course is effected using data such as history of access to Web site or the like.
A serviceman thus authorized can be registered in the server 38 and, upon request, visit the robot user having requested for "Repair" service for repairing the robot of the user.
The repair may be directed for both the hardware and software. Normally, some of the robots have an action program thereof recorded in a memory card or the like. In this case, recovery of the data in the memory card may be included in a repair of the robot.
(3-5) Greeting Card Distribution Service
By operating the personal terminal devices 31A to 31D, the user 200 requests the server 38 for "Greeting card" service. At this time, the user 200 will register, in the server 38, information including "User ID" 51h, "February 7" 51i being a data on which the service in consideration is to be provided, "Mr. Suzuki" 51j to whom the greeting card is destined, content of the greeting cart 51k and "Send greeting card" 51m being a service the user 200 requests as shown in FIG. 12.
The server 38 sends a greeting card as answer information to the user 200 based on the information registered as in the above.
More specifically, when a request for "Greeting card" service as in
For example, th greeting card may be "Your robot is three years old on this birthday. Congratulation!" or "Congratulation for entrance to the primary school!".
As in the above, the server 38 provides various services to the users 200 in the network system 30.
Note that the server 38 may hold and manage the content of a service provided to the user 200, as service history information.
The services provided by the server 38 are not limited to the sending of a diagnostic report as in the above. They may include warranty, insurance, etc. of the robot for example. The server 38 collects internal-state information from the robot. The collected internal-state information on a robot can be used for setting such warranty and insurance services for the robot. More specifically, when the internal-state data of the robot indicates that the robot is in good order, a favorable or extended period of warranty should be set for the robot in consideration. Also, the billing should be made to reflect the warranty and insurance services set for the robot. Also, the warranty and insurance services can be set based on the internal-state data such as a diagnostic report.
(4) Example Construction of the Robot
(4-1) Hardware Configuration of the Robot
As in
As shown in
The head unit 4 houses a microphone 15 working as "ears" of the robot 2, a CCD (charge coupled device) camera 16 as "eyes", a sensor 17, and a speaker 18 as "mouth", disposed in place, respectively.
Further, actuators 191 to 19n are disposed in joints of the leg units 3A to 3D, articulations between the leg units 3A to 3D and body unit 2, an articulation between the head unit 4 and body unit 2, and in an articulation between the tail unit 5 and body unit 2, respectively.
The microphone 15 in the head unit 4 receives a command sound such as "Walk!", "Be prone!" or "Chase the ball!" given as scales to the robot 1 via a sound commander (not shown) from the user to produce a sound signal S1, and sends the sound signal S1 to the controller 10. Also, the CCD camera 16 captures the environment surrounding the robot 1 to produce an image signal S2, and sends the image signal S2 to the controller 10.
As shown in
The battery sensor 12 in the body unit 2 detects the remaining potential in the battery 11, and sends the detected potential as a battery potential detection signal S4 to the controller 10. The thermo-sensor 13 detects the head inside the robot 1 and sends the detected heat as a heat detection signal S5 to the controller 10.
Based on the sound signal S1, image signal S2, pressure detection signal S3, battery potential detection signal S4 and heat detection signal S5 supplied from the microphone 15, CCD camera 16, touch sensor 17, battery sensor 12 and thermo-sensor 13, respectively, the controller 10 judges in what environment the robot 1 stands, and if the user has made any instruction or any action to the robot 1.
The controller 10 decides how to behave or move based on the result of judgment and a control program including a variety of control parameters stored in a memory 10A in advance, and drives, based on the result of decision, an appropriate one of the actuators 191 to 19n to allow the robot 1 to make a behavior or motion by shaking or nodding the head unit 4, wagging a tail 5A of the tail unit 5 or driving each of the leg units 3A to 3D to walk. In the following, a set of behaviors and motions will be referred to as "action".
Also, the controller 10 supplies a predetermined sound signal S6 to the speaker 18 as necessary to provide a sound based on the sound signal S6 to outside and turn on and off an LED (light emitting diode) (not shown) provided at each "eye" position of the robot 1.
Thus, the robot 1 is adapted to autonomously act based on its environment, control program stored in the memory 10A, etc.
(4-2) Software Configuration of the Control Program
In the above software configuration, the meaning-conversion object 20 recognizes external and internal states of the robot 1. More particularly, the meaning-conversion object 20 recognizes the external and internal states based on results of detection by various sensors. And, the meaning-conversion object 20 supplies the result of recognition to the emotion/instinct-generation object 21 and action-decision object 22.
The emotion/instinct-generation object 21 consists of an emotion model to change the emotion according to external and internal factors and an instinct model to change the instinct according to the external and internal factors. It generates information intended for the robot 1 to express its emotion or instinct by an action. Specifically, the emotion/instinct-generation object 21 decides the states of emotion and instinct based on the result of recognition supplied from the meaning-conversion object 20 and various other information. And, if the emotion or instinct exceeds a predetermined level, the emotion/instinct-generation object 21 supplies it to the action-decision object 22.
The action-decision object 22 decides an action to be done by the robot 1. More particularly, the action-decision object 22 decides a next due action of the robot 1 based on the result of recognition supplied from the meaning-conversion object 20 and information supplied from the emotion/instinct generation object 21. The action-decision object 22 informs the action-generation object 23 of the result of decision.
The action-generation object 23 controls the robot 1 to actually make a behavior or motion. Namely, the action-generation object 23 controls relevant devices of the robot 1 based on the information (result of decision) supplied from the action-decision object 22 to behave or act as specified by the information.
Thus the control program is composed of the above-mentioned plurality of objects to control the operation of the robot 1. Each of the objects will further be described below.
Based on a sound signal S1, image signal S2, pressure detection signal S3, battery potential detection signal S4 or heat detection signal S5 from the microphone 15, CCD camera 16, touch sensor 17, battery sensor 12 and thermo-sensor 13, respectively, the meaning-conversion object 20 recognizes predetermined external and internal states of the robot 1 such as "the robot 1 has been hit", "the robot 1 has been patted", "the robot 1 has detected the ball", "the battery potential has become lower" or "the internal temperature has been elevated". Then it supplies the result of recognition to the emotion/instinct-generation object 21 and action-decision object 22.
Based on the result of recognition supplied from the meaning-conversion object 20, personal information 24 which is one of the control parameters stored in the memory 10A, and information representing an action having been done and supplied from the action-decision object 22 as will further be described later, the emotion/instinct-generation object 21 decides emotion and instinct levels of the robot 1. When the emotion and instinct levels exceed predetermined ones, the emotion/instinct-generation object 21 supplies the emotion and instinct levels to the action-decision object 22.
More particularly, the emotion/instinct-generation object 21 has an emotion model holding a total of six parameters each representing the strength of emotions including "joy", "sadness", "anger", "surprise", "fear" and "aversion", respectively, and an instinct model holding a total of four parameters representing the strength of desires for "love", "search", "motion" and "eating", respectively.
The memory 10A has also stored therein, as the personal information 24, data such as one of emotion or desire parameters which has to be increased or decreased in value in response to a result of recognition from the meaning-conversion object 20 and information from the action-decision object 22 noticing that an action has been done and which will further be described later and. For example, when the personal information 24 is that the robot 1 has been "hit", the value of the "angry" parameter is increased while that of the "joy" parameter is decreased. When the robot 1 has been "patted", the value of "joy" parameter is increased while that of the "angry" parameter is decreased. When the robot 1 has made an action, the value of the "desire for motion" parameter and that of the "joy" parameter are both increased.
In response to a result of recognition supplied from the meaning-conversion object 20, information supplied from the action-decision object 22, etc., the emotion/instinct-generation object 21 periodically updates the value of an appropriate emotion or desire parameter in the emotion or instinct model on the basis of the personal information.
When as a result of updating of each parameter value, the value of any emotion or desired parameter exceeds a preset value for the emotion or instinct, the emotion/instinct-generation object 21 informs the action-decision object 22 of the fact.
The action-decision object 22 decides a next action based on a result of recognition supplied from the meaning-conversion object 20, information from the emotion/instinct-generation object 21 and an action model 25 which is one of the control parameters stored in the memory 10A, and supplies the action-generation object 23 with the result of decision.
Note that in this embodiment of the present invention, the action-decision object 22 adopts, as a measure for decision of a next action, an algorithm called "probabilistic automaton" to stochastically decide one of nodes NODE0 to NODEn shown in
More specifically, the memory 10A has stored therein, as the action model 25, a state transition table 26 as shown in
That is, in the state transition table 26, input events (results of recognition) taken as conditions for transition between the nodes NODE0 to NODEm are entered in an "Input event name" column in the order of precedence, and additional conditional data to the transition conditions are entered in "Data name" and "Data range" columns.
Therefore, as shown in the state transition table 26 in
Also, the node NODE100 can transit to another node when any of the parameters "JOY", "SURPRISE" and "SADNESS" included in the emotion and desire parameters in the emotion and instinct models in the emotion/instinct-generation object 21 to which the action-decision object 22 refers periodically takes a value of "50, 100 (50 to 100)" even with entry of no result of recognition.
In the state transition table 26, names of the nodes to which each of the nodes. NODE0 to NODEn can transit are given in a "Transition destination nodes" line covered by a "Probability of transition to other node (Di)" column, and probabilities of transition to other nodes NODE0 to NODEn, which would be when all the requirements given in the "Input event name", "Data name" and "Date range" columns, respectively, are satisfied are given in an "Output action" line covered by the "Probability of transition to other node (Di)" column. Note that the sum of the transition probabilities in the line covered by the "Probability of transition to other node Di" column is 100 (%).
Therefore, the node NODE100 in the state transition table 26 shown in
The action model 25 is constructed so that many of the nodes NODE0 to NODEn given in the state transition table 26 are connected to each other. When a result of recognition is supplied from the meaning-conversion object 20 or when a notice is supplied from the emotion/instinct-generation object 21, the action-decision object 22 is adapted to stochastically decide a next action or motion by using the state transition table 26 stored in the memory 10A for appropriate ones of the nodes NODE0 to NODEn and inform the action-generation object 23 of the result of decision.
The action-generation object 23 drives and controls an appropriate one of the actuators 191 to 19n as necessary based on the information from the action-decision object 22 for the robot 1 to act or move as specified, generates an appropriate sound signal S6 (as shown in
Thus, the robot 1 can make an autonomous action correspondingly to its own state and surrounding environment, and a user's instruction and operation made to the robot 1 on the basis of the control program.
Next, the growth model incorporated in the robot 1 to have the latter act in each of its growth steps will be described.
The growth model is to allow the robot 1 to show an "advanced action" in each higher "growth" step like a real animal on which the robot 1 is modeled.
Namely the growth model allows the robot 1 to make an action corresponding to each level of growth (degree of growth). More specifically, the robot 1 is designed to act in each of four "growth steps" including "infancy", "boyhood", "adolescence" and "adulthood" according to the growth model.
The growth model includes the above-mentioned action model (action-decision object) for each of the four "growth steps" including "infancy", "boyhood", "adolescence" and "adulthood". A model corresponding to each growth step is selected to have the robot 1 act correspondingly to its step of growth. For example, a difference between two successive ones of the "growth steps" in the growth model is such that an action or motion in a higher growth step is more difficult or complicated than in the precedent growth step.
More specifically, the action model for the "infancy" is such that for example "walking" is "toddling" with short steps and "motions" are "simple" ones such as "toddle", "stand", "lie" and the like. "Action" is a "monotonous" one such as repetition of a motion, and "sound" is a "low and short" one amplified by a low factor.
A transition is made from one to another step of growth by always monitoring and counting occurrence of a plurality of elements (will be referred to as "growth element" hereunder) associated with a predetermined "growth" such as a predetermined action and motion.
More particularly, when a sum of cumulative occurrences of each growth element (will be referred to as "total empirical value of growth element" hereunder) in which the cumulative occurrences (count) of the growth element are taken as information indicative of a growth degree of the growth element, exceeds a preset threshold, another action model is selected for use, that is, a transition is made from for example the "infancy" action model to the "boyhood" one which higher in growth level (difficulty and complexity of action or motion) than the "infancy" action model, thereby allowing a transition to a next growth step.
It should be reminded here that the growth elements include for example input of a command by scale from a sound commander (in a remote controller), input of "padding" and "hitting" from the touch sensor 17, intensified learning including a number of successes in a predetermined action and motion, input of other than "padding" and "hitting" from the touch sensor 17, and a predetermined action and motion such as "playing with a ball".
Then, after transition of the growth step from the "infancy" action model, the action model for the "boyhood" is followed to control the actuators 191 to 19n and sound output from the speaker 18 in order to increase the rotation speed of each of the actuators 191 to 19n for a "little more positive" walking, increase the number of "motions" for a "little more advanced and complicate" motion, decide a next "action" with reference to the preceding "action" for an "action with a little more definite intention", and increase the length of "sound" signal and amplify the signal with an increased factor for a "little longer and louder" cry.
Similarly, each time the total empirical value of the growth element exceeds each of thresholds preset for the "adolescence" and "adulthood" after transition through the "boyhood", the action model is sequentially changed to the "adolescence" or "adulthood" higher in "growth step" than the "boyhood" and each of these action models is followed to gradually increase the rotation speed of the actuators 191 to 19n and the length and amplification factor of a sound signal supplied to the speaker 18.
Following the growth model, the robot 1 "toddling" in its "infancy" can "walk more positively" as the "growth step" rises (namely, the action model transits from the "infancy" to the "boyhood", from the "boyhood" to the "adolescence", and from the "adolescence" to "adulthood"), the robot 1 "moving simply" in its "infancy" can "move in a more advanced and complicated manner", and the robot 1 acting "monotonously" in its "infancy" can "act with an intention. Also, the robot 1 can generate a "longer and louder"sound as it grows while it has made a "low and short" sound in its "infancy".
Following the growth model, the robot 1 can make an autonomous action in each of the growth steps correspondingly to its own state and surrounding environment and user's instruction and action made to the robot 1.
The robot management system collects, into the server 38, parameters indicative of states of the emotion, instinct and growth of the robot 1, and generates answer information to a request for diagnosis based on the parameters.
Kasuga, Tomoaki, Konishi, Tetsuya, Okita, Ayako, Matsuoka, Tsunetaro, Otsuki, Tadashi, Takemoto, Kunio, Fujita, Yaeko, Ogura, Toshiya
Patent | Priority | Assignee | Title |
10012985, | Jan 05 2011 | SPHERO, INC. | Self-propelled device for interpreting input from a controller device |
10022643, | Jan 05 2011 | SPHERO, INC. | Magnetically coupled accessory for a self-propelled device |
10056791, | Jul 13 2012 | SPHERO, INC | Self-optimizing power transfer |
10168701, | Jan 05 2011 | SPHERO, INC. | Multi-purposed self-propelled device |
10192310, | May 14 2012 | SPHERO, INC. | Operating a computing device by detecting rounded objects in an image |
10248118, | Jan 05 2011 | SPHERO, INC. | Remotely controlling a self-propelled device in a virtualized environment |
10281915, | Jan 05 2011 | SPHERO, INC | Multi-purposed self-propelled device |
10423155, | Jan 05 2011 | SPHERO, INC. | Self propelled device with magnetic coupling |
10525312, | Mar 25 2011 | VOLTEON LLC | Device for displaying in response to a sensed motion |
10620622, | Dec 20 2013 | SPHERO, INC. | Self-propelled device with center of mass drive system |
10678235, | Jan 05 2011 | SPHERO, INC. | Self-propelled device with actively engaged drive system |
10689242, | Mar 13 2013 | Berg Company, LLC | Wireless control system for dispensing beverages from a bottle |
10926140, | Mar 25 2011 | May Patents Ltd. | Device for displaying in response to a sensed motion |
10953290, | Mar 25 2011 | May Patents Ltd.; MAY PATENTS LTD | Device for displaying in response to a sensed motion |
11141629, | Mar 25 2011 | May Patents Ltd.; MAY PATENTS LTD | Device for displaying in response to a sensed motion |
11173353, | Mar 25 2011 | VOLTEON LLC | Device for displaying in response to a sensed motion |
11192002, | Mar 25 2011 | May Patents Ltd.; MAY PATENTS LTD | Device for displaying in response to a sensed motion |
11260273, | Mar 25 2011 | VOLTEON LLC | Device for displaying in response to a sensed motion |
11298593, | Mar 25 2011 | VOLTEON LLC | Device for displaying in response to a sensed motion |
11305160, | Mar 25 2011 | May Patents Ltd. | Device for displaying in response to a sensed motion |
11454963, | Dec 20 2013 | SPHERO, INC. | Self-propelled device with center of mass drive system |
11460837, | Jan 05 2011 | SPHERO, INC. | Self-propelled device with actively engaged drive system |
11605977, | Mar 25 2011 | May Patents Ltd. | Device for displaying in response to a sensed motion |
11630457, | Jan 05 2011 | SPHERO, INC. | Multi-purposed self-propelled device |
11631994, | Mar 25 2011 | May Patents Ltd. | Device for displaying in response to a sensed motion |
11631996, | Mar 25 2011 | May Patents Ltd. | Device for displaying in response to a sensed motion |
11689055, | Mar 25 2011 | May Patents Ltd. | System and method for a motion sensing device |
11916401, | Mar 25 2011 | May Patents Ltd. | Device for displaying in response to a sensed motion |
6922610, | May 02 2002 | JAPAN AEROSPACE EXPLORATION AGENCY | Robot having offset rotary joints |
7584020, | Jul 05 2006 | Humatics Corporation | Occupancy change detection system and method |
7587260, | Jul 05 2006 | Humatics Corporation | Autonomous navigation system and method |
7620477, | Jul 05 2006 | Humatics Corporation | Robotic intelligence kernel |
7668621, | Jul 05 2006 | Humatics Corporation | Robotic guarded motion system and method |
7801644, | Jul 05 2006 | Humatics Corporation | Generic robot architecture |
7974738, | Jul 05 2006 | Humatics Corporation | Robotics virtual rail system and method |
8073564, | Jul 05 2006 | Humatics Corporation | Multi-robot control interface |
8121729, | Feb 13 2007 | ABB Schweiz AG | Remote diagnostic system for robots |
8271132, | Mar 13 2008 | Humatics Corporation | System and method for seamless task-directed autonomy for robots |
8271157, | May 30 2005 | Mobile mechanism, member for mobile mechanism and control method for mobile mechanism | |
8355818, | Sep 03 2009 | Humatics Corporation | Robots, systems, and methods for hazard evaluation and visualization |
8571781, | Jan 05 2011 | SPHERO, INC | Self-propelled device with actively engaged drive system |
8751063, | Jan 05 2011 | SPHERO, INC | Orienting a user interface of a controller for operating a self-propelled device |
8965578, | Mar 16 2011 | Battelle Energy Alliance, LLC | Real time explosive hazard information sensing, processing, and communication for autonomous operation |
9050932, | Jan 05 2011 | Orbotix, Inc. | Magnetically coupled accessory for a self-propelled device |
9090214, | Jan 05 2011 | SPHERO, INC | Magnetically coupled accessory for a self-propelled device |
9114838, | Jan 05 2011 | SPHERO, INC | Self-propelled device for interpreting input from a controller device |
9150263, | Jan 05 2011 | SPHERO, INC | Self-propelled device implementing three-dimensional control |
9193404, | Jan 05 2011 | SPHERO, INC | Self-propelled device with actively engaged drive system |
9211920, | Jan 05 2011 | SPHERO, INC | Magnetically coupled accessory for a self-propelled device |
9213934, | Mar 16 2011 | Battelle Energy Alliance, LLC | Real time explosive hazard information sensing, processing, and communication for autonomous operation |
9218316, | Jan 05 2011 | SPHERO, INC | Remotely controlling a self-propelled device in a virtualized environment |
9280717, | May 14 2012 | SPHERO, INC | Operating a computing device by detecting rounded objects in an image |
9290220, | Jan 05 2011 | SPHERO, INC | Orienting a user interface of a controller for operating a self-propelled device |
9292758, | May 14 2012 | SPHERO, INC | Augmentation of elements in data content |
9342073, | Jan 05 2011 | SPHERO, INC. | Self propelled device with magnetic coupling |
9389612, | Jan 05 2011 | SPHERO, INC. | Self-propelled device implementing three-dimensional control |
9394016, | Jan 05 2011 | SPHERO, INC. | Self-propelled device for interpreting input from a controller device |
9395725, | Jan 05 2011 | SPHERO, INC. | Self-propelled device implementing three-dimensional control |
9415514, | Sep 07 2012 | FANUC AMERICA CORPORATION | System to monitor/analyze robot related information and display on a smart device |
9429940, | Jan 05 2011 | SPHERO, INC | Self propelled device with magnetic coupling |
9457730, | Jan 05 2011 | SPHERO, INC. | Self propelled device with magnetic coupling |
9481410, | Jan 05 2011 | SPHERO, INC. | Magnetically coupled accessory for a self-propelled device |
9483876, | May 14 2012 | SPHERO, INC. | Augmentation of elements in a data content |
9545542, | Mar 25 2011 | VOLTEON LLC | System and method for a motion sensing device which provides a visual or audible indication |
9555292, | Mar 25 2011 | May Patents Ltd.; MAY PATENTS LTD | System and method for a motion sensing device which provides a visual or audible indication |
9592428, | Mar 25 2011 | May Patents Ltd. | System and method for a motion sensing device which provides a visual or audible indication |
9630062, | Mar 25 2011 | VOLTEON LLC | System and method for a motion sensing device which provides a visual or audible indication |
9757624, | Mar 25 2011 | May Patents Ltd. | Motion sensing device which provides a visual indication with a wireless signal |
9764201, | Mar 25 2011 | VOLTEON LLC | Motion sensing device with an accelerometer and a digital display |
9766620, | Jan 05 2011 | SPHERO, INC. | Self-propelled device with actively engaged drive system |
9782637, | Mar 25 2011 | VOLTEON LLC | Motion sensing device which provides a signal in response to the sensed motion |
9791858, | Jan 05 2011 | System and method for controlling a self-propelled device using a dynamically configurable instruction library | |
9808678, | Mar 25 2011 | VOLTEON LLC | Device for displaying in respose to a sensed motion |
9827487, | May 14 2012 | SPHERO, INC | Interactive augmented reality using a self-propelled device |
9829882, | Dec 20 2013 | SPHERO, INC | Self-propelled device with center of mass drive system |
9836046, | Jan 05 2011 | SPHERO, INC | System and method for controlling a self-propelled device using a dynamically configurable instruction library |
9841758, | Jan 05 2011 | SPHERO, INC. | Orienting a user interface of a controller for operating a self-propelled device |
9868034, | Mar 25 2011 | VOLTEON LLC | System and method for a motion sensing device which provides a visual or audible indication |
9878214, | Mar 25 2011 | May Patents Ltd.; MAY PATENTS LTD | System and method for a motion sensing device which provides a visual or audible indication |
9878228, | Mar 25 2011 | May Patents Ltd. | System and method for a motion sensing device which provides a visual or audible indication |
9886032, | Jan 05 2011 | SPHERO, INC. | Self propelled device with magnetic coupling |
9952590, | Jan 05 2011 | SPHERO, INC. | Self-propelled device implementing three-dimensional control |
Patent | Priority | Assignee | Title |
5615296, | Nov 12 1993 | Nuance Communications, Inc | Continuous speech recognition and voice response system and method to enable conversational dialogues with microprocessors |
5836771, | Dec 02 1996 | HANGER SOLUTIONS, LLC | Learning method and system based on questioning |
5934910, | Dec 02 1996 | HANGER SOLUTIONS, LLC | Learning method and system based on questioning |
6289513, | Jun 01 1999 | MAIMONIDES INNOVATIVE TECHNOLOGIES LTD | Interactive application generation and text processing |
6336029, | Dec 02 1996 | HANGER SOLUTIONS, LLC | Method and system for providing information in response to questions |
6480698, | Dec 02 1996 | HANGER SOLUTIONS, LLC | Learning method and system based on questioning |
6501937, | Dec 02 1996 | HANGER SOLUTIONS, LLC | Learning method and system based on questioning |
6577924, | Feb 09 2000 | Sony Corporation | Robot managing system, robot managing method, and information managing device |
6615109, | Feb 10 2000 | Sony Corporation | System and method for generating an action of an automatic apparatus |
20020018984, | |||
20030060930, | |||
20030158628, | |||
20030158629, | |||
EP898237, | |||
JP10154181, | |||
JP1049221, | |||
JP1124744, | |||
JP6110540, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
May 19 2003 | Sony Corporation | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Mar 10 2008 | REM: Maintenance Fee Reminder Mailed. |
Aug 31 2008 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
Date | Maintenance Schedule |
Aug 31 2007 | 4 years fee payment window open |
Mar 02 2008 | 6 months grace period start (w surcharge) |
Aug 31 2008 | patent expiry (for year 4) |
Aug 31 2010 | 2 years to revive unintentionally abandoned end. (for year 4) |
Aug 31 2011 | 8 years fee payment window open |
Mar 02 2012 | 6 months grace period start (w surcharge) |
Aug 31 2012 | patent expiry (for year 8) |
Aug 31 2014 | 2 years to revive unintentionally abandoned end. (for year 8) |
Aug 31 2015 | 12 years fee payment window open |
Mar 02 2016 | 6 months grace period start (w surcharge) |
Aug 31 2016 | patent expiry (for year 12) |
Aug 31 2018 | 2 years to revive unintentionally abandoned end. (for year 12) |