candidate character strings representing objects disposed in a work cell, models for robot operation instructions which has variable parts, and robot commands related to the objects are defined in advance. By inputting a query such as ‘Workpiece 1 ?’ by voice, the object concerned is indicated by a display color so that the work cell can be confirmed. models for operation instructions and a program to be edited are displayed to allow an operator to select the portion to be edited. When an operation instruction is input by voice in the model pattern, candidate character strings are assigned to the variable parts of the model. A completed statement that matches the voice-recognized statement is found, the robot operation commands defined for the objects are displayed, and the robot operation is displayed on the screen. The operation commands are inserted at the desired location in the program.
|
1. A program editing apparatus that arranges and displays three-dimensional models of at least one robot and at least one peripheral object on a screen, and edits a teaching program for the robot, the program editing apparatus comprising:
voice input means;
means for storing a plurality of models each of which allows one or more character strings to be assigned in certain places therein to complete a statement;
candidate character string storing means for storing a plurality of candidate character strings to be assigned to each of the plurality of models;
correspondence storing means for storing correspondence between a statement completed by assigning the candidate character strings to the model and a command to be used in instruction of the robot;
search means for finding a statement, if any, that matches a statement input from the voice input means, in the statements completed by assigning any of the candidate character strings to any of the stored models; and
means for converting the matching statement found by the search means to a robot command according to the correspondence stored in the correspondence storing means and for inserting the robot command into the teaching program.
2. The program editing apparatus according to
3. The program editing apparatus according to
4. The program editing apparatus according to
5. The program editing apparatus according to
object associated command storing means for storing an operation command for a robot related to an object represented by one of the three-dimensional models, in association with the object; and
means for converting a candidate character string assigned in a statement found by the search means into a robot command according to the contents stored in the object associated command storing means, in case where said candidate character string is an identification character string.
6. The program editing apparatus according to
7. The program editing apparatus according to
8. The program editing apparatus according to
9. The program editing apparatus according to
|
1. Field of the Invention
The present invention relates to a program editing apparatus that can be used for editing a teaching program for a robot system.
2. Description of the Related Art
Teach pendants and offline programming systems have been used to edit (create, modify, add code to, etc.) teaching programs for robot systems. When a teach pendant is used for programming, for example, the programming is performed in the programming language of the robot, and the robot may also be moved, for example, to teach positions and the like. Programming and teaching are also frequently carried out by offline programming methods using a keyboard or mouse.
The creation and editing of teaching programs for a robot system thus involves tasks such as programming at a teach pendant or in an offline programming system, as well as teaching and modification of the program. These tasks generally require the mastering of a programming language specific to the robot, and demand complicated input procedures with a keyboardor another input device, making specialized knowledge and skills indispensable. These tasks have accordingly been difficult for non-specialists lacking such knowledge and skills. No examples of the use of voice input to solve this problem have been disclosed in the literature.
The present invention relates to a program editing apparatus that arranges and displays three-dimensional models of at least one robot and at least one peripheral object on a screen, and edits a teaching program instructing the robot how to operate. The program editing apparatus comprises: voice input means; means for storing a plurality of models each of which allows one or more character strings to be assigned in certain places therein to complete a statement; candidate character string storing means for storing a plurality of candidate character strings to be assigned to each of the plurality of models; correspondence storing means for storing correspondence between a statement completed by assigning the candidate character strings to the model and a command to be used in instruction of the robot; search means for finding a statement, if any, that matches a statement input from the voice input means, in the statements completed by assigning any of the candidate character strings to any of the stored models; and means for converting the matching statement found by the search means to a robot command according to the correspondence stored in the correspondence storing means and for inserting the robot command into the teaching program.
The program editing apparatus may take the following form.
The program editing apparatus may have means for displaying character strings of the models on the screen during editing of the teaching program.
The program editing apparatus may have means for displaying contents of the teaching program on the screen during editing of the teaching program, enabling confirmation of the result of insertion of the robot command in the teaching program.
The candidate character strings may be identification character strings that can identify objects represented by the above three-dimensional models.
The program editing apparatus may have object associated command storing means for storing an operation command for a robot related to an object represented by one of the three-dimensional models, in association with the object, and means for converting a candidate character string assigned in a statement found by the search means into a robot command according to the contents stored in the object associated command storing means, in case where the candidate character string is an identification character string.
The program editing apparatus may have means for operating the three-dimensional model of the robot on the screen, according to the robot command.
The program editing apparatus may have further means for causing the display state to change so as to indicate that the robot or the workpiece on the screen has been recognized, when the robot or the workpiece is specified by voice.
The robot command may be a macro command that includes at least one of a robot operation command, an input/output command to and from a peripheral unit, and a conditional statement.
According to the present invention, provided is program editing apparatus that accepts voice input and thereby inputs related operations and sequences, enabling programming to be performed easily without the need to master specialized knowledge and skills. As a result, person-hours and the human burden required for system construction can be reduced.
An embodiment of the present invention will be described with reference to the drawings. First,
The memory is stored with three-dimensional shape data including dimensional data (simply called three-dimensional shape data hereinafter) describing a robot to be displayed and programmed as described below, three-dimensional shape data describing workpieces and, if necessary, three-dimensional shape data describing a vision sensor (imaging device), tools (such as a hand, arc welding torch, and spot welding gun) mounted on the robot, and peripheral objects (such as, for example, workpiece fixtures and a machine tool).
The memory is also stored with programs, settings, and various other data for operating the graphic control circuit to display the objects defined by these three-dimensional models in a display space (three-dimensional space used for programming) and to display motion corresponding to robot commands in animation form. The positions and orientations of the objects (three-dimensional models) in the display space are defined by work cell data. A work cell is a unit of location for the robot system to be programmed; the work cell includes data specifying objects (three-dimensional models of a robot, workpieces, peripheral devices, and the like) in the robot system being programmed and data specifying initial positions and orientations of the specified objects. The data defining the work cell can be changed at any time by, for example, operating the keyboard or mouse.
In addition to the structure and functions described above, the program editing apparatus according to this embodiment is provided with software (a voice recognition program and other software) for program editing (described below in detail) based on voice input through the microphone and for related display and voice output by procedures and processing that will be described below.
In this embodiment, preparations necessary for program editing based on voice input are performed first by the procedure indicated by the flowchart in
Step S1: Candidate character strings corresponding to a plurality of objects that may be disposed in the work cell corresponding to the program to be edited are input by, for example, using the keyboard. The candidate character strings include robot names, workpiece names, peripheral object names, and so on. When a character string is input, it is stored in memory (nonvolatile memory) in the program editing apparatus. The candidate character strings are in one-to-one correspondence with the object label data in the data describing the work cell. When the work cell is displayed on the display, therefore, if an object disposed in the work cell is specified by a character string, the CPU can determine which object the character string identifies. In this embodiment, an object disposed in the work cell can be specified by voice input (in query form), as described below.
The plurality of candidate character strings that are input are structured so that whether the character strings indicate robot names or the names of objects other than robots can be identified. For example, the candidate character strings for robot names may be distinguished by black circles (●), and the candidate character strings for the names of non-robot objects, such as workpieces, peripheral objects and tools, by white circles (∘).
As an example, it will be assumed here that the following candidate character strings have been input.
●robot 1
●robot 2
∘workpiece 1
∘workpiece 2
∘workpiece 3
∘workpiece 4
∘workpiece 5
∘machine tool 1
∘workpiece port 1 (workpiece port of machine tool 1)
∘table 1
Upon the completion of candidate character string input, the sequence proceeds to step S2.
Step S2: A plurality of models for robot operation instructions are input by, for example, using the keyboard; the coding of a model for a robot operation instruction comprises variable and non-variable parts.
Each variable part of the model is stored as a variable term signifying that the name of a robot is to be assigned, or a variable term signifying that the name of an object other than a robot is to be assigned.
In the following description of models for robot operation instructions, variable terms to which a robot name is to be assigned will be represented as ‘robot X’, ‘robot Y’, and so on, while variable terms to which the names of objects other than robots are to be assigned will be represented as ‘object A’, ‘object B’, and so on. This notation will also be applied to the display screen. It should be noted here that ‘robot X’ and ‘robot Y’ do not refer to the names of particular robots, nor do ‘object A’ and ‘object B’ refer to the names of particular objects other than robots.
Each non-variable part of the model includes a character or character string representing a predicate, a particle, a noun, or another word. A character string representing a noun may include a character string matching one of the specific terms used in the robot operation commands input in step S3 below. In the description that follows, character strings of this type will be represented as specific names enclosed by double quotation marks, e.g., “arc welding position” and “spot welding position”.
In a model for a robot operation instruction, accordingly, as shown in
Step S3: Robot commands related to non-robot objects indicated by candidate character strings supplied in step S1 are now defined. In the case where a model which states ‘robot X’ picks up ‘object A’ is prepared, as shown in
Next, in this embodiment, the work cell is confirmed by the procedure indicated by the flowchart in
Step T1: The work cell associated with the program to be edited is displayed. It will be assumed that the work cell has already been defined by well-known methods. For example, a model for a work cell in which a robot, workpieces, peripheral objects, and other objects are provisionally disposed is prepared by external CAD apparatus (not shown) in advance, and the model is taken in the program editing apparatus via the communication interface, thereby subjecting to necessary modification, additions, deletions and other processing to define the work cell. Of course, other methods can be used; for example, existing data for a similar work cell may be taken in from other electronic data storage media (such as a hard disk or flexible magnetic disk in a personal computer) or a new work cell may be defined by means of the program editing apparatus.
Objects with the following names are disposed in the work cell in this example (the parenthesized items are reference characters used in
robot 1 (RB1)
workpiece 1 (W1)
workpiece 2 (W2)
workpiece 3 (W3)
workpiece 4 (W4)
workpiece 5 (W5)
machine tool 1 (MC1)
workpiece port 1 (MH1)
table 1 (TB1)
Reference character H1 in
Step T2: Voice recognition mode 1 is initiated. In voice recognition mode 1, a sentence, clause, or word input by voice in step T3 below is recognized and interpreted to be a query.
Step T3: A query is input by voice in short form, such as ‘Robot 1?’ or ‘Workpiece 3?’
Step T4: The voice input is recognized. Then it is determined which object (three-dimensional model) in the displayed work cell the query pertains to, and the result is displayed. When ‘Workpiece 2?’ is input by voice for the work cell shown in
Step T5: If there are more queries to be input by voice, the sequence returns to step T3 to accept the next voice-based query. Steps T3 to T5 are repeated until there are no more queries. When all queries have been input, the process is terminated.
The range of input queries and responses can be expanded through the use of terms (character strings) employed in the work cell definitions. If, for example, the character string ‘hand position’ has been prepared in correspondence to the hand Hi in
When preparation and work cell confirmation have been completed as described above, program editing is performed. The flowchart in
Step U1: As in step T1, a work cell corresponding to the program to be edited is displayed. In this step, the work cell shown in
Step U2: Voice recognition mode 2 is initiated. Voice recognition mode 2 is an interpretation mode described below in which a sentence, clause, or word input by voice in step U6 below is recognized and an operation instruction is given (details will be given below).
Step U3: Models for operation instructions are displayed as shown in
Step U4: The program to be edited is displayed. In the example in
Step U5: The portion of the program to be edited is selected, and the selection is indicated on the display. The operator uses the keyboard or another device to select a line and specify insertion of a new line above the selected line (pre-insertion), insertion of a new line below the selected line (post-insertion), or replacement of the selected line (overwriting). The selected line is highlighted. The distinction between pre-insertion, post-insertion, and overwriting can be indicated by, for example, the way in which the selected line is highlighted, e.g., by color coding, shading, underlining, or reverse video. In the example in
Step U6: An operation instruction is input by voice. Specifically, the operator calls up the model display screen, if necessary, and while viewing the screen, inputs by voice the operation instruction the operator wants to insert at the location specified in step U5, following the pattern given by the model. As an example, ‘Robot 1 picks up workpiece 1’ is input by voice here, following the pattern of the first model in the model list in
Steps U7, U8, and U9: The program editing apparatus executes a process for parsing the operation instruction by using the voice recognition function. Specifically, the program editing apparatus generates a series of statements completed by assigning candidate character strings to the variable parts of the models (these statements are referred to as completed statements), and compares the completed statements with the voice-recognized statement to see if they match. When a matching statement is found, the program editing apparatus displays a message on the screen and proceeds to step U10. If a statement matching the voice-recognized statement is not found even after all the candidate character strings have been assigned to all variable parts of all models (in this case, step U8 produces a NO result), the program editing apparatus displays a prompt for reinput (or informs the operator by an artificial voice alert through the headphone) in step U9, and the sequence returns to step U6.
If the program cannot proceed to step U10 even after executing steps U6 to U9 repeatedly, there may be an error (such as a missing candidate character string). When this happens, the process is terminated forcibly to find the cause of the error.
Step U10: The robot operation command (see preparatory step S3 described above) belonging to the model in the complete statement found in step U9 and stored in correspondence to the object is read and displayed. In the above example, the first model in the model list in
Step U11: The operation is displayed on the screen in animation form, according to the data in the displayed robot operation command. For example, the robot 1 (RB1) displayed on the screen in
Step U12: Whether it is safe to adopt the operation command and incorporate it into the program at the location selected in step U5 is determined from, for example, the viewpoint of the risk of interference with peripheral devices. If there is no problem, the sequence proceeds to step U13. If the operation command is not adopted, the sequence is repeated from step U4.
Step U13: The operation command is incorporated into the program at the location selected in step U5. For example, the statement corresponding to the robot operation command adopted above may be inserted between the third line and fourth line in the program shown in
Step U14: If further editing is required, the sequence returns to step U4 and the desired editing is carried out by executing step U4 and the steps that follow. If there is no need to continue the editing, the process is terminated.
So far, the commands defined in correspondence to objects have been limited to operation commands for robots, but macro commands including input/output commands to and from peripheral devices, conditional statements, and the like may also be used. An example will be briefly described below. The following case will be considered.
(1) In step S3 in the above embodiment, the following subroutine is predefined in correspondence to workpiece 1 and machine tool 1. The meaning of each statement is parenthesized.
First, the following subroutine is predefined in correspondence to workpiece 1:
PROG WorkPick (Subroutine name)
JP1 100% CNT, 100 (Move to approach point)
LP2 500M FINE (Move to pickup point)
SDO [1] ON (Grip workpiece; close hand.)
IFRDI[1]=OFF, GOTOLBL[999] (Errorifworkpiece cannot
be gripped; halt system.)
LP3 500M CONT, 100 (Move to back-away point)
END
LBL[999]
END
Next, the following subroutine is predefined in
correspondence to machine tool 1:
PROG MachineToolInsert (Subroutine name)
WAIT SDI[1]=ON (Wait until machine tool 1 has finished
machining previous workpiece and is ready to accept next
workpiece.)
JP1 100% CNT, 100 (Move to approach point)
LP2 500M FINE (Insert workpiece into workpiece port
1 of machine tool 1)
SDO [1] OFF (Release workpiece. Open hand.)
IF RD[1]=ON, GO TO LBL[999] (Error if workpiece
cannot be released; halt system.)
LP3 500M CONT, 100 (Move to back-away point)
END
LBL[999]
END
This completes the subroutines that have to be predefined in step S3 in the above embodiment, in correspondence to workpiece 1 and machine tool 1.
(2) Upon completion of the confirmation of the work cell in the above embodiment, step U1 and the following steps are executed sequentially. It will be assumed here, purely as an example, that the program displayed in step U4 is the same as before (that is, the program name is S5A—01A2) and that the editing location selected in step U5 is now pre-insertion at the third line. In the next step U6, the operation instruction ‘Robot 1 picks up workpiece 1 and places it on machine tool 1 ’ is input by voice.
Then, in step U7, the program editing apparatus uses the voice recognition function to execute the parsing process on the operation instruction, generates complete statements in succession by assigning candidate character strings to the variable parts of models, and searches for a matching statement. In this example, the matching statement is found from the fourth model in the model list in
When the sequence proceeds from step U9 to step U10, the subroutine (PROG WorkPick) stored for workpiece 1 and the subroutine (PROG MachineToolInsert) stored for machine tool 1 are read in succession and displayed. Then the operation corresponding to the displayed data is animated on the screen in step U11.
If the operation command is adopted in step U12, the above subroutines are inserted in the selected location. The relevant part of the program is shown below.
(Before editing)
(After editing)
In this way, editing can be performed by voice input even if the input commands include conditional statements and macro commands.
Watanabe, Atsushi, Nagatsuka, Yoshiharu
Patent | Priority | Assignee | Title |
10821539, | Nov 12 2013 | Lincoln Global, Inc. | System and method for pendant component for a welding system |
8483873, | Jul 20 2010 | Innvo Labs Limited | Autonomous robotic life form |
9069348, | Aug 11 2011 | Kabushiki Kaisha Yaskawa Denki | Portable remote controller and robotic system |
Patent | Priority | Assignee | Title |
6522950, | Sep 06 1999 | ABB S P A | Method for off-line programming an industrial robot |
6684127, | Feb 14 2000 | Sony Corporation | Method of controlling behaviors of pet robots |
6718231, | Sep 28 2000 | Sony Corporation | Authoring system and authoring method, and storage medium |
6853878, | Feb 10 2000 | Kabushiki Kaisha Yaskawa Denki | Robot controller |
7136723, | Feb 10 2000 | Kabushiki Kaisha Yaskawa Denki | Robot controller |
7181315, | Oct 08 2003 | Fanuc Ltd | Manual-mode operating system for robot |
7236854, | Jan 05 2004 | ABB Schweiz AG | Method and a system for programming an industrial robot |
20030018467, | |||
EP145683, | |||
EP1195231, | |||
JP20022337079, | |||
JP2002283259, | |||
JP60156111, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Sep 27 2004 | WATANABE, ATSUSHI | Fanuc Ltd | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 015974 | /0907 | |
Sep 27 2004 | NAGATSUKA, YOSHIHARU | Fanuc Ltd | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 015974 | /0907 | |
Nov 09 2004 | Fanuc Ltd | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Nov 02 2011 | ASPN: Payor Number Assigned. |
Mar 01 2012 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Mar 16 2016 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Mar 19 2020 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
Date | Maintenance Schedule |
Sep 30 2011 | 4 years fee payment window open |
Mar 30 2012 | 6 months grace period start (w surcharge) |
Sep 30 2012 | patent expiry (for year 4) |
Sep 30 2014 | 2 years to revive unintentionally abandoned end. (for year 4) |
Sep 30 2015 | 8 years fee payment window open |
Mar 30 2016 | 6 months grace period start (w surcharge) |
Sep 30 2016 | patent expiry (for year 8) |
Sep 30 2018 | 2 years to revive unintentionally abandoned end. (for year 8) |
Sep 30 2019 | 12 years fee payment window open |
Mar 30 2020 | 6 months grace period start (w surcharge) |
Sep 30 2020 | patent expiry (for year 12) |
Sep 30 2022 | 2 years to revive unintentionally abandoned end. (for year 12) |