medical and/or robotic devices, systems and methods can provide an indicator associated with each manipulator assembly of a multi-arm telerobotic or telesurgical system. The exemplary indicator comprises a multi-color light emitting diode (LED) mounted to a manipulator moving an associated surgical instrument, allowing the indicator to display any of a wide variety of signals. The invention may provide an additional user interface to facilitate communications between the telesurgical system and/or members of a telesurgical team.

Patent
   10258416
Priority
Jun 30 2005
Filed
Feb 01 2016
Issued
Apr 16 2019
Expiry
Sep 19 2026

TERM.DISCL.
Extension
83 days
Assg.orig
Entity
Large
48
96
currently ok
13. A medical system comprising:
a master controller configured to be operated by a user;
a manipulator adapted to hold and manipulate a surgical instrument, the manipulator having at least one activatable indicator on or proximate to the manipulator so as to viewable by an assistant positioned near the manipulator;
a display screen proximate to the manipulator so as to be viewable by the assistant positioned near the manipulator, but not viewable by the user while the user is operating the master controller; and
a computer processor programmed to control movement of the manipulator in response to manipulation of the master controller, control activation of the at least one activatable indicator so as to indicate that an action is to be taken by the assistant, and provide instructions on the display screen for the action to be taken when the at least one activatable indicator is activated.
1. A medical system comprising:
a first manipulator supporting a first tool, the first manipulator having a first indicator on an outer surface of the first manipulator;
a first input device operatively coupled to the first manipulator;
a second manipulator supporting a second tool, the second manipulator having a second indicator on an outer surface of the second manipulator;
a second input device operatively coupled to the second manipulator; and
a processor programmed to command a selected one of the first and second indicators to output a first identifiable signal;
wherein the selected one of the first and second indicators indicates one of: a selected manipulator of the first and second manipulators, a selected tool of the first and second tools, and a selected input device of the first and second input devices; and
wherein the first identifiable signal indicates one of: a state or an identity of the selected manipulator, a state or an identity of the selected tool, and a state or an identity of the selected input device.
2. The medical system of claim 1, wherein the selected one of the first and second indicators is selected by a system operator.
3. The medical system of claim 1, wherein the selected one of the first and second indicators is selected by the processor.
4. The medical system of claim 1, wherein the selected one of the first and second indictors comprises a visual indicator, and wherein the first identifiable signal comprises at least one of a visual pattern and a color.
5. The medical system of claim 4, wherein the visual pattern comprises a blinking pattern.
6. The medical system of claim 4, wherein the first and second visual indicators each comprise a light emitting diode.
7. The medical system of claim 4, wherein the first indicator comprises a plurality of light emitting diodes, wherein the first identifiable signal comprises the color, and wherein each of the plurality of light emitting diodes is a different color so that the color of the first identifiable signal is generated by energizing a combination of the plurality of light emitting diodes.
8. The medical system of claim 1, wherein the selected tool comprises an electrosurgical instrument, and wherein the first identifiable signal indicates an energized state of the selected tool.
9. The medical system of claim 1, wherein the first identifiable signal indicates one of: a low battery state, a power failure state, and a system fault state.
10. The medical system of claim 1, wherein the first identifiable signal comprises a yellow color that communicates a warning regarding operation of the selected tool.
11. The medical system of claim 1, wherein the first identifiable signal comprises a red color that communicates a system fault rendering the selected manipulator inoperative.
12. The medical system of claim 1, further comprising:
a display;
the processor being programmed to cause a graphical indication that corresponds to the first identifiable signal to be displayed on the display.
14. The medical system of claim 13, further comprising:
a monitor screen positioned proximate to the master controller so as to be viewable by the user, but not viewable by the assistant near the manipulator;
wherein the computer processor is programmed to display information associated with the activation of the at least one activatable indicator on the monitor screen.
15. The medical system of claim 13, wherein the activation of the at least one activatable indicator is initiated by an occurrence of a system condition.
16. The medical system of claim 13, wherein the activation of the at least one activatable indicator is initiated by the user operating the master controller.
17. The medical system of claim 13, wherein the at least one activatable indicator comprises a plurality of individually activatable light emitting diodes.
18. The medical system of claim 13, wherein at least one activatable indicator comprises a plurality of different colored elements.

This application is a continuation of U.S. patent application Ser. No. 13/214,439 (filed Aug. 22, 2011), now U.S. Pat. No. 9,259,276 B2, which is a divisional of U.S. patent application Ser. No. 11/478,416 (filed Jun. 26, 2006), now U.S. Pat. No. 8,100,133 B2, which claims the benefit of U.S. Provisional Patent Application No. 60/695,611 (filed Jun. 30, 2005), each of which is incorporated herein by reference.

NOT APPLICABLE

NOT APPLICABLE

The present invention is generally related to medical and/or robotic devices, systems, and methods. In an exemplary embodiment, the invention provides an indicator associated with one or more robotic manipulator assemblies for communication of a state of the manipulator assembly or other component of the robotic system, for identification of one or more particular manipulators, or the like. The indicator(s) can provide an additional user interface between the robotic system and, for example, a surgical assistant, system operator, or the like.

Minimally invasive medical techniques are intended to reduce the amount of extraneous tissue which is damaged during diagnostic or surgical procedures, thereby reducing patient recovery time, discomfort, and deleterious side effects. One effect of minimally invasive surgery, for example, may be reduced post-operative hospital recovery times. Because the average hospital stay for a standard surgery is typically significantly longer than the average stay for an analogous minimally invasive surgery, increased use of minimally invasive techniques could save millions of dollars in hospital costs each year. While many of the surgeries performed each year in the United States could potentially be performed in a minimally invasive manner, only a portion of the current surgeries use these advantageous techniques due to limitations in minimally invasive surgical instruments and the additional surgical training involved in mastering them.

Minimally invasive robotic surgical or telesurgical systems have been developed to increase a surgeon's dexterity and avoid some of the limitations on traditional minimally invasive techniques. In telesurgery, the surgeon uses some form of remote control, e.g., a servomechanism or the like, to manipulate surgical instrument movements, rather than directly holding and moving the instruments by hand. In telesurgery systems, the surgeon can be provided with an image of the surgical site at the surgical workstation. While viewing a two or three dimensional image of the surgical site on a display, the surgeon performs the surgical procedures on the patient by manipulating master control devices, which in turn control motion of the servomechanically operated instruments.

The servomechanism used for telesurgery will often accept input from two master controllers (one for each of the surgeon's hands) and may include two or more robotic arms or manipulators on each of which a surgical instrument is mounted. Operative communication between master controllers and associated robotic arm and instrument assemblies is typically achieved through a control system. The control system typically includes at least one processor which relays input commands from the master controllers to the associated robotic arm and instrument assemblies and back from the instrument and arm assemblies to the associated master controllers in the case of, e.g., force feedback or the like. One example of a robotic surgical system is the DA VINCI® system available from Intuitive Surgical, Inc. of Mountain View, Calif.

The roles and interaction among the persons making up a telesurgical team may differ from those of conventional surgery. As telesurgery is often performed in an internal surgical site, at least some of those working under the direction of the lead surgeon (or other robotic system operator) may not have direct access to or direct visualization of the ongoing tissue manipulations. For example, surgical assistants within the operating room may remove a first surgical instrument (such as an electrosurgical scalpel) from a manipulator and replace it with a different surgical instrument (such as a needle holder), as a surgeon may desire the use of different surgical instruments during different phases of a surgical procedure. Similarly, the assistant may reposition a surgical instrument during a procedure, so that rather than approaching the procedure from a first minimally invasive access site, the instrument is advanced toward the internal surgical site from a second, different access site. More complex robotic systems (and team interactions) may also be used. For example, as surgery is often performed with more than two tools, input devices may be provided for one or more additional surgeons, with each additional surgeon robotically controlling (for example) at least one surgical instrument.

While the new telesurgical systems, devices and methods have proven highly effective and advantageous, still further improvements would be desirable. In general, it would be desirable to provide improved robotic and/or surgical devices, systems and methods, particularly for performing telesurgical procedures. It may also be desirable to provide improved techniques for communication among the members of a telesurgical team, and for interfacing with the telesurgical apparatus so as to more fully take advantage of the capabilities of telesurgery to provide enhanced patient outcomes with improved efficiencies. It may be particularly beneficial to avoid unnecessary interruptions and distractions of a surgeon or other system operator, and to avoid delays and/or mistakes in the coordinated activities of a telesurgical team.

The present invention is generally related to medical and/or robotic devices, systems and methods. An exemplary embodiment of the present invention provides a visual indicator associated with each manipulator assembly (and, hence, each surgical instrument) of a multi-arm telesurgical system. The exemplary indicator comprises a multi-color light emitting diode (LED), allowing the indicator to display any of a wide variety of signals through combinations of color, blinking, and the like. By establishing and using an appropriate indicator signal lexicography, the indicator can identify, for example:

In a first aspect, the invention provides a robotic method comprising moving a first end effector of a first manipulator assembly in response to a first robotic command. The first manipulator assembly has a first indicator. A second end effector of a second manipulator assembly is moved in response to a second robotic command. The second manipulator assembly has a second indicator. A command is transmitted from a processor, with the command being selectably directed to the first indicator and/or the second indicator.

In many embodiments, the end effectors will include the working ends of surgical instruments. The person receiving the identifiable signal will often be a member of a robotic surgical team within an operating room, with the processor acting as a master/slave telesurgical controller. In such embodiments, a movement command may be input by a system operator, and the system operator may also select which of the manipulator assemblies is to output the first signal. This can allow the system operator to maintain his or her concentration on movements of the end effectors shown in a display while communicating to, for example, an assistant regarding a specific surgical instrument. For example, rather than telling the assistant to “replace the electrosurgical scalpel on the arm now wiggling back and forth with a needle driver,” the surgeon can instead merely request a needle driver and activate the indicator of the appropriate manipulator assembly. Optionally, the processor may select the appropriate indicator and manipulator assembly, such as when the tool mounted on a manipulator reaches or exceeds its intended useful life.

The indicator will often include a visual indicator, with the first signal including a visual pattern such as blinking or the like, an identifiable color, or some other identifiable visual signal. In an exemplary telesurgical system, the indicators each include a plurality of light emitting diodes (LED). A variety of alternative indicators might also be employed, including any of the wide variety of light sources capable of generating different colors, audible indicators for generating tones or verbal signals, and the like. Preferably, a plurality of separately identifiable signals can be output from each indicator, thereby providing a user interface for communication to the assistant or the like regarding an associated manipulator assembly.

Typically, each manipulator assembly will include a manipulator movably supporting a tool holder, and a tool releasably mounted to the tool holder. The tool will generally include the end effector. The first identifiable signal may indicate, for example, a state of the tool, which specific manipulator assembly is operatively associated with a specific input device, an overall condition or state of all of the manipulator assemblies, an identity of one or more components of the selected manipulator assembly, or the like. Some listing or lexicography explaining the meaning of each identifiable signal will often be provided or known by the person receiving the signal, and that person will often take action in response to the signal, such as reconfiguring the manipulator assembly from which the signal is being output. The reconfiguration may include replacement of a tool of the manipulator assembly with another tool having a different (or the same) end effector, changing a mode of the manipulator assembly so as to initiate or inhibit master/slave movement of the end effector, or the like.

It will often be beneficial to include readily identifiable signal types or groups. For example, when the first signal includes a yellow color, it may communicate a warning that the associated manipulator assembly remains functional, but needs attention now or in the near future. A signal including a red color may communicate a sufficient fault along the robotic system chain to interrupt end effector movement.

A variety of refinements may be implemented with use of the manipulator assembly indicator. For example, the system operator who is inputting movement commands may benefit from graphical indicia in his or her display that corresponds to the first identifiable signal. Hence, when the system indicates that a tool is near the end of its life or the like by flashing the indicator, the same information may be communicated to the system operator by simple flashing of an icon on the operator's display screen, by superimposing a flashing color on the end effector of the instrument as shown in the display, or the like. In some embodiments, along with indicating which currently mounted tool or end effector is to be replaced, the processor may also energize another indicator associated with the new tool that is now desired, such as by flashing an LED at an appropriate location on a tool tray, or the like.

In another aspect, the invention provides a surgical robotic method comprising inputting a robotic movement command from a system operator to an input device. A surgical end effector of a manipulator assembly moves in response to the robotic command. The system operator views the movements of the end effector in a display while inputting the command. An identifiable visual signal is transmitted from a visual indicator of the manipulator assembly to a person nearby. In response, the person manually reconfigures the manipulator assembly.

In yet another aspect, the invention provides a surgical robotic system comprising a first manipulator assembly having a first indicator and a first end effector. The first end effector is movable in response to a first movement command. The second manipulator assembly has a second indicator and a second end effector. The second end effector is movable in response to a second movement command A processor is coupled to the first and second indicators, and selectively induces at least one of the indicators to output a first signal identifiable by a person nearby.

FIG. 1 is a schematic plan view of a portion of an operating theater illustrating a robotic surgical system in use, including a master surgeon console or workstation for inputting a surgical procedure and a robotic manipulator system for robotically moving surgical instruments having surgical end effectors at a surgical site within a patient.

FIG. 2 is a perspective view of an exemplary manipulator system, including positioning linkages or set up joints which allow two patient side robotic manipulators and one endoscope or camera robotic manipulator to be pre-configured for surgery.

FIG. 3 is a perspective view of an exemplary articulated surgical instrument for use in the system of FIG. 1.

FIG. 4 is a perspective from above of an alternative manipulator system including a plurality of positioning linkages, each supporting a manipulator assembly.

FIG. 5 is a perspective view of an exemplary manipulator for use in the manipulator system of FIG. 4.

FIG. 6 is a detailed view of a portion of the manipulator of FIG. 5, showing a tool holder and adjacent indicator for outputting visual signals.

FIG. 7 shows the indicator of FIG. 6 in isolation.

FIG. 8 is a schematic block diagram of an embodiment of an alternative robotic system and method for its use by a plurality of system operators.

FIG. 9 is a flow chart schematically describing a method for reconfiguring a manipulator assembly in response to an identifiable signal from an associated indicator.

FIG. 10 is a listing of separately identifiable visual signals which may be then generated by the indicator of FIG. 6, along with their associated meanings.

FIG. 11 illustrates colors which can be generated by the indicator of FIG. 6, along with the general meanings of those colors.

FIG. 12 is a listing of indicator signal colors and associated general meanings.

FIG. 13 is a listing of indicator signals and associated specific meanings.

FIG. 14 illustrates An exemplary quick reference guide to the meanings of indicator signals for a telesurgical system.

The present invention generally provides improved robotic and/or medical devices, systems and methods. Embodiments of the present invention are particularly well suited for telesurgery, often providing an additional form of user interface that can enhance communication between a surgical system operator, an assistant, any other members of a telesurgical team and/or the telesurgical system. Other embodiments of the invention may be well suited for use in other telerobotic or robotic environments, particularly with robotic systems having a plurality of manipulators.

FIGS. 1-3 illustrate components of a robotic surgical system 1 for performing minimally invasive robotic surgery. System 1 is similar to that described in more detail in U.S. Pat. No. 6,246,200, the full disclose of which is incorporated herein by reference. A system operator O (generally a surgeon) performs a minimally invasive surgical procedure on a patient P lying on an operating table T. The system operator O manipulates one or more input devices or masters 2 at a surgeon's console 3. In response to the surgeon's input commands, a computer processor 4 of console 3 directs movement of endoscopic surgical instruments or tools 5, effecting servomechanical movement of the instruments via a robotic patient-side manipulator system 6 (a cart-mounted system in this example).

Typically, manipulator system 6 includes at least 3 robotic manipulator assemblies. Two linkages 7 (mounted at the sides of the cart in this example) support and position manipulators 8 with linkages 7 in generally supporting a base of the manipulators 8 at a fixed location during at least a portion of the surgical procedure. Manipulators 8 move surgical tools 5 for robotic manipulation of tissues. One additional linkage 9 (mounted at the center of the cart in this example) supports and positions manipulator 10 which controls the motion of an endoscope/camera probe 11 to capture an image (preferably stereoscopic) of the internal surgical site. The fixable portion of positioning linkages 7, 9 of the patient-side system is sometimes referred to herein as a “set-up arm.”

The image of the internal surgical site is shown to operator O by a stereoscopic display 12 in surgeon's console 3. The internal surgical site is simultaneously shown to assistant A by an assistance display 14. Assistant A can help prior to and during a surgical procedure. Prior to surgery, assistant A typically covers at least a portion of the system with sterile drapes. Such draping of the system may include attaching sterile adapters, cannulas, and/or the like to manipulators 8, switching one or more of the manipulators into and out of a manually articulatable clutch mode, and the like. Assistant A also assists in prepositioning manipulator assemblies 8 and 10 relative to patient P using set-up linkage arms 7, 9; in swapping tools 5 from one or more of the surgical manipulators for alternative surgical tools or instruments 5′; in operating related non-robotic medical instruments and equipment; in manually moving a manipulator assembly so that the associated tool accesses the internal surgical site through a different aperture, and the like. Some or all of these activities may be facilitated by providing feedback to assistant A via the indicators described herein, with the indicators often providing feedback to the assistant regarding a state of the manipulator assembly, a state of the tool mounted thereon, and/or a state of the robotic surgical system.

In general terms, the linkages 7, 9 are used primarily during set-up of patient-side system 6, and typically remain in a fixed configuration during at least a portion of a surgical procedure. Manipulators 8, 10 each comprise a driven linkage which is actively articulated under the direction of surgeon's console 3. Although one or more of the joints of the set-up arm may optionally be driven and robotically controlled, at least some of the set-up arm joints may be configured for manual positioning by assistant A. Each associated combination of a set-up joint, manipulator and tool is encompassed within the term “manipulator assembly” as that term is used herein, although some manipulator assemblies may not include set-up joints, and the manipulator assembly will not include a tool at all times.

For convenience, a manipulator such as manipulator 8 that is supporting a surgical tool used to manipulate tissues is sometimes referred to as a patient-side manipulator (PSM), while a manipulator 10 which controls an image capture or data acquisition device such as endoscope 11 may be referred to as an endoscope-camera manipulator (ECM). The manipulators may optionally actuate, maneuver and control a wide variety of instruments, tools, image capture devices and the like which are useful for surgery.

As can be seen in FIGS. 1 and 2, indicators 20 are disposed on each manipulator assembly. In the exemplary embodiments, indicators 20 are disposed on manipulators 8, 10 near the interface between the manipulators and their mounted tools 5. In alternative embodiments, indicators 20 and may instead be disposed on set-up joints 7, 9, on tools 5, elsewhere on manipulators 8, 10, or the like, with the indicators preferably being sufficiently close to the tools so that a signal generated by a particular indicator can be readily associated with a particular tool when the signal is viewed by assistant A. In FIG. 1, additional indicators 20 are shown in a rack or other structure supporting additional tools 5′, with each indicator again being associated with a particular tool (or type of tool).

As can be seen in FIG. 1, the system operator O is largely immersed in the environment of and interaction with workstation 3. The system operator sees images presented by display 12 and manipulates input devices 2, and in the exemplary embodiment, processor 4 correlates the movement of the end effectors of tools 5 so that the images of the end effectors follow the movements of the input devices in the hands of the system operator O.

It can be advantageous to avoid distracting system operator O during a surgical procedure so as to require the operator to shift his or her attention away from the internal surgical site. Even when assistant A is replacing a tool 5 with an alternative tool 5′, the system operator may continue manipulating tissues with another tool, or may want to continue to view the surgical site to monitor bleeding, and/or the like. Nonetheless, it may be difficult for the system operator to communicate clearly with assistant A regarding which tool 5 is to be replaced by an alternate tool. Specifically, as endoscope 11 may be at any convenient orientation, the tool associated with the right hand of system operator O will often not be disposed to the right of the endoscope, particularly as viewed by the assistant.

So as to unambiguously identify a tool 5 to be replaced assistant A, system operator O may input a command into workstation 3 (such as by pushing a button on the input device 2, actuating foot peddle, inputting a voice command, or the like) so that indicator 20 on the manipulator assembly associated with the specific tool 5 generates a visually identifiable signal that can be viewed by the assistant. A corresponding graphical indication of the signal may be provided to system operator O in display 12, such as presentation of an appropriate icon, superimposing of text on the surgical site, the use of image processing to superimpose false colors on the image of the appropriate end effector, or the like. In response to the signal on indicator 20, assistant A can remove and replace the identified tool 5. Optionally, each alternative tool 5′ may also have an associated indicator 20, allowing the processor to transmit a signal so as to indicate which alternative tool the system operator O is to be mounted on the robotic system.

There are a number of additional uses for indicators 20 in telesurgical system 1. For example, assistant A will often manually position tools 5 and endoscope 11 when setting up for a surgical procedure, when reconfiguring the manipulator system 6 for a different phase of a surgical procedure, when removing and replacing a tool with an alternate tool 5′, and the like. During such manual reconfiguring of the manipulator assembly by assistant A, the manipulator assembly may be placed in a different mode than is used during master/slave telesurgery, with the manually repositionable mode as sometimes being referred to as a clutch mode. The manipulator assembly may change between the tissue manipulation mode and the clutch mode in response to an input such as pushing a button on manipulator 8 (or some other component to the manipulator assembly) thereby allowing assistant A to change the manipulator mode. By generating appropriate visually identifiable signals with indicators 20 whenever the manipulator is in clutch mode, assistant A can avoid errors and increase the efficiency of the surgical procedure.

Indicators 20 may also show an associated identifiable signal when, for example, no tool is attached to the manipulator, when a sterile adaptor or interface between tool 5 and manipulator 8 is not properly attached, if an instrument mounted onto a manipulator is incompatible with that manipulator, if the instrument has reached the end of its useful life, and/or the like. For some of these signals, system operator O may initiate generation of the signal and may specify the associated manipulator assembly from which the signal is to be produced. In others, processor 4 may initiate the signal and/or indicate which manipulator assembly(s) are to be indicated. For example, in the case of a power failure, robotic system 1 may continue to operate using backup batteries. So as to indicate to the assistant A that a power failure has occurred, indicators 20 on all manipulators may blink, optionally blinking with a yellow light as a warning. As the power in the battery is drained so that robotic system 1 can no longer continue to provide robotic movement of the tools, all indicators 20 may blink red, thereby indicating a system fault. A wide variety of alternative signals may also be provided, some of which are indicated by the exemplary lexicographies of FIGS. 10-14.

FIG. 4 illustrates a perspective view of the articulated surgical tool or instrument 5. Tool 5 has a proximal housing 24 which interfaces with a tool holder of the manipulator, generally providing a quick release mounting engagement through a sterile adapter or interface. Tool 5 includes an elongate shaft 23 supporting an end effector 28 relative to proximal housing 24. Proximal housing 24 includes a drive mechanism that accepts and transmits drive signals or drive motion between the manipulator 8 and the end effector 28. Proximal housing 24 also contains circuitry that can generate signals for transmission to processor 4 so as to identify a type of the tool, indicate a remaining useful life of the tool, and/or the like. The exemplary circuitry may include a memory, such as a Dallas™ part or the like, and exemplary structures and methods for transmitting of information between the memory of tool 5 and processor 4 may be described in application Ser. No. 10/839,727, filed on May 4, 2004 and entitled “Tool Memory-Based Software Upgrades for Robotic Surgery” the full disclosure of which is incorporated herein by reference. An articulated wrist 29 may provide two degrees of freedom of motion between end effector 28 and shaft 23, and the shaft may be rotateable relative to proximal housing 24 about the axis of the shaft so as to provide the end effector 28 with the three orientational degrees of freedom within the patient's body. Control over actuation of end effector 28, such as opening or closing of jaws or the like, may represent an additional degree of freedom, for a total of four distal degrees of freedom. In some embodiments, an indicator 20 may be mounted on housing 24.

Referring now to FIG. 4, a perspective view from above of an alternative modular manipulator support assembly 30 may be mounted to a ceiling of an operating room. The modular manipulator support 30 aligns and supports a robotic manipulator system relative to a set of desired surgical incision sites in a patient's body. Modular manipulator support 30 generally includes an orientating platform 36 and a plurality of configurable set-up linkage arms 38, 40, 42, 44 coupleable to the orienting platform. Each arm movably supports an associated manipulator 32, 34, which in turn movably supports an associated tool or an image capture device. Orienting platform 36 also supports an assistant display 104, which may be used for set-up, instrument changes, viewing of the procedure and the like. The structures and use of any of the components of modular manipulator support assembly 30 are analogous to those described above regarding manipulator system 6, and are more fully and described in copending U.S. patent application Ser. No. 11/043,688, filed on Jan. 24, 2005, and entitled “Modulator Manipulator Support For Robotic Surgery”, the full disclosure of which is incorporated herein by reference. As generally described above, each manipulator 32, 34 of modular manipulator support 30 may include an indicator 20.

Manipulator 32 is shown in more detail in FIG. 5, which also shows indicator 20 near a tool holder 52. The exemplary indicator 20 is shown in more detail in FIG. 6, and in isolation (with some of the internal components being schematically shown by dashed lines) in FIG. 7. The exemplary indicator 20 generally comprises a clear and/or translucent polymer body 54 in which a pair of light emitting diode arrays (LED) 56 are mounted. LED 56 include a red LED, a blue LED and a green LED. The LEDs of each array can be independently energized to any of a plurality of illumination or brightness levels, allowing indicator 20 to generate signals in a wide variety of colors, as schematically illustrated in FIG. 11. Additionally, the overall level of illumination from indicator 20 may be varied, and a pattern may be imposed on the cover signal by blinking and the illumination on and off, alternating between two different colors, alternating between two different illumination levels, simultaneously displaying two different colors, or the like. The speed of blinking may also be controlled or altered in more complex modulation patterns (with long and short blinks, Morse code, and the like).

As different colors may be combined with different illumination levels and different modulation patterns in a signal, a very large number of independently identifiable signals can be generated, often being more than three separately identifiable signals, typically being more than 10 separately identifiable signals, and optionally of being more than 100 separately identifiable signals. Interpretation of the identifiable signals may be facilitated by a listing correlating signals and their meaning, such as that included as FIG. 10. In some embodiments, a relatively simple signal scheme with a more limited number of signals output by indicator 20 may be combined with supplemental text or graphics output from the assistant display 114 or the like. Hence, the assistant may know that they should look for additional information on the assistant display in response to one or more signals.

So as to facilitate interpretation of the signals generated by indicators 20, any of a wide variety of signal conventions may be established and/or employed. For example, as can be understood with reference to FIG. 11, signals including the color yellow may generally indicate a warning. Optionally, the manipulator assembly may continue to function while its indicator 20 displays such a yellow warning signal, but the assistant may understand that some action may be advisable. The specific action or meaning of the warning signal may depend on other aspects of the signal, such as other colors being interspersed with a blinking yellow illumination pattern, speed of blinking, or the like. Signals including red may indicate a current or imminent fault that is now or may soon interfere with operation of the manipulator assembly. Green may optionally indicate normal operation is underway, blue may indicate an instrument or a tool is engaged, and/or that a guided tool change is underway or has been requested. Purple may indicate that a second surgeon or system operator is actively controlling the manipulator assembly, or the like. Slow blinking lights may generally indicate a less urgent state than a rapidly blinking indicator 20. Some of these possible conventions are graphically illustrated in FIG. 11.

Referring now to FIG. 8, a schematic block diagram of a somewhat more complex system 60 is provided. Exemplary cooperative telesurgical systems which may be modified to take advantage of the additional user interface provided by indicators 20 is described in more detail in U.S. Pat. No. 6,659,939, the full disclosure of which is incorporated herein by reference. In the exemplary system of FIG. 8, a first system operator O1 provides input to a first input device 2. Processor 4 selectively couples the movement commands from the first input device 2 to any of a plurality of manipulator assemblies, for example, to manipulator A and manipulator D. A second system operator O2 inputs movement commands to a second input device 2 so as to effect movement of a manipulator 32 manipulator assembly B.

When the second system operator O2 desires the tool 5 attached to manipulator assembly B be replaced an alternate tool 5′, for example, tool F, the system operator may verbally instruct assistant A to mount tool F to his manipulator assembly. The indicator 20 of manipulator assembly B generates a signal in response to an appropriate input by the second system operator O2, clearly indicating to the assistant A which tool from which manipulator assembly is to be replaced.

As noted above, additional indicators may optionally be associated with each of the alternative tools 5′, thereby foregoing any need for verbal instruction to the assistant regarding which alternative tool is to be mounted to the manipulator. Toward that end, alternative tools 5′ may be included in a tool rack 62 which is coupled to processor 4, often so as to provide communication between the circuitry of alternative tools 5′ and the processor. This can allow the processor to read identifying information from the tools, thereby allowing the processor to determine which alternative tools or tool types are available and/or appropriate to mount on the manipulator system. An LED or other indicator may be associated with each alternative tool 5′ included in the tool rack, and processor 4 may transmit a signal to the tool rack so as to energize one or more indicator associated with one or more tools. Hence, when system operator O1 indicates to processor 4 a desire to replace a mounted tool 5 (for example, tool B on manipulator B) with a different type of tool, a list of available tools and/or tool types may be displayed to the system operator. In response to the system operator selecting (for example, a tool type corresponding to tool F, indicators 20 associated with tool B and tool F may be activated, indicating to the assistant A that these two tools should be exchanged.

Processor 4 will typically include data processing hardware and software, with the software typically comprising machine-readable code. The machine-readable code will embody software programming instructions to implement some or all of the methods described herein. While processor 4 is shown as a single block in the simplified schematic of FIG. 8, the processor may comprise a number of data processing circuits, with at least a portion of the processing optionally being performed adjacent input device 1, a portion being performed adjacent manipulator B, and the like. Any of a wide variety of centralized or distributed data processing architectures may be employed. Similarly, the programming code may be implemented as a number of separate programs or subroutines, or may be integrated into a number of other aspects of the robotic systems described herein.

Referring now to FIG. 9, a flowchart 70 schematically illustrates a method for implementing an embodiment of the present invention. The method 70 may begin with a change 72 in state of the manipulator, tool, system, or the like. For example, an assistant may actuate a clutch mode button on the manipulator, the tool may reach the end of its useful life, a manipulator fault may be detected, or the like. In other embodiments, the system operator may initiate method 70 by requesting a new tool 74, or in some other manner indicating which of one or more manipulator assemblies is to generate a signal to be perceived by an assistant or some other person.

Regardless of whether the signal is initiated by a change in state or a system operator, the processor will typically transmit an appropriate command to one or more of the manipulator assemblies 76, and, in response, the indicator on that manipulator assembly will display a visual signal 78. Optionally, corresponding graphics may be shown in a system operator display 80, thereby allowing the system operator to maintain his or her concentration on the internal surgical site.

In response to the visual signal, the other person, such as an assistant, may optionally reconfigure the manipulator assembly 82. For example, the assistant may remove and replace a tool (with its end effector) or may manually reposition the manipulator linkage so as to move the end effector into a desired position. In some embodiments, the assistant may merely monitor the manipulator assembly in response to one or more visual signals, and may optionally prepare to take some appropriate action if the visual signal changes so as to indicate that such an action is appropriate.

FIGS. 10-14 indicate meanings of different signals which may be generated by the exemplary multicolor LED indicators described herein. FIG. 10 shows a potential lexicography for using the indicators as a manipulator user interface (“UI”), while FIG. 11 illustrates some general meanings of signal colors from the indicators. FIG. 12 shows alternative general meanings of the two three-color LED indicators 20 included in each manipulator assembly. By transmitting these or other color signals with sufficient brightness through a clear or translucent plastic cap at a position on the manipulator assembly along the tool insertion axis away from the patient, the signals may be readily identified throughout a wide area. As the red, green, and blue elements of each LED array have a brightness that can be independently controlled, and as the two LED arrays of an indicator may be independently controlled, a wide variety of identifiable signal colors may be generated.

FIG. 13 specifically identifies system, manipulator assembly, and/or tool states that may be associated with several specific signals from the manipulator assembly indicators. Synchronous blinking encompasses blinking of both LED arrays within a single indicator of a manipulator assembly, as can be understood with reference to FIGS. 6 and 7. Alternate blinking encompasses energizing of the two LED arrays with out-of-phase blinking, so that when one LED array is on, the other is off. Still further additional signals may be generated by the indicators to communicate alternative information. For example, signals (such as a quick green blink) may indicate that a component (such as a cannula, sterile adapter, tool or surgical instrument, or the like) that has recently or just been mounted to the manipulator system is valid and accepted. Alternative signals may indicate that a tool change is requested by the system operator, that an energizable instrument (such as an electrocautery instrument or the like) is being activated by the system operator, and the like. Different signal characteristics may optionally be associated with recoverable faults and unrecoverable faults. Hence, a wide variety of still further alternative signals may be included.

While described in some detail, for clarity of understanding and by way of example, a variety of changes, adaptations, and modifications will be obvious to those of skill in the art. Hence, the scope of the present invention is limited solely by the appended claims.

Larkin, David Q., Mintz, David S., Walker, Theodore C., Hanuschik, Michael L.

Patent Priority Assignee Title
11202683, Feb 22 2019 AURIS HEALTH, INC Surgical platform with motorized arms for adjustable arm supports
11304764, Jun 30 2005 Intuitive Surgical Operations, Inc Indicator for tool state and communication in multi-arm robotic telesurgery
11337765, Jun 30 2005 Intuitive Surgical Operations, Inc Indicator for tool state and communication in multi-arm robotic telesurgery
11432890, Jan 04 2018 Covidien LP Systems and assemblies for mounting a surgical accessory to robotic surgical systems, and providing access therethrough
11432891, Jun 03 2015 Covidien LP Offset instrument drive unit
11446099, Jun 03 2016 Covidien LP Control arm for robotic surgical systems
11464593, Jun 03 2016 Covidien LP Passive axis system for robotic surgical systems
11484372, Feb 15 2019 Covidien LP Articulation mechanisms for surgical instruments such as for use in robotic surgical systems
11504197, Mar 31 2021 Moon Surgical SAS Co-manipulation surgical system having multiple operational modes for use with surgical instruments for performing laparoscopic surgery
11510747, May 25 2017 Covidien LP Robotic surgical systems and drapes for covering components of robotic surgical systems
11517183, Oct 23 2015 Covidien LP Surgical system for detecting gradual changes in perfusion
11523509, May 26 2016 Covidien LP Instrument drive units
11529203, Sep 25 2015 Covidien LP Robotic surgical assemblies and instrument drive connectors thereof
11547508, May 26 2016 Covidien LP Robotic surgical assemblies
11553974, May 25 2017 Covidien LP Systems and methods for detection of objects within a field of view of an image capture device
11553984, Jun 03 2016 Covidien LP Robotic surgical system with an embedded imager
11576562, Apr 07 2016 Covidien LP Camera positioning method and apparatus for capturing images during a medical procedure
11576733, Feb 06 2019 Covidien LP Robotic surgical assemblies including electrosurgical instruments having articulatable wrist assemblies
11576739, Jul 03 2018 Covidien LP Systems, methods, and computer-readable media for detecting image degradation during surgical procedures
11583358, Sep 06 2017 Covidien LP Boundary scaling of surgical robots
11586106, Dec 28 2018 Covidien LP Imaging apparatus having configurable stereoscopic perspective
11596489, Mar 10 2015 Covidien LP Measuring health of a connector member of a robotic surgical system
11612446, Jun 03 2016 Covidien LP Systems, methods, and computer-readable program products for controlling a robotically delivered manipulator
11615884, Mar 06 2018 DIGITAL SURGERY LIMITED Techniques for virtualized tool interaction
11618171, Dec 11 2013 Covidien LP Wrist and jaw assemblies for robotic surgical systems
11622824, Jun 16 2015 Covidien LP Robotic surgical system torque transduction sensing
11622826, Mar 31 2021 Moon Surgical SAS Co-manipulation surgical system for use with surgical instruments for performing laparoscopic surgery while compensating for external forces
11628022, Sep 05 2017 Covidien LP Collision handling algorithms for robotic surgical systems
11628024, Mar 08 2018 Covidien LP Surgical robotic systems
11633243, Oct 10 2018 Covidien LP Instrument insertion system, method, and apparatus for performing medical procedures
11647888, Apr 20 2018 Covidien LP Compensation for observer movement in robotic surgical systems having stereoscopic displays
11666395, Jun 23 2015 Covidien LP Robotic surgical assemblies
11690691, Feb 15 2017 MEDTRONIC GMBH, EARL-BAKKEN-PLATZ System and apparatus for crush prevention for medical robot applications
11717355, Jan 29 2019 Covidien LP Drive mechanisms for surgical instruments such as for use in robotic surgical systems
11717361, May 24 2017 Covidien LP Electrosurgical robotic system having tool presence detection
11723735, Jun 30 2005 Intuitive Surgical Operations, Inc. Indicator for tool state and communication in multi-arm robotic telesurgery
11737840, Mar 31 2021 Moon Surgical SAS Co-manipulation surgical system having a robot arm removeably attachable to surgical instruments for performing laparoscopic surgery
11779413, Nov 19 2015 Covidien LP Optical force sensor for robotic surgical system
11786323, Mar 31 2021 Moon Surgical SAS Self-calibrating co-manipulation surgical system for use with surgical instrument for performing laparoscopic surgery
11806096, Dec 04 2018 MAKO SURGICAL CORP Mounting system with sterile barrier assembly for use in coupling surgical components
11812938, Mar 31 2021 Moon Surgical SAS Co-manipulation surgical system having a coupling mechanism removeably attachable to surgical instruments
11819302, Mar 31 2021 Moon Surgical SAS Co-manipulation surgical system having user guided stage control
11832909, Mar 31 2021 Moon Surgical SAS Co-manipulation surgical system having actuatable setup joints
11832910, Jan 09 2023 Moon Surgical SAS Co-manipulation surgical system having adaptive gravity compensation
11839441, May 25 2017 Covidien LP Robotic surgical system with automated guidance
11839442, Jan 09 2023 Moon Surgical SAS Co-manipulation surgical system for use with surgical instruments for performing laparoscopic surgery while estimating hold force
11844583, Mar 31 2021 Moon Surgical SAS Co-manipulation surgical system having an instrument centering mode for automatic scope movements
D963851, Jul 10 2020 Covidien LP Port apparatus
Patent Priority Assignee Title
4281447, Mar 01 1979 McDonnell Douglas Corporation Detachable tool interface system for a robot
4332066, Jan 07 1980 Lockheed Martin Corporation Compliance mechanism
4486928, Jul 09 1981 Magnavox Electronic Systems Company Apparatus for tool storage and selection
4500065, Mar 01 1982 ABB ROBOTICS, INC , A NY CORP Releasable tool mount for manipulator
4512709, Jul 25 1983 ABB ROBOTICS, INC , A NY CORP Robot toolchanger system
4706372, Jun 11 1985 Brown & Sharpe Dea SpA Device for effecting automatic exchange of measuring tools in a measuring robot or machine
4710093, Jun 08 1984 Kuka Schweissanlagen & Roboter GmbH Device for the automatic gripping and releasing of a tool holder in a manipulator
4793053, Apr 16 1987 General Motors Corporation; GENERAL MOTORS CORPORATION, A CORP OF DE Quick disconnect device
4809747, Jul 31 1987 General Motors Corporation Quick disconnect device
4830569, Mar 31 1987 Asea Brown Boveri AB Industrial robot having a detachable electrical connection between housing on robot arm and tool holder
4832198, Jun 15 1987 Container for packaging and counting surgical sponges
4943939, Aug 29 1988 Surgical instrument accounting apparatus and method
4979949, Apr 26 1988 The Board of Regents of The University of Washington Robot-aided system for surgery
4996975, Jun 01 1989 Kabushiki Kaisha Toshiba Electronic endoscope apparatus capable of warning lifetime of electronic scope
5018266, Dec 07 1987 Megamation Incorporated Novel means for mounting a tool to a robot arm
5078140, May 08 1986 Imaging device - aided robotic stereotaxis system
5143453, Oct 04 1988 G.I.R. Temperature monitoring device containing at least one element of an alloy which memorizes its shape
5154717, Apr 26 1988 The Board of Regents of The University of Washington Robot-aided system for surgery
5174300, Apr 04 1991 Symbiosis Corporation Endoscopic surgical instruments having rotatable end effectors
5217003, Mar 18 1991 Wilk Patent Development Corporation Automated surgical system and apparatus
5221283, May 15 1992 General Electric Company Apparatus and method for stereotactic surgery
5236432, Apr 26 1988 Board of Regents of the University of Washington Robot-aided system for surgery
5255429, Apr 09 1991 MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD Component mounting apparatus
5257998, Sep 20 1989 Mitaka Kohki Co., Ltd.; Takaaki, Takizawa Medical three-dimensional locating apparatus
5271384, Sep 01 1989 Tyco Healthcare Group LP Powered surgical retractor
5294209, Jul 25 1991 Yamaha Hatsudoki Kabushiki Kaisha Tool attaching device
5305203, Feb 01 1988 XENON RESEARCH, INC Computer-aided surgery apparatus
5312212, Sep 28 1992 HOLDEN AMERICA INC ; 4077776 CANADA INC Axially compliant tool holder
5313935, Dec 31 1992 Symbiosis Corporation Apparatus for counting the number of times a surgical instrument has been used
5343385, Aug 17 1993 International Business Machines Corporation Interference-free insertion of a solid body into a cavity
5354314, Dec 23 1988 MEDICAL INSTRUMENTATION AND DIAGNOSTICS CORPORATION MIDCO Three-dimensional beam localization apparatus and microscope for stereotactic diagnoses or surgery mounted on robotic type arm
5355743, Dec 19 1991 Board of Regents, The University of Texas System Robot and robot actuator module therefor
5359993, Dec 31 1992 Symbiosis Corporation Apparatus for counting the number of times a medical instrument has been used
5372147, Jun 16 1992 TYCO HEALTHCARE GROUP AG; Covidien AG Peritoneal distension robotic arm
5397323, Oct 30 1992 International Business Machines Corporation Remote center-of-motion robot for surgery
5399951, May 12 1992 UNIVERSITE JOSEPH FOURIER Robot for guiding movements and control method thereof
5400267, Dec 08 1992 Hemostatix Corporation Local in-device memory feature for electrically powered medical equipment
5402801, Nov 02 1993 International Business Machines Corporation System and method for augmentation of surgery
5403319, Apr 26 1988 Board of Regents of the University of Washington Bone imobilization device
5417210, May 27 1992 INTERNATIONAL BUSINESS MACHINES CORPORATION A CORP OF NEW YORK System and method for augmentation of endoscopic surgery
5427097, Dec 10 1992 PACIFIC REPUBLIC CAPITAL CORP Apparatus for and method of carrying out stereotaxic radiosurgery and radiotherapy
5451368, Feb 25 1987 Ethicon, Inc Process and apparatus for dry sterilization of medical devices and materials
5649956, Jun 07 1995 SRI International System and method for releasably holding a surgical instrument
5697939, Aug 20 1992 Olympus Optical Co., Ltd. Apparatus for holding a medical instrument in place
5762458, Feb 20 1996 Intuitive Surgical Operations, Inc Method and apparatus for performing minimally invasive cardiac procedures
5784542, Sep 07 1995 ALBERTO RICARDO CUETO, TRUSTEE OF THE GUILLERMO RODRIGUEZ REVOCABLE TRUST - 2005 Decoupled six degree-of-freedom teleoperated robot system
5792135, May 16 1997 Intuitive Surgical Operations, Inc Articulated surgical instrument for performing minimally invasive surgery with enhanced dexterity and sensitivity
5800423, May 14 1993 SRI International Remote center positioner with channel shaped linkage element
6071281, May 05 1998 Boston Scientific Scimed, Inc Surgical method and apparatus for positioning a diagnostic or therapeutic element within the body and remote power control unit for use with same
6132368, Dec 12 1996 Intuitive Surgical Operations, Inc Multi-component telepresence system and method
6223100, Jan 21 1992 SRI, International Apparatus and method for performing computer enhanced surgery with articulated instrument
6246200, Aug 04 1998 Intuitive Surgical Operations, Inc Manipulator positioning linkage for robotic surgery
6331181, Dec 08 1998 Intuitive Surgical Operations, Inc Surgical robotic tools, data architecture, and use
6522906, Dec 08 1998 Intuitive Surgical Operations, Inc Devices and methods for presenting and regulating auxiliary information on an image display of a telesurgical system to assist an operator in performing a surgical procedure
6574355, Jan 21 1992 Intuitive Surigical, Inc. Method and apparatus for transforming coordinate systems in a telemanipulation system
6659939, Nov 20 1998 Intuitive Surgical Operations, Inc Cooperative minimally invasive telesurgical system
6731988, Jan 21 1992 SRI International System and method for remote endoscopic surgery
7024750, Dec 20 2000 Outokumpu Oyj Method for the manufacture of layered metal product slabs and layered metal product slabs
7048745, Dec 08 1998 Intuitive Surgical Operations, Inc Surgical robotic tools, data architecture, and use
7107090, Dec 08 1998 Intuitive Surgical Operations, Inc Devices and methods for presenting and regulating auxiliary information on an image display of a telesurgical system to assist an operator in performing a surgical procedure
7525274, Mar 28 2003 KUKA Roboter GmbH Method and device for controlling a plurality of manipulators
7643862, Sep 15 2005 Biomet Manufacturing, LLC Virtual mouse for use in surgical navigation
7666191, Dec 12 1996 Intuitive Surgical Operations, Inc Robotic surgical system with sterile surgical adaptor
7699855, Dec 12 1996 Intuitive Surgical Operations, Inc Sterile surgical adaptor
7727244, Nov 21 1997 Intuitive Surgical Operations, Inc Sterile surgical drape
7763015, Jan 24 2005 Intuitive Surgical Operations, Inc Modular manipulator support for robotic surgery
7896815, Mar 01 2005 CHECKPOINT SURGICAL, INC Systems and methods for intra-operative stimulation
7955322, Dec 20 2005 Intuitive Surgical Operations, Inc Wireless communication in a robotic surgical system
8100133, Jun 30 2005 INTUITIVE SURGICAL OPERATION, INC Indicator for tool state and communication in multi-arm robotic telesurgery and method of use
8182470, Dec 20 2005 Intuitive Surgical Operations, Inc Telescoping insertion axis of a robotic surgical system
8273076, Jun 28 2005 Intuitive Surgical Operations, Inc Indicator for tool state and communication in multi-arm robotic telesurgery
8527094, Nov 20 1998 Intuitive Surgical Operations, Inc Multi-user medical robotic system for collaboration or training in minimally invasive surgical procedures
8894634, Jun 30 2005 Intuitive Surgical Operations, Inc. Indicator for tool state and communication in multi-arm robotic telesurgery
9259276, Jun 30 2005 Intuitive Surgical Operations, Inc. Indicator for tool state and communication in multiarm telesurgery
20030109780,
20040106916,
20040267254,
20050200324,
20060052684,
20060178559,
20070142969,
20120059390,
20150073437,
EP1462895,
GB2350696,
JP2003339736,
JP2005536314,
JP7024750,
JP7194610,
WO3094768,
WO2007005555,
WO9313916,
WO9426167,
WO9516396,
WO9530964,
WO9639944,
/
Executed onAssignorAssigneeConveyanceFrameReelDoc
Feb 01 2016Intuitive Surgical Operations, Inc.(assignment on the face of the patent)
Date Maintenance Fee Events
Oct 04 2022M1551: Payment of Maintenance Fee, 4th Year, Large Entity.


Date Maintenance Schedule
Apr 16 20224 years fee payment window open
Oct 16 20226 months grace period start (w surcharge)
Apr 16 2023patent expiry (for year 4)
Apr 16 20252 years to revive unintentionally abandoned end. (for year 4)
Apr 16 20268 years fee payment window open
Oct 16 20266 months grace period start (w surcharge)
Apr 16 2027patent expiry (for year 8)
Apr 16 20292 years to revive unintentionally abandoned end. (for year 8)
Apr 16 203012 years fee payment window open
Oct 16 20306 months grace period start (w surcharge)
Apr 16 2031patent expiry (for year 12)
Apr 16 20332 years to revive unintentionally abandoned end. (for year 12)