A system and method for controlling and selecting sources in a room on a network. The system allows a remote viewer to create a virtual presence within the room by providing the available displays, corresponding to the sources, to the remote viewer. The system includes a standardizing technique for improving the communication and overall switching of data for streaming on a network. The system can include a recording server for performing dual recording of the video files in each of a local database and a remote database. A graphical user interface (GUI) display is provided to guide a local user through a medical procedure in the standardized system.
|
16. An adapter unit comprising:
an input configured to be coupled to a single source, the single source being a source corresponding to the adapter unit and configured for use during a medical procedure, the source being configured to transmit video data in a source-specific format; and
an output configured to be coupled to a digital switch,
wherein the adapter unit is configured to:
receive information identifying the source from the source via the input;
provide the information identifying the source to the digital switch via the output;
receive the video data in the source-specific format from the source via the input;
normalize the video data in the source-specific format into a standard format; and
provide the video data in the standard format via the output.
7. A method for identifying sources used in a medical procedure performed at a medical treatment location, the method comprising:
identifying, by a first adapter unit configured to couple to one source, a single source, the single source being a first source coupled to the first adapter unit, the first source being configured to be used during a procedure in a medical treatment facility;
providing, by the first adapter unit, first information identifying the first source to a digital switch coupled to the first adapter unit, the digital switch connected to a control computer;
receiving, by the first adapter unit, first video data in a first source-specific format from the first source;
normalizing, by the first adapter unit, the first video data into a standard format; and
providing, by the first adapter unit, the first video data in the standard format to the digital switch.
1. A system for identifying sources used in a medical procedure performed at a medical treatment location, the system comprising:
a control computer configured to
couple to a digital switch;
receive information identifying a first source to be used in the medical procedure performed at the medical treatment location; and
present, via a graphical user interface configured to couple to the control computer, source identifying information based on the information identifying the first source; and
a first adapter unit configured to
couple to the digital switch;
couple to a single source corresponding to the first adapter unit, the single source being the first source;
identify the first source;
provide the information identifying the first source to the control computer via the digital switch;
receive video data in a first source-specific format from the first source;
normalize the video data into standard format video data; and
provide the standard format video data in the standard format to the digital switch.
21. A system for identifying sources used in a medical procedure performed at a medical treatment location, the system comprising:
a first adapter unit configured to couple to a first single source corresponding to the first adapter unit, the first single source being a first source used in the medical procedure performed at the medical treatment location, the first adapter unit configured to receive first video data in a first source-specific format from the first source and normalize the first video data into a standard format;
a second adapter unit configured to couple to a second single source corresponding to the second adapter unit, the second single source being a second source used in the medical procedure performed at the medical treatment location, the second adapter unit configured to receive second video data in a second source-specific format from the second source and normalize the second video data into the standard format; and
a digital switch configured to couple to the first adapter unit and the second adapter unit,
wherein the digital switch is configured to identify the first source based on first information identifying the first source that is received from the first adapter unit and to identify the second source based on second information identifying the second source that is received from the second adapter unit.
2. The system of
3. The system of
4. The system of
5. The system of
6. The system of
8. The method of
identifying, by a second adapter unit coupled to the digital switch, a second source coupled to the second adapter unit;
providing, by the second adapter unit, second information identifying the second source to the digital switch;
receiving, by the second adapter unit, second video data in a second source-specific format from the second source;
normalizing, by the second adapter unit, the second video data into the standard format; and
providing, by the second adapter unit, the second video data in the standard format to the digital switch.
9. The method of
10. The method of
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
17. The adapter unit of
18. The adapter unit of
19. The adapter unit of
20. The adapter unit of
22. The system of
23. The system of
24. The system of
25. The system of
|
This application claims the benefit under 35 U.S.C. § 120 as a continuation of U.S. application Ser. No. 13/708,903 filed Dec. 7, 2012, entitled “SYSTEM AND METHOD FOR CONTROLLING AND SELECTING SOURCES IN A ROOM ON A NETWORK”, which claims the benefit under 35 U.S.C. § 119(e) to U.S. Provisional Application Ser. No. 61/568,013, filed Dec. 7, 2011, entitled “SYSTEM AND METHOD FOR CONTROLLING AND SELECTING SOURCES IN A ROOM ON A NETWORK”, all of which are hereby incorporated herein by reference in their entireties for all purposes.
The present invention relates to systems and methods for controlling a plurality of video and other information sources in a room connected through a network.
In a hospital environment, it is crucial for all rooms, and more particularly, the various sources and personnel within each room to communicate effectively. The sources within the room can include a control computer, an overview camera, a microscope, a PACS (picture archiving and communication system) computer, an in-light camera, a surgical navigation system, a C-Arm in the room, an ultrasound system, a physiological monitor, a laparoscope, an endoscope, and other devices and instruments available in a room for performing a medical procedure or operation.
Communication in a room of a hospital network, or other appropriate network, is often hindered by the multitude of formats used for each source available in the room. It is a challenge to effectively communicate between the various sources using current communication and network techniques. There is a lack of a system that allows a single user to manipulate all sources within a room using a standardized technique. Current systems require complex switching to account for different formats and connectivities (i.e. some devices employ DVI, others employ HDMI, and yet others employ S-Video, or some combination thereof, etc.).
Furthermore, in the communication context, there are several systems and methods available that allow remote users to gain access to the room. These systems disadvantageously provide access to the sources themselves directly to the remote viewer. Accordingly, a local in-room user experiences undesirable interruptions and potential loss of control of the procedure and the devices and sources that perform the functions of the procedure.
It is desirable to avoid these and other disadvantages by providing a system that standardizes sources within a room for effective and efficient communication and provides an in-room user with exclusive control of the sources and the capture and/or display thereof while improving usability and streaming operating room (OR) workflow.
A system and method standardizes and controls communication within a hospital network or other appropriate medical environment network. The system allows a remote viewer to create a virtual presence in a room connected to the network. A virtual presence system can include a plurality of displays within a room that respectively correspond to a plurality of sources. A processing arrangement on a controller is provided that selects a display, in response to a selection by the remote viewer, to provide only the display of the source to the remote viewer. The selection and viewing by the remote viewer occurs free of interference to the local user during a medical procedure. This allows the local user within the room to have exclusive control of the sources in the room, free of control by the remote viewer. Further, the remote viewer has a virtual presence in the room by viewing based on the displays of sources and content available to the local user in the room. Illustratively, a virtual presence can be established by a plurality of cameras in the room (such as an operating room) that are dedicated to creating and maintaining the virtual presence, such that the context of the room determines the content available.
Illustratively, the system also can include a recording server that records the data corresponding to the plurality of displays within the room in a local database. This server can be in the room or remote. The recording server can also record the displays that are selected by the remote viewer in a local or remote database, thereby creating dual recordings of the displays as viewed within the room. This further allows for verification that all videos and pertinent data have been accurately and appropriately recorded. Sources available for the displays include a room overview camera, a microscope, a PACS computer, an in-light camera, a surgical navigation system, a C-Arm, an ultrasound system, a physiological monitor, a laparoscope and an endoscope, among others readily apparent to those having ordinary skill. More than one of a given device type can be active at any given time.
Additionally, locally within the room and/or at a remote location, a continuous-loop recording system can be provided. In an illustrative embodiment, the system includes a database and application that stores data from cameras (the virtual presence cameras, for example, or other cameras or devices within the room) onto the database. The continuous-loop recording system stores the data for a predetermined amount of time (for example, 1 day or 12 hours), and the data is available if something goes wrong. If there are no problems with a particular procedure continues to record in a loop, recording over previously recorded data such that a lesser amount of memory is employed to record.
A graphical user interface (GUI) display can be provided in connection with the virtual presence system that has a plurality of screens for guiding a local user through a medical procedure. The GUI can be provided using context-aware techniques to streamline the overall experience for the local staff and indirectly improve the patient experience.
To standardize the sources within the room and facilitate the in-room control by the local user, a controller or control computer can be provided that is operatively connected to a video matrix switch. The plurality of sources is also connected to the video matrix switch to perform a plurality of functions associated with the sources and other content in the room using the control computer. The system can also include a plurality of adapters, one for each of the control computer and each of the plurality of sources, so as to render the control computer and sources in a standard format to improve connectivity for various sources, for further control and manipulation of the sources by the local user.
The invention description below refers to the accompanying drawings, of which:
A system and method for controlling and selecting sources and other content in a room on a network allows a local user to control a plurality of sources in a room. The system further enables a remote viewer to experience a virtual presence in the room using a control computer by selecting from available displays corresponding to sources in the room. The room is a room in a medical environment, such as, but not limited to, an operating room (OR) in a hospital. The term “medical environment” as used herein, refers to a location in which a surgical, diagnostic, therapeutic, or other medical procedure is performed. The medical environment, or “medical treatment location” (used interchangeably herein) typically, but not always, has a doctor, hospital staff, and other medical personnel present, that assist in a medical procedure. The medical environment includes, but is not limited to, a hospital, trauma center, doctor's office, surgical center, and other locations known to those having ordinary skill in the art. The room can be a pathology room or a catheter lab, or other room in which a procedure occurs.
The MainStream system 110, as will be described in greater detail hereinbelow with reference to
A user viewing computer 112 selects one (or a plurality) of displays that are available in a room via datastream 115, which is interpreted by the LiveStream server 120 to select the sources via datastream 125 in the MainStream system 110. The available content sources are then displayed to the user viewing computer 112 so as to create a virtual presence in the room for a remote viewer, as though they are within the MainStream system 110. A version of the LiveStream server 120 is shown and described in co-pending commonly assigned U.S. patent application Ser. No. 11/510,337, entitled STREAMING VIDEO NETWORK SYSTEM, by Peter Renzi, et al., the teachings of which are incorporated by reference as useful background information, and also described in pertinent part below.
Reference is now made to
Referring again to
The displays and sources are collectively referred to as “content” herein. In an illustrative embodiment. The content of a room can automatically be determined by the context of the room. The context refers to the type of procedure, the status of the room, the state of the patient, equipment status, and determines a higher level of abstraction of a procedure to thereby determine the content (e.g., device) for a particular procedure. For example, if a patient is having laparoscopy performed, the context automatically determines that the content will include a laparoscope. Additionally, the content may change relative to the status of the room. For example, if a patient is entering into the room, the remote viewer will want to review the content of the virtual presence cameras, not the laparoscope. Accordingly, the content is determined by the context of the procedure (including patient context). As described in greater detail with reference to
Live videos and/or recorded videos (on the recording server 130) are accessed via datastream 127 by the LiveStream server 120. The LiveStream server 120 is a broker that controls access to live content. This enables a remote viewer to access both live and recorded video files. The VaultStream server 150 receives requests from a user viewing computer 112 via datastream 151 to select and view data stored in the recording server 130. The VaultStream server selects stored videos and data via datastream 152 to be accessed by the recording server 130. The stored videos and data are transmitted to the VaultStream server 150 via datastream 153 and then provided to the remote user viewing computer 112 via datastream 154. In this manner, a user viewing computer can view both live and stored video and data based upon the sources available within the MainStream system, while enabling the control of the actual sources to be exclusively within the MainStream system. A user also has access to an optional second set of sources that can be remotely selected. The data from the MainStream system, including videos and other data, is transmitted from the recording server 130 via datastream 134 into a local database 135. This allows the videos to be stored in local database. Simultaneously, videos can be transmitted to a remote database 155 via datastream 156. Accordingly, a dual recording of the videos is stored in both a local database 135 and a remote database 155. This improves overall accuracy and efficiency of the system, and allows users to access both locally stored videos and remotely stored videos without waiting for lengthy file transfer operations. Further, verification can be performed to ensure that all videos are properly stored at both locations (see
Reference is now made to
The Data Center 230 can also include a medpresence gateway computing platform 232 operatively connected to the servers 120, 130, 150. The medpresence gateway platform 232 has a virtual presence application running thereon that provides a virtual view to a remote viewer (for example on the medpresence network 294). The virtual view can be established on the medpresence network through video received from the virtual presence cameras (e.g. 205, 206, 207 or 165, 166, 167) located in room(s) of the medical environment. The virtual view can be a 360-degree view from combined camera views in accordance with ordinary skill. A “platform” as used herein refers to a hardware architecture and software framework that allows software, particularly application software, to run. Typical platforms include computer architecture, operating systems, programming languages, related user interface, and other platforms known in the art and readily applicable to those having ordinary skill.
The operating rooms 201, 202 and specialty area room 203 can also be connected through the network 225 to a control room 240 for remote monitoring and control applications. The control room 240 is employed in alternate embodiments for external control of the MainStream systems, as desired. Various other components and entities of the medical environment can also be connected through the network 225, including a full pathology room 250, a basic pathology room 255, a nurse station 260, a typical conference room 270 and a doctor's office 280.
The pathology room 250 is similar to any operating room in the control system, and includes a MainStream system for standardizing communication between sources within the room and overall communication. The pathology room 250 includes a router 210, operatively connected to a grossing station 251, a microscope 252, a PIP (picture-in-picture) option 253, and a wall display 254. A nStream RX device 221 is provided for appropriate connectivity to the network 225. The pathology room 250 is a full pathology room showing many available functionalities and hardware components available using the MainStream system. A basic pathology room 255 can also be employed, which includes a grossing station 251, a microscope 252, and a display (LSP) 256. An nStream RX device is capable of receiving two streams of input and converting into multiple streams. The RX device converges the recording and streaming so that the streams are sent to the server and the recording is sent to the server. The nStream RX device is capable of taking a single video input and turning it into three outbound streams of different resolution. For example, the RX device can take a single input stream and convert into a 480p, 720p and 1080p resolution output. The illustrative system employs a spatial matrix from a single stream to provide multiple output streams at varying resolutions. Multiple output streams at different resolutions is particularly useful in the medical context where the output stream can be predetermined based upon the particular source from which video is being streamed. For example, certain streams are desired to be in higher resolution, like certain streams can be in a lower resolution.
The nurse station 260 is also in communication through the network 225 and includes a status board 261 (shown and described in greater detail below with reference to
The system can also be connected through an appropriate router-firewall 290 through a WAN 292 to a medpresence network 294 to establish a virtual presence for users in the room. The virtual presence is established through the gateway platform 232 operatively connected to system servers (e.g. 120, 130, 150). In accordance with illustrative embodiments, a client application on a computing device of a remote user or remote viewer of the medpresence network 294 is employed to select a display within the room of the medical environment, after the virtual view has been provided to the remote user on the medpresence network 294. Remote users can thereby establish the virtual presence in a room connected to the LAN 225, including the Operating Room One 201, Operating Room N 202, Specialty Area Room 203, full pathology room 250, basic pathology room 255 and typical conference room 270.
The MainStream system 110, shown by way of example, as Operating Room One 201, Operating Room N 202, full Pathology room 250, basic Pathology room 255 and Conference Room 270 in
According to the standardized system, each device is provided with an adapter (adapter A 307 for device A 301, adapter B 308 for device B 302, and through to adapter N 309 for device N 303) which renders the device format 304, 305, 306 into a standard format 310, 311, 312 for the digital switch. This improves overall system connectivity by standardizing various formats for streaming via the switch. Moreover, ports 315, 316, 317 are provided for connectivity to the digital switch 320 so that the devices 301, 302, 303 can be quickly connected to the digital switch by simply connecting the appropriate adapter 307, 308, 309 into a port 315, 316, 317 of the digital switch 320. More particularly, in an exemplary embodiment, the devices includes formats such as SVideo (Separate Video), DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), and other source formats. The adapters normalize the source-specific format into a standardized format for input to the switch. The standard output from the adapters can be DVI format and include a power channel and a data channel. Because the adapters are equipped with a power channel, no additional power is needed to perform the standardization of data. The switch 320 then switches only standardized formatted data and a single switch can accordingly be used for a plurality of differing type sources. An exemplary adapter configured to perform the standardization of format is performed implementing readily available video input chipsets, FPGA chips, FPGA firmware, and a chipset provided by Gennum Corporation of Burlington, Ontario, Canada for long distance video transport. The adapters provide appropriate connectivity for a source, depending on the output format type for the source. Adapters include the Gennum chipset to standardize the output from the source and include appropriate output connectivity to be connected to the switch for video streaming and other transmission. Adapters are also able to identify their corresponding source and relay this information to the switch using the data channel. Accordingly, there is no need to register the devices with the network or other systems, there is only a need to perform a one-time configuration with the adapter to its source, and the adapter can communicate the source identifying information through the switch.
A charting PC (personal computer) 330 is provided in the MainStream system to control the overall system and data pertaining to a patient. The PC format 331 is sent to the adapter 332, which provides a standard format 333 to the port 334. The port 334 is in communication with the switch 320 for further manipulation and control of the charting PC 330 output. The controller 340 is in communication with a software or hardware based keyboard, video and mouse switch (KVM) 341 and in communication with a system database 345. The controller 340 controls the KVM 341 to determine which computing device within the room will be controlled by the controller 340. In this manner, a single controller can perform the various functions of the system without requiring multiple keyboards, mice and displays for each computer. The computer 330 includes its own adapter for communication to the switch, and orchestrates the switching between the various computers and sources by switching keyboard and mouse using the KVM 341. Typical KVM switches also control video, however this embodiment controls only the keyboard and mouse for the various sources in the room. The KVM switch can be implemented in hardware or software in accordance with ordinary skill. Furthermore, the standard format improves communication and can be performed for other devices on the network when being sent from the digital switch 320.
The digital switch 320 transmits data in standard format 350, 351 to appropriate adapters that are connected to various devices within the room. Adapter AA 352 receives the standard format 350 and transmits appropriate device format 354 to device AA 356. Likewise, adapter AB receives the standard format 351 and transmits appropriate device format 355 to the device AB 357. In this manner, devices can appropriately receive data from the digital switch using a standard format by applying an adapter according to the system.
It is noted that the devices (A, B, . . . N, AA, AB) and various adapters (A, B, . . . N, AA, AB) are described hereinabove as being within a single “room”, however any orientation is appropriate for a closed network in which the devices are used for performing a procedure. For example, the devices can represent various devices, cameras, monitors, and other sources that observe or are used during a surgery, operation, treatment or other medical procedure.
The digital switch 320 is also operatively connected to the LiveStream server through appropriate adapter and IP Encoder/Compressor 361 for outbound streaming of video files and other pertinent data. The standard format is transmitted from the switch via 360 to the Encoder/Compressor 361, which is then transmitted to LiveStream server 120. This can be performed, for example, using H264 encoding as appropriate for networked communication. Accordingly, the LiveStream server 120 can receive device data quickly and efficiently by applying adapters to the devices and employing a digital switch for switching the standardized data. Illustratively, the LiveStream server performs bi-directional communication with the room through the switch and appropriate encoders and decoders, as is readily apparent to those having ordinary skill. The LiveStream server essentially brokers the connectivity of all rooms through interconnectivity with a network.
Reference is now made to
During the briefing phase 540, an operating room checklist 541 is verified to ensure operating room is prepared for the operation. See
After the intra-op stage 550, the procedure enters a done phase 560, where a safety checklist 561 is provided and the user may complete the procedure by saving or discarding video, images and/or deciding whether or not to print images. See
Reference is now made to
From the home screen 600, a local user selects the Setup tab 602 and is directed to, for example, a setup screen 700 as shown in
In accordance with the illustrative embodiments, the entry phase screen 800 can contain information that is obtained automatically based upon the HL7 messages for automatic registration, and corresponding procedure information available within the system or acquired automatically based upon the context (e.g. type of surgery, state of patient, equipment status, staff present) of the room. Accordingly, the checklist 810 of
Once the patient data has been acquired, procedure data is then acquired at step 848, either by manual entry by the user or through automated data transfer of the particular procedure that is related to the patient associated with the HL7 message. The procedure data can also be acquired by determining the status of the procedure based upon the location of persons within the room and status of equipment in the room. At step 850, the content (e.g. sources and displays) associated with the patient and procedure are determined. The contextual factors (i.e. patient status and procedure status) can automatically determine the content of the room. For example, if no equipment is in use and the patient is entering the room, the status can be assigned “entry” and the associated presets for types of content can be set, as defined by a physician or in accordance with hospital or HIPAA standards for care during a procedure. If the context determines that a laparoscopy is to be performed, then the content includes a laparoscope and is set to a desired display, as determined by a physician, or in accordance with HIPAA standards. In this manner, when a patient enters into a room, the room becomes context-aware that the patient has entered the room, based upon the HL7 messages. This advantageously reduces the chances of a problem during surgery by properly identifying the patient and the particular procedure to be performed. In addition, a burden is removed from the circulating nurse responsible for managing the operating room during the procedure, as he/she no longer needs to manually enter data into redundant systems.
Next, during the briefing phase, a local user is directed to the exemplary GUI display of
During the intra-op phase of the medical procedure, a local user is directed to a intra-op screen 1000, for example as shown in the diagram of the exemplary GUI display of
An exemplary GUI display showing a diagram of a nurses charting function screen 1100 is shown in
At the end of the medical operation, the local user enters the done stage and is directed to a done screen 1200, as shown in the diagram of the exemplary GUI display of
The exemplary GUI display also includes a plurality of tabs for adjusting the various settings within a particular room or performing various functions including the audio, video, lights, and communication in the room, among other settings and preferences. Selecting the “Audio” tab 613 directs the local user to an audio settings screen 1400, as shown in the exemplary GUI display of
Selecting the “Status” tab 608 directs a support person to a status screen 1700, as shown in the exemplary GUI display of
Reference is now made to
The exemplary GUI displays described above employ the systems and methods described herein to implement the MainStream system and LiveStream server in the illustrative embodiments. The system provides an application for performing a medical procedure and allows the local user to have exclusive control of the procedure and functions associated with various sources. A remote viewer is able to view the available displays corresponding to the sources being controlled by the local user to create a virtual presence as though the remote viewer is in the room where the operation is being performed. The system improves communication by standardizing the format for switching and streaming files and content over a network.
The teachings herein enable a VaultStream server (e.g. server 150 of
Reference is now made to
With continued reference to
The devices, systems and methods shown and described herein improve usability of OR systems and streamline OR workflow, as well as usability and workflow of any medical environment in which a procedure is performed. The sources in the room are standardized for effective and efficient communication to allow for real-time access to live video streams.
The foregoing has been a detailed description of illustrative embodiments of the invention. Various modifications and additions can be made without departing from the spirit and scope of this invention. Each of the various embodiments described above may be combined with other described embodiments in order to provide multiple features. Moreover, while the foregoing describes a number of separate embodiments of the apparatus and method of the present invention, what has been described herein is merely illustrative of the application of the principles of the present invention. For example, specific arrangements of various components and entities of the system have been shown for illustrative purposes. However, any appropriate arrangement which employs the teachings herein is expressly contemplated. For example, the particular arrangement of various servers within a network is highly variable. Furthermore, appropriate access controls can be provided for the networked communication to comply with health insurance and data management standards, as appropriate. Additionally, several particular sources and medical procedures have been described for illustrative purposes, however any number of sources and medical procedures can implement the systems and methods described herein, without departing from the scope of this invention. Accordingly, this description is meant to be taken only by way of example, and not to otherwise limit the scope of this invention.
Mitchell, Eddie E., Renzi, Peter
Patent | Priority | Assignee | Title |
10965912, | Dec 07 2011 | GYRUS ACMI, INC D B A OLYMPUS SURGICAL TECHNOLOGIES AMERICA | System and method for controlling and selecting sources in a room on a network |
11557392, | May 03 2019 | GYRUS ACMI, INC | Context and state aware treatment room efficiency |
11676716, | Dec 07 2011 | Gyrus ACMI, Inc. | System and method for controlling and selecting sources in a room on a network |
11783193, | May 03 2019 | Gyrus ACMI, Inc. | Context and state aware treatment room efficiency |
Patent | Priority | Assignee | Title |
6002398, | Oct 30 1997 | Apple Inc | Navigation between property pages with tabs and menus |
7616594, | Apr 22 2005 | Microsoft Technology Licensing, LLC | Wireless device discovery and configuration |
20070050828, | |||
20090213140, | |||
20100157016, | |||
20100257190, | |||
20100295870, | |||
20100325546, | |||
20120166996, | |||
EP1143334, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Aug 11 2016 | Image Stream Medical, Inc. | (assignment on the face of the patent) | / | |||
Sep 01 2016 | MITCHELL, EDDIE E | IMAGE STREAM MEDICAL, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 040118 | /0660 | |
Sep 01 2016 | RENZI, PETER | IMAGE STREAM MEDICAL, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 040118 | /0660 | |
Feb 01 2021 | IMAGE STREAM MEDICAL, INC | GYRUS ACMI, INC | MERGER SEE DOCUMENT FOR DETAILS | 056101 | /0400 |
Date | Maintenance Fee Events |
Nov 08 2018 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Jul 01 2022 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Date | Maintenance Schedule |
Jan 01 2022 | 4 years fee payment window open |
Jul 01 2022 | 6 months grace period start (w surcharge) |
Jan 01 2023 | patent expiry (for year 4) |
Jan 01 2025 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jan 01 2026 | 8 years fee payment window open |
Jul 01 2026 | 6 months grace period start (w surcharge) |
Jan 01 2027 | patent expiry (for year 8) |
Jan 01 2029 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jan 01 2030 | 12 years fee payment window open |
Jul 01 2030 | 6 months grace period start (w surcharge) |
Jan 01 2031 | patent expiry (for year 12) |
Jan 01 2033 | 2 years to revive unintentionally abandoned end. (for year 12) |