A resource integration method includes the following steps: a receiving module receives access information from a guest operating system on the host device; the access information is used to determine whether the frame rate is lower than a frame rate threshold; when the receiving module determines that the frame rate is lower than the frame rate threshold, the receiving module transmits an external resource request signal to the receiving module; after the receiving module receives the external resource request signal, a resource management module (which is located in the bridge module) selects an optimal external device from a specific category (among a plurality of categories in a candidate list), and a calculation operation or a storage operation corresponding to the specific category is transmitted to the optimal external device for calculation or storage by the bridge module.
|
1. A resource integration system suitable for a virtual environment, the resource integration system comprising:
a receiving module, configured to receive access information from a guest operating system and transmits the access information to a host device, and determine whether a frame rate is lower than a frame rate threshold according to the access information;
when the receiving module determines that the frame rate is lower than the frame rate threshold, the receiving module transmits an external resource request signal; and
a bridge module, wherein after receiving the external resource request signal, a resource management module in the bridge module selects a specific external device from a specific category among a plurality of categories in a candidate list based on a result of a packet test transmitted using the candidate list, and transmits a calculation operation or a storage operation corresponding to the specific category to the specific external device for calculation or storage.
9. A resource integration method suitable for a virtual environment, the resource integration method comprising:
receiving access information from a guest operating system and transmitting the access information to a host device by a receiving module;
determining whether the frame rate is lower than the frame rate threshold according to the access information;
when the receiving module determines that the frame rate is lower than the frame rate threshold, the receiving module transmits an external resource request signal to the receiving module; and
after the external resource request signal is received, a resource management module in the bridge module selects a specific external device from a specific category among a plurality of categories in a candidate list based on a result of a packet test transmitted using the candidate list, and transmits a calculation operation or storage operation corresponding to the specific category to the specific external device for calculation or storage by a bridge module.
2. The resource integration system of
an external resource detection module, configured to detect hardware information of an external device when the external device is coupled to the host device, and to detect whether the external device comprises an operating system; wherein when the external device comprises an operating system, the external resource detection module enumerates the external device in the candidate list according to the hardware information; wherein the hardware information comprises hardware performance or hardware specifications.
3. The resource integration system of
4. The resource integration system of
5. The resource integration system of
6. The resource integration system of
7. The resource integration system of
8. The resource integration system of
a connection medium detection module, configured to identify a connection medium used by the external device when the external device is coupled to the host device.
10. The resource integration method of
detecting hardware information of an external device when the external device is coupled to the host device, and using an external resource detection module to detect whether the external device comprises an operating system, wherein when the external device comprises an operating system, the external resource detection module enumerates the external device in the candidate list according to the hardware information; wherein the hardware information comprises hardware performance or hardware specifications.
11. The resource integration method of
the resource management module creates the plurality of categories according to the hardware information returned by the external resource detection module and the connection medium returned by the connection medium detection module, wherein the plurality of categories comprise a bandwidth category, a memory category, and a graphics bandwidth type.
12. The resource integration method of
13. The resource integration method of
14. The resource integration method of
15. The resource integration method of
16. The resource integration method of
using an external resource detection module to identify a connection medium used by the external device when the external device is coupled to the host device.
|
This Application claims priority of Taiwan Patent Application No. 109127894, filed on Aug. 17, 2020, the entirety of which is incorporated by reference herein.
The present disclosure relates to an integration system and, in particular, to a resource integration system and resource integration method suitable for a virtual environment.
Most electronic sports (e-sports) game competitions run on the Windows operating system. At present, large e-sports companies (such as Valve Corporation) have purchased the Windows operating system versions of 6,500 games for execution on Linux platforms.
It can be seen that playing the Windows operating system version of games on platforms that run Linux has gradually become a trend, and will likely remain a trend well into the future. The Windows operating system version of a game or program must be run on a Chromebook or Linux operating system platform. If it is executed with a simulated tool program, all application programming interfaces (API) and runtimes are set together, and it directly accesses the operating system of the Linux platform during execution. This method is not very stable, and it can cause the program to crash easily. Therefore, the industry mostly adds a virtual machine to protect it.
However, there are still differences in performance between running e-sports games on virtual machines and running e-sports games directly on a Windows operating system. Therefore, how to balance stability and game experience has become one of the problems that need to be solved in this field.
In accordance with one feature of the present invention, the present disclosure provides a resource integration system suitable for a virtual environment. The resource integration system comprises a receiving module and a bridge module. The receiving module is configured to receive access information from a guest operating system on the host device, and determine whether the frame rate is lower than a frame rate threshold according to the access information; when the receiving module determines that the frame rate is lower than the frame rate threshold, the receiving module transmits an external resource request signal. After the external resource request signal is received, a resource management module (which is located in the bridge module) selects an optimal external device from a specific category (among a plurality of categories in a candidate list), and the resource management module transmits a calculation operation or a storage operation corresponding to the specific category to the optimal external device for calculation or storage.
In accordance with one feature of the present invention, the present disclosure provides a resource integration method suitable for a virtual environment. The resource integration method includes the following steps: the receiving module receives access information from a guest operating system on the host device; the receiving module determines whether the frame rate is lower than the frame rate threshold according to the access information; when the frame rate is lower than the frame rate threshold, the receiving module transmits an external resource request signal to the receiving module; after the receiving module receives the external resource request signal, the resource management module (which is located in the bridge module) selects an optimal external device from a specific category (among a plurality of categories in a candidate list), and a calculation operation or a storage operation corresponding to the specific category is transmitted to the optimal external device for calculation or storage by a bridge module.
Therefore, during rendering, or during other operations that use the resource integration system and resource integration method, the bridge module and the receiving module allow the original system to integrate system resources after connecting with external devices. The guest operating system in the virtual environment determines whether or not there are sufficient hardware resources, and it detects which types of hardware resource are insufficient. Based on the insufficient hardware types, the bridge module selects the best hardware type from the external device, and a computing operation or a storage operation is handed over to the best hardware type, so that performance and user experience of running applications on the guest operating system can be improved.
In order to describe the manner in which the above-recited and other advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific examples which are illustrated in the appended drawings. It should be understood that these drawings depict only exemplary aspects of the disclosure and are not therefore to be considered to be limiting of its scope. The principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings, in which:
The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
The present invention will be described with respect to particular embodiments and with reference to certain drawings, but the invention is not limited thereto and is only limited by the claims. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Use of ordinal terms such as “first”, “second”, “third”, etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having the same name (but for use of the ordinal term) to distinguish the claim elements.
Refer to
When rendering is required, the application program 10 sends a request (rendering-related execution instructions) to the hardware access application program interface (API) 20. The hardware access application program interface 20 sends the request to the virtio-GPU 22. The virtio-GPU 22 then transmits the request to the virtio-GPU 24. The virtio-GPU 24 will pass the request to hardware acceleration application program interface (Hardware Acceleration API) 60 of the host. The hardware accelerated application program interface 60 transmits the request to the rendering management driver 72 of the host kernel 70. The rendering management driver 72 transmits the request to the graphics processing unit 88 and the graphics memory 87 in the hardware layer 80. The graphics processing unit 88 mainly performs rendering work and sends the rendering result back to the guest operating system Guest OS according to the path of the transmission request. The guest operating system Guest OS presents the image on the display.
The hardware acceleration application programming interface 60 is an acceleration API (it is known that OpenGL and Vulkan can be used to implement the acceleration API).
It can be seen from the above that the information transmission between the hardware access and the image presentation is covered by the virtual machine VM, and the upper- and lower-layer instructions need to be converted again. The architecture in
Therefore, the present invention proposes the resource integration system 200, the resource integration method 300, and the resource integration system 400 shown in
Please refer to
In one embodiment, the bridge module 52 can be implemented by using an integrated circuit, such as a microcontroller, a microprocessor, a digital signal processor, an application specific integrated circuit (ASIC), or a logic circuit. However, it is not limited thereto. In one embodiment, the bridge module 52 can be implemented by software or firmware.
In one embodiment, the receiving module 54 can be implemented by using an integrated circuit, such as a microcontroller, a microprocessor, a digital signal processor, an application specific integrated circuit, or a logic circuit. However, it is not limited thereto. In one embodiment, the receiving module 54 can be implemented by software or firmware.
In one embodiment, the bridge module 52 is used to enable a host operating system Host OS of a host device to communicate with an external device (for example, external devices ED1-ED3) and obtain external device information.
In one embodiment, when the external devices ED1-ED3 are communicatively coupled with the Universal Serial Bus (USB) 82 and/or the network interface 84 of the host device, the host kernel 70 transmits the external device information of the external devices ED1-ED3 to the bridge module 52. For example, when the external device ED1 is communicatively coupled with the Universal Serial Bus 82 and/or the network interface 84, the host kernel 70 transmits the external device information of the external devices ED1-ED3 through the path of the hardware layer 80, the host kernel 70, and the hardware accelerated application program interface 60 to the bridge module 52.
In one embodiment, the external devices ED1 and ED2 are electronic devices such as mobile phones, tablets, laptops, servers, etc.
In one embodiment, the external device information is, for example, the memory size of the external devices ED1, ED2, the size of the storage device, processor-related information, whether there is an operating system, whether there is a virtual environment or virtual device . . . and other software and/or hardware information.
In one embodiment, the receiving module 54 is used to receive an access command from a guest operating system Guest OS to the host device, receive a hardware processing result from the host device, and selectively transmit the access command through the bridge module 52 to the external devices ED1, ED2 and receive the hardware processing results from the external devices ED1, ED2. The hardware processing result corresponds to the access command.
In one embodiment, the host device executes a virtual environment VE (for example, implemented by a virtual machine) on the operating system Host OS of the host device. The guest operating system Guest OS executes in the virtual environment VE. And, the guest operating system Guest OS executes an application program 10. The application program 10 is used to issue access commands.
For example, when the application program 10 wants to perform rendering, it transmits the rendering command (treated as an access command) and related information to the hardware access application program interface 20. The hardware access application program interface 20 transmits the rendering command and related information to the virtualized device 30. The virtualized device 30 transmits the rendering command and related information to the virtualized device 40. The virtualized device 40 sends rendering commands and related information to the hardware access library 50. The hardware access library 50 transmits rendering commands and related information to the hardware acceleration application program interface 60. The hardware accelerated application program interface 60 sends the rendering commands and related information to the host kernel 70. The host kernel 70 transmits the rendering commands and related information to the graphics processing unit 88, the memory and video random access memory 86 and the processor 89. The graphics processing unit 88 is mainly used for rendering. The graphics memory 87 can temporarily store information related to rendering. The processor 89 can selectively assist in the calculation. The graphics processing unit 88 returns the rendering result to the guest operating system Guest OS according to the path of the transmission request, and the guest operating system Guest OS presents the image to the user.
The bridge module 52 and receiving module 54 are in a hardware access library 50 of the host device. The hardware access library 50 is between a virtualization device 40 in the virtual environment VE and a hardware acceleration application program interface 60 of the host device. The internal operation of hardware access library 50 will be described in the explanatory paragraphs corresponding to
Please refer to
In step 310, a receiving module 54 receives access information SIG from a guest operating system Guest OS and transmits the access information SIG to a host device.
In an embodiment, the access information SIG may be a numeric value or machine code output from an intermediary layer (for example, the virtualized device 40).
In one embodiment, the access information SIG includes frame rate (frame per second, FPS). The frame rate can reflect the number of pictures presented per second. When the application program 10 is a three-dimensional game, an e-sports game or a application requiring detailed and fast presentation of the image, the sufficient hardware resources (such as processor, graphics processor, memory, etc.) are required to generate the application program 10 required frame rate to provide users with a good user experience.
In step 320, the receiving module 54 determines whether a frame rate is lower than a frame rate threshold according to the access information SIG.
When the receiving module 54 determines that the frame rate is lower than the frame rate threshold according to the access information SIG, step 330 is performed. For example, when the frame rate (for example, 30 fps) is lower than the frame rate threshold (for example, 80 fps), step 330 is performed.
When the receiving module 54 determines that the frame rate is not lower than the frame rate threshold according to the access information SIG, it returns to step 310 to continuously detect the frame rate, or ends the process.
In step 330, the receiving module 54 transmits an external resource request signal REQ.
In one embodiment, as shown in
In one embodiment, the rendering resource analysis module 541, the frame rate determination module 542, and the external request module 543 can be implemented as, for example, a micro-control unit, a microprocessor, a digital signal processor, and a special application integrated circuit, or a logic circuit. In one embodiment, the rendering resource analysis module 541, the frame rate determination module 542, and the external request module 543 can be implemented by software or firmware separately or together.
In step 340, after a bridge module 52 receives the external resource request signal REQ, a resource management module 521 in the bridge module 52 selects an optimal external device (for example, the external device ED1) from a specific category (for example, category P1) in a plurality of categories P1 to P3 in a candidate list, and sends a calculation operation or a storage operation corresponding to a specific type to the optimal external device for calculation or storage.
In
In one embodiment, the external resource detection module 522 is used to detect hardware information of the external device when the external device (e.g., external devices ED1 to ED3) is coupled to the host device, and to detect whether the external device contains an operating system. When the external device contains an operating system, the external resource detection module 522 enumerates the external device into the candidate list according to the hardware information. When the external device does not contain an operating system, this external device is not used.
The hardware information includes the hardware performance or hardware specifications of at least one external device. For example, the external device ED1 has two 8 GB memories and an Intel® Core™ i7 processor. For example, the external device ED2 has two 4 GB memories and an Intel® Core™ i5 processor. The candidate list can include hardware information corresponding to each external device.
In one embodiment, the external resource detection module 522 will check whether the external device connected to the host device has an operating system, hardware acceleration application interface, and the perform the resource detection such as detecting the hardware performance of the external device (such as processor performance), hardware specifications (such as graphic processor specifications), memory size, etc.
In one embodiment, the connection medium detection module 523 identifies a connection medium used by the external device when an external device is coupled to the host device. In one embodiment, the connection medium detection module 523 identifies the connection medium of the connected external device (such as universal serial bus (USB) 2.0, USB 3.0, local area network (LAN)), wireless local area networks (WLAN), etc.
In one embodiment, the resource management module 521 creates these categories according to hardware information returned by the external resource detection module 522 and a connection medium returned by a connection medium detection module 523. These categories include a bandwidth category P1, a memory category P2, and a graphics bandwidth category P3. In other words, when the resource management module 521 detects that an external device is connected to the host device, it creates these categories. However, the present invention is not limited to these categories, and the resource management module 521 can create categories according to various hardware in the external device.
In one embodiment, the bandwidth category records external devices having processor (for example, external devices ED1, ED2, ED3), and processor-related information, such as the number of processors, performance, specifications . . . , and other information and the bandwidth of the external device.
In one embodiment, the memory category records external devices having memory (for example, external devices ED1, ED2, ED3), and memory-related information, such as the number of memories, size, specification, etc.
In one embodiment, the graphics bandwidth category records external devices having graphics processor (for example, external devices ED1 and ED2), and graphics processor-related information, such as the number of graphics processors, size, specification, etc.
In one embodiment, when the bridge module 52 receives the external resource request signal REQ, the resource management module 521 in the bridge module 52 determines whether the host device (for example, the processor 89 in
In one embodiment, when the frame rate is lower than the frame rate threshold and the resource management module 521 determines whether the host device is busy, the resource management module 521 will perform a packet test, send the packet to each external device ED1-ED3, and select the one with the fastest bandwidth network transmission speed from the external devices ED1-ED3 is regarded as the optimal external processor, and the calculation operation corresponding to the bandwidth category P1 is transmitted to the optimal external processor (the processor of the external device ED1) for calculation.
In one embodiment, when the optimal external processor (for example, the processor of the external device ED1) performs calculations, the resource management module 521 determines that the host system is still busy, which means that the processor of the main system plus the processor of the external device ED1 is still insufficient in terms of calculation speed, the resource management module 521 selects another external processor (such as the processor of the external device ED2) corresponding to the second best bandwidth from the bandwidth category P1 in the candidate list to prompt the other external processor to assist in performing the calculation operation.
In one embodiment, the processor of the external device ED1 is, for example, Intel® Core™ i7, which uses 5G bandwidth. The processor of the external device ED2 is, for example, Intel® Core™ i5, using 4G bandwidth. The processor of the external device ED3 is, for example, Intel® Core™ i3, using 3G bandwidth. The models of these processors are recorded in the bandwidth category P1 in the candidate list. The resource management module 521 selects the processor of the external device ED1 with the best performance (which may be the fastest computing speed or the fastest transmission speed) from the bandwidth category P1 in the candidate list as an optimal external processor.
In one embodiment, if the resource management module 521 determines that the host device is not busy, it determines whether other devices have insufficient resources. For example, the resource management module 521 determines whether the memory space resources are insufficient and/or the graphics processor computing resources are insufficient.
In one embodiment, the external device ED1 has two 8 GB memories. The external device ED2 has two 4 GB memories. The external device ED3 has a 4 GB memory. The memory of the external device ED1 is the largest of the external devices ED1, ED2, and ED3. The models of these memories are recorded in the memory category P2 in the candidate list. If the resource management module 521 determines that the host device is not busy, the resource management module 521 determines whether the memory usage rate is higher than a memory usage threshold. If the resource management module 521 determines that the memory usage rate is higher than the memory usage threshold, and the specific category is regarded as the memory category P2, the resource management module 521 selects an optimal external memory from the memory category P2 in the candidate list (because the memory capacity of the external device ED1 is larger, the memory of the external device ED1 is selected), and the storage operation corresponding to the memory type P2 is transferred to the optimal external memory for storage. The optimal external memory is an external memory with the most storage space in the memory category P2.
In one embodiment, when the storage space of the host device (such as the memory and video random access memory 86 in
In one embodiment, both the external device ED1 and the external device ED2 have graphics processors. The performance of the graphics processor of the external device ED1 is better. In one embodiment, if the resource management module 521 determines that the host device is not busy, the resource management module 521 determines whether the graphics processor usage rate is higher than the graphics processor usage threshold. If the resource management module 521 determines that the graphics processor usage rate is higher than the graphics processor usage threshold, and the specific category is regarded as the graphics bandwidth category P3, the resource management module 521 selects an optimal external graphics processor from bandwidth category P3 in the candidate list (for example, the graphics processor of the external device ED1), and transmits the calculation operation corresponding to the graphics bandwidth category P3 to the optimal external graphics processor for calculation. The optimal external graphics processor is an external graphics processor with the best computing performance in the graphics bandwidth category P3.
In one embodiment, when the computing performance of the host device (such as the graphics processing unit 88 in
Therefore, in the mechanism of rendering or other operations using the resource integration system and resource integration method, the mechanism of the bridge module and the receiving module allow the original system to integrate the system resources after connecting with external devices. The guest operating system in the virtual environment determines whether there is insufficient hardware resource at present, and detects those types of hardware resource that are insufficient. According to the insufficient hardware type, the bridge module selects the best one of this hardware type from the external device, the computing operation or storage operation is handed over to the best one of this hardware type, so that the performance and user experience of running applications on the guest operating system can be improved.
The method of the present invention, or a specific type or part thereof, can exist in the form of code. The code can be contained in physical media, such as floppy disks, CDs, hard disks, or any other machine-readable (such as computer-readable) storage media, or not limited to external forms of computer program products. When the program code is loaded and executed by a machine, such as a computer, the machine becomes a device for the present application. The code can also be transmitted through some transmission medium, such as a wire or cable, optical fiber, or any transmission type. When the code is received, loaded and executed by a machine, such as a computer, the machine becomes used to apply in the present application. When implemented in a general-purpose processing unit, the program code combined with the processing unit provides a unique device that operates similar to the application of specific logic circuits.
Although the invention has been illustrated and described with respect to one or more implementations, equivalent alterations and modifications will occur or be known to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In addition, while a particular feature of the invention may have been disclosed with respect to only one of several implementations, such a feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application.
Huang, Chih-Wen, Hsu, Wen-Cheng, Yang, Chao-Kuang, Chang, Hung-Ming, Chen, Kuan-Ju
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
10217444, | Nov 28 2012 | Nvidia Corporation | Method and system for fast cloning of virtual machines |
10853118, | Dec 21 2015 | Intel Corporation | Apparatus and method for pattern-driven page table shadowing for graphics virtualization |
9167055, | Dec 24 2012 | WIWYNN CORPORATION | Method and system of displaying frame |
9311169, | May 02 2012 | Nvidia Corporation | Server based graphics processing techniques |
9996892, | Nov 21 2014 | Intel Corporation | Apparatus and method for efficient graphics processing in a virtual execution environment |
20130332456, | |||
20140055470, | |||
20140196034, | |||
20140281008, | |||
20140330888, | |||
20150205630, | |||
20160044139, | |||
20170371614, | |||
20190034161, | |||
20200201733, | |||
20200249936, | |||
TW201407536, | |||
TW201426325, | |||
TW201506843, | |||
TW201624278, | |||
TW201706834, | |||
TW201727576, | |||
WO2020101142, | |||
WO2020149722, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Sep 30 2020 | CHEN, KUAN-JU | Acer Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056286 | /0309 | |
Oct 15 2020 | HSU, WEN-CHENG | Acer Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056286 | /0309 | |
Oct 15 2020 | CHANG, HUNG-MING | Acer Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056286 | /0309 | |
Oct 15 2020 | HUANG, CHIH-WEN | Acer Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056286 | /0309 | |
Oct 15 2020 | YANG, CHAO-KUANG | Acer Incorporated | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056286 | /0309 | |
May 19 2021 | Acer Incorporated | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
May 19 2021 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Date | Maintenance Schedule |
May 28 2027 | 4 years fee payment window open |
Nov 28 2027 | 6 months grace period start (w surcharge) |
May 28 2028 | patent expiry (for year 4) |
May 28 2030 | 2 years to revive unintentionally abandoned end. (for year 4) |
May 28 2031 | 8 years fee payment window open |
Nov 28 2031 | 6 months grace period start (w surcharge) |
May 28 2032 | patent expiry (for year 8) |
May 28 2034 | 2 years to revive unintentionally abandoned end. (for year 8) |
May 28 2035 | 12 years fee payment window open |
Nov 28 2035 | 6 months grace period start (w surcharge) |
May 28 2036 | patent expiry (for year 12) |
May 28 2038 | 2 years to revive unintentionally abandoned end. (for year 12) |