In one form, scheduling data migration comprises determining whether the data is likely to be used by an input/output (i/O) device, the data being at a location remote to the i/O device; and scheduling the data for migration from the remote location to a location local to the i/O device in response to determining that the data is likely to be used by the i/O device.
|
1. A method of scheduling data migration comprising:
determining whether the data is likely to be used by an input/output (i/O) device, the data being at a location remote to the i/O device; and
scheduling the data for migration from the remote location to a location local to the i/O device in response to determining that the data is likely to be used by the i/O device, wherein determining that the data is likely to be used by the i/O device comprises monitoring an event associated with translating addresses by a memory management unit local to the i/O device,
wherein monitoring the event associated with translating addresses by the memory management unit local to the i/O device includes one or more of: monitoring data having a memory address in a peripheral page service request event log file, monitoring data having a memory address in an error event log file, and monitoring data having a memory address in a command issued by the i/O device to invalidate the data.
7. A data processing node comprising:
a data processor; and
an input/output memory management unit (IOMMU) coupled to the data processor and adapted to associate logical memory pages to corresponding physical pages wherein each physical page may be located in either a local memory device associated with the data processing node or in a remote memory device associated with another processing node, wherein the IOMMU is adapted to determine whether data in a physical page in the remote memory device is likely to be used by an input/output (i/O) device coupled to and associated with the processing node, and in response to determining that the data is likely to be used by the i/O device, to schedule migration of the data in the physical page at the remote memory device to a physical page at the local memory device,
wherein the IOMMU determines that data is likely to be used by the i/O device by monitoring the output of at least one address translation circuit associated with the i/O device, and
wherein the IOMMU monitors the output of the at least one address translation circuit associated with the i/O device by monitoring a memory address in a log file.
14. A data processing node comprising:
a data processor; and
an input/output memory management unit (IOMMU) coupled to the data processor and adapted to associate logical memory pages to corresponding physical pages wherein each physical page may be located in either a local memory device associated with the data processing node or in a remote memory device associated with another processing node, wherein the IOMMU is adapted to determine whether data in a physical page in the remote memory device is likely to be used by an input/output (i/O) device coupled to and associated with the processing node, and in response to determining that the data is likely to be used by the i/O device, to schedule migration of the data in the physical page at the remote memory device to a physical page at the local memory device,
wherein the IOMMU determines that data is likely to be used by the i/O device by monitoring the output of at least one address translation circuit associated with the i/O device, and
wherein the IOMMU monitors the output of the at least one address translation circuit associated with the i/O device by monitoring a memory address in a command issued by the i/O device to invalidate the data.
2. The method of
determining, in response to determining that the data is likely to be used by the i/O device, whether the data is to be migrated; and
scheduling the data for migration in response to determining that the data is to be migrated.
3. The method of
5. The method of
6. The method of
8. The data processing node of
9. The data processing node of
10. The data processing node of
11. The data processing node of
12. The data processing node of
13. The data processing node of
|
This disclosure relates generally to data processing systems, and more specifically to scheduling of data migration in data processing systems.
With the advent of ubiquitous information gathering and producing mobile devices such as digital cameras, smart phones, tablets etc., the world has been experiencing an explosion in the amount of data being gathered. To process this huge amount of data (also known as Big Data), massively parallel software programs running on tens, hundreds, or even thousands of servers (also known as Big Compute) are being used. Due to this new Big Data and Big Compute paradigm, it is no longer enough to deliver relevant data to where processing is to occur, the data has to also be processed quickly in order to retain any business value.
One method that has been used to tackle this ever-increasing demand for data processing has been to rethink the traditional way of designing computing systems. For example, instead of having central processing units (CPUs) primarily process data, various other processing devices located throughout a computing system have been configured to process data. This configuration has led to a decrease in data transfer overhead as well as to a reduction in latency.
Further, computing systems have been designed based on a heterogeneous system architecture (HSA). HSA is a computer architecture that integrates CPUs and graphics processor units (GPUs) onto a single chip called an accelerated processing unit (APU). CPUs and GPUs in an APU use a common bus and share tasks and system memory. To facilitate the sharing of tasks between the integrated CPUs and GPUs, a unified memory address space is used. The unified memory address space is supported by specified memory management units (MMUs). The MMUs provide virtual to physical memory address translations as well as protection functionalities for the integrated CPUs and GPUs.
To provide virtual to physical memory address translations as well as protection functionalities to input/output (I/O) devices and/or the various other processing devices located throughout the computing system, input/output memory management units (IOMMUs) are used. Just as in the case of the MMUs, the IOMMUs also support the unified memory address space.
In certain computing environments, two or more HSA systems may be combined together to provide more computing power. In such cases, different system memories may be local to different HSA systems. Consequently, the time needed for a device (e.g., an I/O device or one of the various other processing devices located throughout the computing system) to perform a memory access is dependent on the location of the memory system relative to the device. One of HSA's aims, however, is to reduce communication latency between CPUs, GPUs and the various other processing elements that are located throughout the computing system (note that CPUs, GPUs and the various other processing elements may generally be referred to as compute devices), and to make the compute devices more compatible to each other from a programmer's perspective.
One method of reducing communication latency between the compute devices is to ensure that data that is needed by a compute device is loaded into a memory system that is local to the HSA system to which the compute device is attached. Hence, in cases where the data is in a remote memory system, the data may have to be migrated to a local memory system to reduce latency.
However, in order for the compute devices to be compatible to each other from a programmer's perspective, the programmer should not have to plan to move data from one memory system to another.
In the following description, the use of the same reference numerals in different drawings indicates similar or identical items. Unless otherwise noted, the word “coupled” and its associated verb forms include both direct connection and indirect electrical connection by means known in the art, and unless otherwise noted any description of direct connection implies alternate embodiments using suitable forms of indirect electrical connection as well.
The present disclosure provides a method of dynamically scheduling memory page migration in a computing system. The method determines whether data is likely to be used by an input/output (I/O) device. In response to determining that the data is likely to be used by the I/O device and if the data is at a location remote to the I/O device, then the data may be scheduled for migration from the remote location to a location local to the I/O device.
With reference now to the figures,
The one or more CPU cores 210 are connected to at least one memory management unit (MMU) 212 and to at least one cache system 214. Likewise, the one or more GPU cores 220 are connected to at least one MMU 222 and to at least one cache system 224. The MMUs 212 and 222 provide virtual to physical memory address translations as well as protection functionalities for the one or more CPU cores 210 and GPU cores 220, respectively. Both the MMUs 212 and 222 support a unified memory address space allowing for the integration of the one or more CPU cores 210 and GPU cores 220 into one processing chip in accordance with a heterogeneous system architecture (HSA).
Returning to
For clarity purposes, each APU is shown connected to an MMU (see APUs 102, 104, 122 and 124 connected to MMUs 106, 108, 126 and 128, respectively). In actuality, each of MMUs 106, 108, 126 and 128 may be located within the respective APU to which it is connected and represents both MMUs 212 and 222 of
APUs 102 and 104 form a non-uniform memory access (NUMA) node 112 and APUs 122 and 124 form a NUMA node 132. NUMA is a method of configuring a cluster of processors in a multiprocessing system such that the processors may share memory locally, thereby improving performance. The APUs in NUMA node 112 and those in NUMA node 132 share memory devices 114 and 134 over link 118.
Attached to APUs 102 and 104, over link 116, is an I/O hub 140. Likewise, attached to APUs 122 and 124 over link 136 is an I/O hub 160. Through I/O hubs 140 and 160, various peripheral or I/O devices may be connected to the server 100. For example, storage device 150, which may include hard drives, NVRAMs, flash drives etc., and digital signal processing (DSP) module 152 may be attached to the server 100 through I/O hub 140 via a peripheral component interconnect express (PCIe) bus 148. Likewise, storage device 170, network interface card (NIC) 172 and discrete GPU (dGPU) 174 may be added to the server 100 through I/O hub 160 via PCIe bus 168. Note that henceforth, the terms “I/O device” and “peripheral device” will be used interchangeably.
The storage devices 150 and 170 may contain user data, at least one operating system (OS), a hypervisor in cases where the server 100 is logically partitioned, as well as software applications that may be needed by the server 100 to perform any particular task. In operation, the OS, hypervisor, firmware applications and the software application needed by the server 100 to perform a task may all be loaded into either memory device 114 or 134 or partly into both memory devices 114 and 134.
The server 100 may interact with other computing systems via NIC 172 over network 180. Network 180 may include connections, such as wire, wireless communication links, fiber optic cables, etc. Further, network 180 may include the Internet or may be implemented as a number of different types of networks, such as for example, an intranet, a local area network (LAN), a wide area network (WAN), a cellular phone network etc.
DSP module 152 is used to measure, filter and/or compress continuous real-world analog signals. Real-world analog signals may include audio signals, video signals, radio signals etc. The real-world analog signals may be obtained through a microphone (not shown), a camera (not shown) radio or television antennae (not shown) etc. attached to the server 100 or received from another computing system through NIC 172. The first step that may be performed by DSP module 152 is to convert the signals from an analog to a digital form, by sampling and then digitizing the signals using an analog-to-digital converter (ADC) if the signals are not already in a digital form. The second step is to process the digitized signals.
A dGPU, such as dGPU 174, is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display device (not shown).
Each I/O hub includes an IOMMU (see IOMMUs 142 and 162). An IOMMU is a system function that translates addresses used in direct memory access (DMA) transactions, disallow memory accesses to peripheral devices with improper access permissions, and remaps peripheral interrupts. As in the case of MMUs 212 and 222 of
IOMMU 142 performs address translations on behalf of I/O devices connected to the I/O hub 140 while IOMMU 162 performs address translations on behalf of I/O devices connected to I/O hub 160. The IOMMUs 142 and 162 are allowed to cache page table and device table contents in translation lookaside buffers or TLBs (see TLBs 144 and 164) to speed up future address translation processes.
Note that the depicted example in
In operation, the I/O devices attached to the server 100 may access data loaded in memory pages in either memory device 114 or 134. As an example, NIC 172 may access data loaded in memory devices 114 and 134. To access data stored in a memory device, NIC 172 issues a request that includes a device address (i.e., a virtual address). The request goes to IOMMU 162 where the device address is translated into a corresponding physical memory address and stored into TLB 164 if there is not already an entry in TLB 164 corresponding to the device address. If there is an entry in TLB 164 corresponding to the device address, the entry is used to obtain the corresponding physical memory address. If, based on the physical memory address, the data is in memory device 114, the request, along with the physical memory address, will go from IOMMU 162 to node controller 130 and node controller 110 before arriving at memory device 114. By contrast, if the data is in memory device 134, the request will go from IOMMU 162 to only node controller 130 before arriving at memory device 134. Thus, NIC 172 may access data in memory device 134 faster than data in memory device 114. Therefore, moving data that is to be accessed by NIC 172 from memory device 114 to memory device 134 may reduce latency and data overhead associated with the data access.
Page Migration for IOMMU Enhanced Devices by Tomsz Stanislawski, dated Aug. 20, 2013, provides a method by which files are migrated from one memory device to another. According to the reference, every mapping in a TLB includes a radix tree to track different parts of a file that are already in RAM (memory). Using the radix tree, a determination may be made as to the location of (i.e., which memory system contains) the different parts of the file as well as to whether to migrate the different parts of the file from one memory device to another. Based on the location, some or all the parts of the file may be moved from one memory system to another.
Likewise, each entry in TLBs 144 and 164 contains a radix tree that may be used by the IOMMUs 142 and 162, respectively, to determine the memory device in which memory pages containing a file are located. Based on the location of the memory pages, the pages may be moved from one memory system to another.
However, not all memory pages should be moved. For example, certain regions in system memory are reserved for memory-based queues for exchanging commands and status information between an IOMMU and an APU. Consequently, memory pages in these regions of memory should not be moved.
In addition, in order to perform permission checks, interrupt remapping, and address translation for a memory access requestor, an IOMMU uses three tables. These three tables are shared by both MMUs and IOMMUs of a system. Because the MMUs also use these tables, memory pages containing the tables should not be moved.
Further, memory pages containing log files should preferably not be moved as doing so may result in a loss of data. For example, according to IOMMU Architectural Specification, Rev. 2.0, published by Advanced Micro Devices (AMD) on Mar. 24, 2011, an IOMMU may detect and report several kinds of events that may arise due to unusual hardware or software behavior. When the IOMMU detects an event of any kind and event logging is enabled, the IOMMU writes an appropriate event entry into an event log located in system memory. Events detected by the IOMMU include I/O page faults as well as hardware memory errors detected when walking I/O page tables. The IOMMU can also optionally write to peripheral page service request queues and to guest virtual advanced programmable interrupt controller (APIC) logs in system memory. Moving log locations in system memory can introduce errors and finish unsuccessfully. When memory errors prevent recording of information in the in-system memory event log, critical events are reported in the hardware error registers. The hardware error registers can quickly fill up and overflow in such cases. This then may result in a loss of event information. Consequently, moving log files, while generally allowable, is not recommended due to the possibility of information loss.
All other memory pages may be moved if doing so may reduce data transfer overhead and latency. According to the present disclosure, a scheduler of a servicing IOMMU (see schedulers 146 and 166 of IOMMUs 142 and 162, respectively) may schedule memory pages for migration when applicable.
In order to schedule a memory page for migration, however, the page has to first be identified as a page that an I/O device attached to a servicing IOMMU may soon want to access. Below, three methods are disclosed that may be used to identify pages that an I/O device may soon want to access and thus may be scheduled for migration. Note that, although only three methods are disclosed, the present disclosure is not limited to only those three methods. For example, more or less than three methods may be used. Further, methods that are different from the ones disclosed may be used. Thus, the three methods are only used for illustration purposes.
A first method that may be used to identify pages that an I/O device may soon want to access is to use translation tables used by a servicing IOMMU. A first example of the first method includes using the peripheral page service request (PPR) logging (Section 3.5 of the IOMMU Architectural Specification) feature of the IOMMU. When an I/O device needs memory page services, the I/O device issues a special bus request to the IOMMU. If supported, the IOMMU converts the special bus request to a PAGE_SERVICE_REQUEST format. When the peripheral page service is enabled for the device, the IOMMU creates a PAGE_SERVICE_REQUEST entry in the PPR log buffer. PAGE_SERVICE_REQUEST PPR Log Buffer Entry format outlined in Table 67 of the IOMMU Architectural Specification contains the device virtual address of the page that the device was attempting to access. The virtual address or its correspondent physical memory address may be used to identify the page that may be scheduled for migration.
A second example of the first method includes using the Event Logging feature of the IOMMU. According to Section 3.4 of the IOMMU Architectural Specification, an IOMMU reports events to a host software by means of a shared circular buffer in system memory. Although this type of logging primarily reports errors related to page faults (see Table 45: IO_PAGE_FAULT Event Types, for a translation, the IOMMU returns response with data and no event log entry is created), it is still useful for determining a memory range of interest to the peripheral device, because Table 56: IO_PAGE_FAULT Event Log Buffer Entry Fields contains the address field which has the device virtual address that the peripheral device was attempting to access. The virtual address or its correspondent physical memory address in Table 56 may be used to identify the page that may be scheduled for migration.
A third example of the first method is to use ill-formed PCIe page request interface (PRI) requests. Certain types of ill-formed PCIe PRI requests are logged in the PPR request log with RZ=1 so that system software may attempt recovery (e.g., reserved bit error in FIG. 38 of the IOMMU Architectural Specification). When the peripheral page service is not enabled, the IOMMU creates an entry in the IOMMU event log to report the error (see Section 3.4.9 [INVALID_PPR_REQUEST Event]). The address in the event log entry may be used to identify the page that may be scheduled for migration.
A fourth example of the first method includes using the performance counter monitoring feature of the IOMMU. MMIO Offset [40-7F][0-F]00h IOMMU Counter Register counts events as programmed by IOMMU Counter Source Register [MMIO Offset [40-7F] [0-F]08h] and IOMMU PASID Match Register [MMIO Offset [40-7F][0-F]10h]. The value of this register may be used to determine the most memory-bound peripheral devices for which memory scheduling is worthwhile. In one embodiment, that can be done by setting the event counter source to type “5” from Table 71 of the IOMMU Architectural Specification, which corresponds to total peripheral memory operations translated. In this case, the page addresses that are accessed the most by these I/O devices may be scheduled for migration.
A fifth example of the first method includes using the device specific monitoring feature of the IOMMU. I/O translation buffer (IOTLB) on a peripheral device works in concert with an IOMMU. An I/O device with IOTLB support can cooperate with the IOMMU to maintain the I/O device's own cache of address translations. IOTLB-capable I/O devices contain private TLBs tailored for the devices own needs, creating a scalable distributed system of TLBs. Software can configure where the address translation is cached. For example, the address translation may be cached at the IOTLBs of the devices and/or at the IOMMU TLBs. The hardware allows for “exclusive” (IOMMU doesn't cache, but does walk tables) or “inclusive” (IOMMU caches). Even in the “inclusive” case, the IOMMU and IOTLB can have different ejection rules. Thus, the inclusivity is not strict and is completely unenforced. Depending on the capabilities of a particular device, certain logging and performance counter options may be made available, which can help identifying relevant memory pages that may be scheduled for migration.
A second method that may be used to identify pages that an I/O device may soon want to access and thus may be scheduled for migration is to take advantage of certain address translation “hint” commands. According to the IOMMU specification, special “hint” commands may be issued by peripheral devices in an attempt to preemptively load/clear address translation information. These commands can be injected into the IOMMU command queue. They can further give schedulers the addresses of pages to be migrated closer to an IOMMU (in anticipation of their later use) or away from near-IOMMU's congested memory (once the pages are no longer needed).
The “hint” commands may include a PREFETCH_IOMMU_PAGES command. This command is a hint to the IOMMU that the associated translation records will be needed relatively soon and that the IOMMU should execute a page table walk to fetch and store the translation information into the IOMMU's TLBs. If an entry is already in a TLB, the IOMMU may adjust a least recently used (LRU) algorithm or other control tags to lengthen cache residency. The addresses in the translation information may be used to identify the pages that should be scheduled for migration.
Another “hint” command may include INVALIDATE_IOTLB_PAGES. A software module may issue an INVALIDATE_IOTLB_PAGES command to cause the IOMMU to generate an invalidation request to a peripheral device (see Section 3.3.4 [INVALIDATE_IOTLB_PAGES] of the IOMMU Architectural Specification) in order for the device to invalidate the relevant cached translation entry. These requests are usually issued by system software after updating a table entry that can be cached, so the peripheral device IOTLBs can also be invalidated (see Section 3.2.1 of the IOMMU Architectural Specification). The addresses in the cached translation entry may be used to identify the pages that are to be scheduled for migration.
A third method that may be used to identify pages that an I/O device may soon want to access and thus may be scheduled for migration is to use explicit software assistance in memory scheduling. For instance, a software module may include code for utilizing a particular peripheral device. While using the peripheral device, the software module has to share the virtual memory maintenance with an IOMMU. At the time of executing the maintenance functions, the software can optionally notify the memory scheduler of the upcoming memory accesses, or perform the page migration using common NUMA API.
One example of the third method includes using “accessed” and “dirty” bits. The IOMMU typically sets the “accessed” and “dirty” bits of the memory pages in a single locked operation. However, the IOMMU never clears the “accessed” or “dirty” bits. The software module is responsible to do so. Because the IOMMU is allowed to cache these bits, the software must explicitly issue invalidation commands when clearing the bits in page table entry (PTE). According to Section 3.2.7.5 of the IOMMU Architectural Specification, to clear an accessed bit in a descriptor, the software module must modify the PTE in memory and then invalidate the page table entry in the IOMMU for all devices using the translation table as is outlined in the pseudo-code in Section 6.2 [Clear Accessed Bit] of the IOMMU Architectural Specification. Thus, the pages containing the PTE may be scheduled for migration as the software module will soon have to access those pages to modify the PTE.
Another example of the third method includes using application programs that can explicitly migrate memory pages to a relevant IOMMU. Specifically, an application program can explicitly migrate memory pages to a relevant IOMMU if the working memory set required by a peripheral device is limited. This is unique to near-IOMMU memory scheduling, as opposed to the general MMU memory scheduling on a NUMA system. This is because MMUs service compute cores that are often homogeneous or do not differ significantly. Thus, the application can be executed on any core. IOMMUs, on the other hand, service significantly different devices (e.g. a network card, an audio module and an APU). Because of this heterogeneity, system software can find out upfront what peripheral device will service the request. Based on the location of the peripheral device and using system tools, the location of the memory can be easily obtained. Depending on the location of the peripheral device, the memory pages may be scheduled for migration.
Note that scheduling a memory page for migration entails maintaining the address of the page in a TLB. Maintaining an inordinate number of page addresses in a TLB may result in TLB “thrashing”. TLB “thrashing” occurs when there is insufficient locality of reference (i.e., due to a lack of space in the TLB, previous address translations are not effectively held within the TLB because the IOMMU constantly replaces previous address translations with newer ones). “IOMMU: Strategies for Mitigating the IOTLB Bottleneck” by Amit Nadav et al., Proceedings of the 2010 International Conference on Computer Architecture, ISCA'10, Berlin, Heidelberg, (2012), pages 256-274 (2010), the disclosure of which is herein incorporated by reference, proposes a method by which both system software and hardware enhancements are used to reduce input/output translation lookaside buffer (IOTLB) miss rate and accelerate address resolution. This method may be used to prevent TLB “thrashing” in accordance with the present disclosure.
Thus, a system as disclosed herein uses profiling of memory accesses from peripheral devices to dynamically schedule migration of data to those memory devices that are aligned with an IOMMU that services a memory access requestor. Having the data closer to the servicing IOMMU, and thus to the requestor, decreases the time between data delivery to processing units and subsequent analysis of the data, as well as reduces the load on on-chip systems.
If there are not any memory pages that are likely to be used by an I/O device, the scheduler waits at box 402. If, on the other hand, there is one or more memory pages that are likely to be used by an I/O device, at box 404, the scheduler obtains the address of the page if there is only one page or the address of the first page if there is more than one page. Upon obtaining the address of the page or of the first page, the scheduler determines at decision box 406 whether the page is to be migrated. As explained earlier, pages that are to be migrated are pages that are in a memory system in a different NUMA node than a servicing IOMMU to which the I/O device is attached. The radix table in the TLB associated with the servicing IOMMU may be used to determine whether the data or file is loaded in memory pages that are in a memory system in a different NUMA node than the IOMMU to which the I/O device is attached.
However and as previously explained, although a page may be in a different NUMA node than the servicing IOMMU, if the page is in a region in system memory reserved for memory-based queues for exchanging commands and status information between an IOMMU and an APU, the page may not be migrated. Further, pages that are used by both an IOMMU and an MMU may not be moved and neither are pages that contain event log files.
If the page is not to be moved, the process jumps to box 412. If the page can be moved, at box 410 the scheduler schedules the page for migration and at decision box 412 checks to see whether there are other pages to schedule for migration. If so, the scheduler obtains the address of the next page at box 414 and the process returns to box 406. If there are not anymore pages that are to be scheduled for migration, the process returns to box 402. The process ends when the server 100 is turned off or is rebooted.
Some or all of the software components may be stored in a non-transitory computer readable storage medium for execution by at least one processor. In various embodiments, the non-transitory computer readable storage medium includes a magnetic or optical disk storage device, solid-state storage devices such as FLASH memory, or other non-volatile memory device or devices. The computer readable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted and/or executable by one or more processors.
The circuits of
While particular embodiments have been described, various modifications to these embodiments will be apparent to those skilled in the art. For example, although schedulers 146 and 166 are shown as being part of IOMMUs 142 and 162, respectively, they need not be. Schedulers 146 and 166 may be part of the OS running on server 100, or part of a hypervisor etc. Further, the process running in schedulers 146 and 166 may be implemented in hardware, software or firmware. Thus, the depiction of schedulers 146 and 166 in IOMMUs 142 and 162 is for illustration purposes only.
Accordingly, it is intended by the appended claims to cover all modifications of the disclosed embodiments that fall within the scope of the disclosed embodiments.
Kegel, Andrew G., Blagodurov, Sergey
Patent | Priority | Assignee | Title |
10331583, | Sep 26 2013 | Intel Corporation | Executing distributed memory operations using processing elements connected by distributed channels |
10380063, | Sep 30 2017 | Intel Corporation | Processors, methods, and systems with a configurable spatial accelerator having a sequencer dataflow operator |
10387319, | Jul 01 2017 | Intel Corporation | Processors, methods, and systems for a configurable spatial accelerator with memory system performance, power reduction, and atomics support features |
10416999, | Dec 30 2016 | Intel Corporation | Processors, methods, and systems with a configurable spatial accelerator |
10417175, | Dec 30 2017 | Intel Corporation | Apparatus, methods, and systems for memory consistency in a configurable spatial accelerator |
10445098, | Sep 30 2017 | Intel Corporation | Processors and methods for privileged configuration in a spatial array |
10445234, | Jul 01 2017 | Intel Corporation | Processors, methods, and systems for a configurable spatial accelerator with transactional and replay features |
10445250, | Dec 30 2017 | Intel Corporation | Apparatus, methods, and systems with a configurable spatial accelerator |
10445451, | Jul 01 2017 | Intel Corporation | Processors, methods, and systems for a configurable spatial accelerator with performance, correctness, and power reduction features |
10459866, | Jun 30 2018 | Intel Corporation | Apparatuses, methods, and systems for integrated control and data processing in a configurable spatial accelerator |
10467183, | Jul 01 2017 | Intel Corporation | Processors and methods for pipelined runtime services in a spatial array |
10469397, | Jul 01 2017 | Intel Corporation | Processors and methods with configurable network-based dataflow operator circuits |
10474375, | Dec 30 2016 | Intel Corporation | Runtime address disambiguation in acceleration hardware |
10496574, | Sep 28 2017 | Intel Corporation | Processors, methods, and systems for a memory fence in a configurable spatial accelerator |
10515046, | Jul 01 2017 | Intel Corporation | Processors, methods, and systems with a configurable spatial accelerator |
10515049, | Jul 01 2017 | TAHOE RESEARCH, LTD | Memory circuits and methods for distributed memory hazard detection and error recovery |
10558575, | Dec 30 2016 | Intel Corporation | Processors, methods, and systems with a configurable spatial accelerator |
10564980, | Apr 03 2018 | Intel Corporation | Apparatus, methods, and systems for conditional queues in a configurable spatial accelerator |
10565134, | Dec 30 2017 | Intel Corporation | Apparatus, methods, and systems for multicast in a configurable spatial accelerator |
10572376, | Dec 30 2016 | Intel Corporation | Memory ordering in acceleration hardware |
10678724, | Dec 29 2018 | Intel Corporation | Apparatuses, methods, and systems for in-network storage in a configurable spatial accelerator |
10691365, | Jan 30 2019 | Red Hat, Inc.; Red Hat, Inc | Dynamic memory locality for guest memory |
10817291, | Mar 30 2019 | Intel Corporation | Apparatuses, methods, and systems for swizzle operations in a configurable spatial accelerator |
10853073, | Jun 30 2018 | Intel Corporation | Apparatuses, methods, and systems for conditional operations in a configurable spatial accelerator |
10853276, | Sep 26 2013 | Intel Corporation | Executing distributed memory operations using processing elements connected by distributed channels |
10891240, | Jun 30 2018 | Intel Corporation | Apparatus, methods, and systems for low latency communication in a configurable spatial accelerator |
10915471, | Mar 30 2019 | Intel Corporation | Apparatuses, methods, and systems for memory interface circuit allocation in a configurable spatial accelerator |
10942737, | Dec 29 2011 | Intel Corporation | Method, device and system for control signalling in a data path module of a data stream processing engine |
10965536, | Mar 30 2019 | Intel Corporation | Methods and apparatus to insert buffers in a dataflow graph |
11029927, | Mar 30 2019 | Intel Corporation | Methods and apparatus to detect and annotate backedges in a dataflow graph |
11037050, | Jun 29 2019 | Intel Corporation | Apparatuses, methods, and systems for memory interface circuit arbitration in a configurable spatial accelerator |
11086816, | Sep 28 2017 | Intel Corporation | Processors, methods, and systems for debugging a configurable spatial accelerator |
11200186, | Jun 30 2018 | Intel Corporation | Apparatuses, methods, and systems for operations in a configurable spatial accelerator |
11307873, | Apr 03 2018 | Intel Corporation | Apparatus, methods, and systems for unstructured data flow in a configurable spatial accelerator with predicate propagation and merging |
11593295, | Jun 30 2018 | Intel Corporation | Apparatuses, methods, and systems for operations in a configurable spatial accelerator |
11693633, | Mar 30 2019 | Intel Corporation | Methods and apparatus to detect and annotate backedges in a dataflow graph |
11750504, | May 23 2019 | Cray Inc | Method and system for providing network egress fairness between applications |
11757763, | May 23 2019 | Cray Inc | System and method for facilitating efficient host memory access from a network interface controller (NIC) |
11757764, | May 23 2019 | Cray Inc | Optimized adaptive routing to reduce number of hops |
11765074, | May 23 2019 | Cray Inc | System and method for facilitating hybrid message matching in a network interface controller (NIC) |
11777843, | May 23 2019 | Cray Inc | System and method for facilitating data-driven intelligent network |
11784920, | May 23 2019 | Cray Inc | Algorithms for use of load information from neighboring nodes in adaptive routing |
11792114, | May 23 2019 | Cray Inc | System and method for facilitating efficient management of non-idempotent operations in a network interface controller (NIC) |
11799764, | May 23 2019 | Cray Inc | System and method for facilitating efficient packet injection into an output buffer in a network interface controller (NIC) |
11818037, | May 23 2019 | Cray Inc | Switch device for facilitating switching in data-driven intelligent network |
11848859, | May 23 2019 | Cray Inc | System and method for facilitating on-demand paging in a network interface controller (NIC) |
11855881, | May 23 2019 | Cray Inc | System and method for facilitating efficient packet forwarding using a message state table in a network interface controller (NIC) |
11863431, | May 23 2019 | Cray Inc | System and method for facilitating fine-grain flow control in a network interface controller (NIC) |
11876701, | May 23 2019 | Cray Inc | System and method for facilitating operation management in a network interface controller (NIC) for accelerators |
11876702, | May 23 2019 | Cray Inc | System and method for facilitating efficient address translation in a network interface controller (NIC) |
11882025, | May 23 2019 | Cray Inc | System and method for facilitating efficient message matching in a network interface controller (NIC) |
11899596, | May 23 2019 | Cray Inc | System and method for facilitating dynamic command management in a network interface controller (NIC) |
11902150, | May 23 2019 | Cray Inc | Systems and methods for adaptive routing in the presence of persistent flows |
11907713, | Dec 28 2019 | Intel Corporation | Apparatuses, methods, and systems for fused operations using sign modification in a processing element of a configurable spatial accelerator |
11916781, | May 23 2019 | Cray Inc | System and method for facilitating efficient utilization of an output buffer in a network interface controller (NIC) |
11916782, | May 23 2019 | Cray Inc | System and method for facilitating global fairness in a network |
11929919, | May 23 2019 | Cray Inc | System and method for facilitating self-managing reduction engines |
11962490, | May 23 2019 | Cray Inc | Systems and methods for per traffic class routing |
11968116, | May 23 2019 | Cray Inc | Method and system for facilitating lossy dropping and ECN marking |
11973685, | May 23 2019 | Hewlett Packard Enterprise Development LP | Fat tree adaptive routing |
11985060, | May 23 2019 | Cray Inc | Dragonfly routing with incomplete group connectivity |
11991072, | May 23 2019 | Cray Inc | System and method for facilitating efficient event notification management for a network interface controller (NIC) |
12056401, | Mar 28 2019 | Nutanix, Inc | Machine learning for local caching of remote data in a clustered computing environment |
12058032, | May 23 2019 | Cray Inc | Weighting routing |
12058033, | May 23 2019 | Cray Inc | Method and system for providing network ingress fairness between applications |
ER1450, | |||
ER6305, | |||
ER8357, | |||
ER9647, |
Patent | Priority | Assignee | Title |
6952821, | Aug 19 2002 | SAMSUNG ELECTRONICS CO , LTD | Method and system for memory management optimization |
7155572, | Jan 27 2003 | Advanced Micro Devices, Inc. | Method and apparatus for injecting write data into a cache |
20060112228, | |||
20080307178, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Feb 12 2015 | BLAGODUROV, SERGEY | Advanced Micro Devices, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 035008 | /0552 | |
Feb 12 2015 | KEGEL, ANDREW G | Advanced Micro Devices, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 035008 | /0552 | |
Feb 23 2015 | Advanced Micro Devices, Inc. | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Feb 08 2017 | ASPN: Payor Number Assigned. |
Sep 03 2020 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Sep 02 2024 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Date | Maintenance Schedule |
Mar 14 2020 | 4 years fee payment window open |
Sep 14 2020 | 6 months grace period start (w surcharge) |
Mar 14 2021 | patent expiry (for year 4) |
Mar 14 2023 | 2 years to revive unintentionally abandoned end. (for year 4) |
Mar 14 2024 | 8 years fee payment window open |
Sep 14 2024 | 6 months grace period start (w surcharge) |
Mar 14 2025 | patent expiry (for year 8) |
Mar 14 2027 | 2 years to revive unintentionally abandoned end. (for year 8) |
Mar 14 2028 | 12 years fee payment window open |
Sep 14 2028 | 6 months grace period start (w surcharge) |
Mar 14 2029 | patent expiry (for year 12) |
Mar 14 2031 | 2 years to revive unintentionally abandoned end. (for year 12) |