A method and apparatus determine which of a plurality of queue buffers (26, 126, 426) contains a complete packet and transmit a de-queue signal to one of the plurality of queue buffers (26, 126, 426) determined to contain a complete packet.
|
9. A method, comprising:
a queue buffer outputting a tail signal indicating a complete packet entirely contained within the queue buffer;
an arbiter receiving the tail signal;
the arbiter outputting a de-queue signal to the queue buffer from which the tail signal was received; and
initiating dequeuing of the complete packet from the queue buffer, from which the tail signal was received by the arbiter, in response to receiving the dequeue signal;
wherein the arbiter, in response to receiving multiple instances of the tail signal from different queue buffers of a plurality of the queue buffers prior to outputting a dequeue signal in response to any of the multiple instances of the tail signal, indicating that the different queue buffers each concurrently contain a complete packet, outputs the dequeue signal to at least one of: a queue buffer of the plurality of queue buffers having a highest ranked fixed priority, a next in order queue buffer in a round robin ordering of a plurality of queue buffers, or a queue buffer randomly selected from the plurality of queue buffers.
1. An apparatus comprising:
queue buffers, each queue buffer to concurrently receive and contain portions of different packets, each queue buffer to receive and contain portions of packets and to output a tail signal indicating when the portions amount to a complete packet entirely contained in the queue buffer;
an arbiter to receive the tail signal and, in response to receiving the tail signal, to output a de-queue signal to one of the queue buffers from which the tail signal was received to initiate dequeuing of the complete packet entirely contained within said one of the queue buffers; and
wherein the arbiter, in response to receiving multiple instances of the tail signal from different queue buffers of a plurality of the queue buffers prior to outputting a dequeue signal in response any of the multiple instances of the tail signal, indicating that the different queue buffers each concurrently contain a complete packet, is to output the dequeue signal to one of: a next in order queue buffer of the plurality of buffers in a round robin of the plurality of queue buffers, a queue buffer of the plurality of queue buffers having a highest ranked fixed priority, or a queue buffer randomly selected from the plurality of queue buffers.
2. The apparatus of
3. The apparatus of
5. The apparatus of
6. The apparatus of
7. The apparatus of
10. The method of
each of a plurality of queue buffers outputting a packet portion signal indicating a packet portion in the queue buffers; and
the arbiter transmitting a de-queue signal to one of the queue buffers from which a packet portion signal was received if none of the queue buffers contain a complete packet.
11. The method of
13. The method of
14. The method of
15. The method of
16. The method of
|
Fabric switches transmit packets between input and output ports. When multiple packets are addressed to the same port, such packets may be queued. During such queuing, transmission delays may occur, reducing performance of the fabric switch.
In the example illustrated, queue buffer arbitration system 20 comprises a circuit utilized as part of a switching fabric or fabric switch in the form of an output queued networking switch. An output queued networking switch is a switch in which data is held or queued for each output. Queue buffer arbitration system 20 comprises queue buffers 26 and arbiter 28. Queue buffers 26 comprise data or memory storage devices to temporarily store data in the form of flits (or other packet portions) which, together, form a larger complete packet. In one implementation, queue buffers 26 comprise SRAM memory forming at least one first in first out (FIFO). In other implementations, queue buffers 26 may comprise other memory storage devices.
Queue buffers 26 are provided as part of a network switch having inputs and outputs, wherein each output has a queue buffer 26 for each input. Each output's queue buffers 26 hold or queue data, in the form of packets or smaller portions of a packet (such as flits), addressed to the same output (or output port) from multiple inputs.
Arbiter 28 arbitrates an order at which the data held in the queue buffers 26 is released or transmitted through an output port to the output. Once arbiter 28 initiates de-queuing, the transmission or transfer of held data from a queue buffer to the associated output, such de-queuing continues until an entire packet of data has been transferred to the output. If de-queuing is initiated before an entire packet has been completely received at the queue buffer 26, arbiter 28 may have to wait for the queue buffer 26 to receive the missing final flits of the packet being transmitted. This may result in arbiter 28 inserting dead cycles on the output, increasing latency and reducing performance of the switch.
Arbiter 28 arbitrates the order at which data held in queue buffers 26 is released to the associated output in a manner so as to reduce a likelihood that de-queuing of a queue buffer will be initiated before an entire packet has been completely received at the queue buffer 26. As schematically shown in
Although network 100 is schematically illustrated as including a single “layer” of queue buffer systems 20 between inputs 102 and outputs 104, in other implementations, network 100 may comprise multiple “layers” of independent transmission channels (virtual channels) between inputs 102 and outputs 104 in protocol layers. For example, network 100 may include an additional “layer” of additional queue buffer systems 20 to provide two layers of virtual channels. In such an implementation, each queue buffer 26 comprises a first in first out (FIFO) for each virtual channel, wherein arbiter 28 further arbitrates the de-queuing of the FIFOs of each queue buffer 26 on each virtual channel to reduce or avoid conflicts. In one implementation, one virtual channel may be for responses while the other virtual channel is for requests.
As shown by
In operation, by way of example with respect to output 104A and queue buffer arbitration system 20A, data sent concurrently from inputs 102 to output 104A is buffered or queued by buffers 26A-26E, depending upon from which inputs 102 such data is being sent. Arbiter 28 outputs a de-queue signal to one of the queue buffers 26 from which a tail signal has been received indicating that a complete packet is in the particular queue buffer 26. Once the particular queue buffer 26 receives the de-queue signal, the queuing is initiated, wherein the de-queuing continues until the entire packet has been transferred through the port to output 104A. Once the complete packet has been transferred, arbiter 28 once again determines whether any of the queue buffers 26 of system 20A have transmitted a tail signal indicating the presence of another complete packet. Arbiter 28 of system 20A once again outputs a de-queue signal to one of the queue buffers 26 from which a tail signal has been received indicating that a complete packet is in the particular queue buffer 26. This process repeats itself. In instances where more than one of queue buffers 26 of system 20A contain a complete packet, arbiter 28 may utilize a prioritization scheme to select which queue buffer 26 is to receive the de-queue signal. Examples of prioritization schemes that may be utilized by arbiter 28 when selecting a queue buffer 26 include, but are not limited to, first in time, round robin, fixed priority and random.
Although network 100 is illustrated as including five inputs 102, five outputs 104 and an output corresponding five queue buffer arbitration systems 20, in other implementations, network 100 may include a greater or fewer of such inputs 102, outputs 104 and queue buffer arbitration systems 20. As noted above, in some implementations, network 100 may include multiple protocol layers of virtual channels.
As shown by
Arbiter 128 is similar arbiter 28 described above. Arbiter 28 arbitrates the order at which data held in queue buffers 126 is released to the associated output in a manner so as to reduce a likelihood that de-queuing of a queue buffer will be initiated before an entire packet has been completely received at the queue buffer 26. Based upon tail signals received from tail counters 130, arbiter 128 outputs a de-queue signal (DS) to one of the queue buffers 126 from which a tail signal has been received. By favoring those queue buffers 126 that contain a complete packet when determining for which queue buffer 126 de-queuing should be initiated, arbiter 128 is less likely to have to subsequently wait for the queue buffer 126 to receive the missing final packet portions of the packet being transmitted.
As indicated by step 204, upon identifying a buffer queue 26, 126 as containing a complete packet, arbiter 28, 128 transmits a de-queue signal to the particular queue 26, 126 containing the complete packet. As noted above, in instances where more than one of queue buffers 26, 126 for a particular output contain a complete packet, arbiter 28, 128 may utilize a prioritization scheme to select which queue buffer 26, 126 is to receive the de-queue signal. Examples of prioritization schemes that may be utilized arbiter 28, 128 when selecting a queue buffer 26, 126 include, but are not limited to, first in time, round robin, fixed priority and random.
As indicated by step 312, if the particular buffer queue 26, 126 (corresponding to the buffer queue identified with counter p) as a tail count greater than zero indicating that at least one complete packet is contained within the particular queue buffer 26, 126, arbiter 28, 128 outputs a de-queue signal to the particular queue buffer 26, 126. As indicated by step 314, the queuing of the particular queue buffer 26, 126 continues until the complete packet has been de-queued or transferred to the associated output. Once the complete packet has been transferred, the counter p is recommended by one and step 308 and the process continues with arbiter 28, 128 proceeding to the next ordered queue buffer 26, 126.
Arbiter 428 is similar arbiter 128 described above except that arbiter 428 initiates the queuing of low priority queue buffers (those queue buffers that do not presently have a complete packet) in the absence of high priority queue buffers (queue buffers presently having a complete packet). As with arbiter 128, arbiter 428 arbitrates the order at which data held in queue buffers 426 is released to the associated output in a manner based upon tail signals received from tail counters 130. Arbiter 428 is configured to output a de-queue signal (DS) to one of the queue buffers 426 from which a tail signal has been received. By favoring those queue buffers 426 that contain a complete packet when determining for which queue buffer 426 de-queuing should be initiated, arbiter 428 is less likely to have to subsequently wait for the queue buffer 426 to receive the missing final packet portions of the packet being transmitted.
If none of the queue buffers 426 contain a complete packet, arbiter 428 is configured to determine if any of the queue buffers 426 contain at least a portion of a packet available for transmission to the output. Upon identifying a queue buffer 426 containing a portion of a packet (flit in the example illustrated), arbiter 428 outputs a de-queue signal to queue buffer containing or holding the portion of a packet. As a result, time or clock cycles are not wasted waiting for reception of a complete packet by any of the queue buffers 426. In instances where none of the queue buffers 426 presently contain a complete packet, but multiple queue buffers 426 contain a portion of a packet, arbiter 428 may utilize a prioritization scheme to select which queue buffer 426 is to receive the de-queue signal. Examples of prioritization schemes that may be utilized arbiter 428 when selecting a queue buffer 426 include, but are not limited to, first in time, round robin, fixed priority and random.
As indicated by step 508, upon identifying a queue buffer 426 containing a portion of a packet (flit in the example illustrated), arbiter 428 outputs a de-queue signal to queue buffer containing or holding the portion of a packet. In instances where none of the queue buffers 426 presently contain a complete packet, but multiple queue buffers 426 contain a portion of a packet, arbiter 428 may utilize a prioritization scheme to select which queue buffer 426 is to receive the de-queue signal. Examples of prioritization schemes that may be utilized arbiter 428 when selecting a queue buffer 426 include, but are not limited to, first in time, round robin, fixed priority and random.
As shown by
As indicated by block 606, arbiter 428 determines whether any of the requesters or queue buffers 426 contain a complete packet ready for transfer. As indicated by blocks 304-314, if arbiter 428 determines that at least one of queue buffers 426 contains a complete packet as indicated by the reception of at least one tail signal by arbiter 428, arbiter 428 proceeds through queue buffers 426 and a round robin fashion and transmits de-queue signals to those queue buffers 426 having a complete packet. In instances where more than one of queue buffers 426 for a particular output contain a complete packet, arbiter 128 selects which queue buffer 426 is to receive the de-queue signal using the same round-robin prioritization scheme.
If, however, arbiter 428 determines that none of the queue buffers 426 contain a complete packet ready for transfer to the output, arbiter 428 proceeds the flow diagram steps for carrying out “low priority” arbitration. As indicated by block 650, the counter F is incremented by one. As indicated by block 652, arbiter 428 determines whether arbiter 428 has received a flit signal from the particular queue buffer QF indicating that the particular queue buffer QF contains a complete packet. As indicated by block 654, if the particular queue buffer QF does not, arbiter 428 increments the counter value p to look at the next queue buffer 428. As indicated by block 656, if arbiter 428 has looked at each of the queue buffers (when F>n), arbiter 428 begins a process a new by resetting the counter F back to one in step or block 658 and returning to step or block 652. As a result, arbiter 428 reviews the buffer queues 426 in a round-robin fashion and similarly outputs de-queue to the queue buffers 426 in a round-robin sequential order of prioritization.
As indicated by step 662, if the particular buffer queue 426 (corresponding to the buffer queue identified with counter F) has a flit count greater than zero indicating that at least one packet portion or flit is contained within the particular queue buffer 426, arbiter 428, outputs a de-queue signal to the particular queue buffer 426. As indicated by step 664, de-queuing of the particular queue buffer 426 continues until the complete packet has been de-queued or transferred to the associated output. Once the complete packet has been transferred, arbiter 428 returns to step 606, once again determining whether any tail signals TS have been received indicating that at least one of the queue buffers 426 presently contains a complete packet.
In the example illustrated in
In other implementations, arbiter 428 may prioritize between high priority queue buffers and low priority queue buffers in alternative fashions. For example, instead of utilizing a round robin prioritization scheme for both high priority and low priority queue buffers, arbiter 428 may employ a first priority schemes for high priority queue buffers and a second different priority scheme for low priority queue buffers. As indicated by broken line 670, in one implementation, arbiter 428 may employ a fixed prioritization scheme, wherein the queue buffers 426 are ranked in order of priority or importance with the highest priority queue buffer having a counter value for p equal to 1. As shown by line 670, instead of returning to step 308 after a complete packet has been transferred from a high priority queue (per step 314), arbiter 428 may alternatively return to step 304, wherein arbiter 428 returns to the highest priority queue buffers when determining which of the high priority queue buffers to de-queue first.
In other implementations, arbiter 428 may prioritize amongst high priority queue buffers using a somewhat random prioritization scheme. For example, in one implementation, arbiter 428 may employ a hybrid of a round-robin in a random prioritization scheme. In one implementation, block or step 304 may alternatively be replaced with block or step 674 (shown in broken lines). In such an implementation, the beginning of the sequential prioritization of the series of queue buffers is randomly chosen by arbiter 428. In yet other implementations, arbiter 428 may select which of the high priority queue buffers is to receive the de-queue signal based upon when arbiter 428 received the tail count signal from the particular type priority queue buffer. For example, the prioritization of multiple high priority queue buffers may be based on the order in which tail count signals received from the queue buffers. Similar prioritization schemes may be utilized by arbiter 428 when arbitrating between low priority queue buffers.
Queue buffers 426 are described above with respect to system 420. Each queue buffer 426 transmits a tail count signal (TS) to arbiter 728 when the queue buffer 426 contains a complete packet and transmits a flit count signal (FS) to arbiter 728 when the queue buffer contains at least one portion of a packet or flit.
Arbiter 728 comprises a circuit comprising high priority selector circuit 740, low priority selector circuit 742 and de-queue output 744. High priority selector circuit 740 receives tail count signals from buffer queues 426 to identify high-priority buffer queues, to arbitrate between high priority buffer queues 426 (those buffer queues that contain a complete packet) and to transmit a single to de-queue output 744 indicating which of the high party buffer queues should receive a de-queue signal. Similarly, low priority selector circuit 742 receives flit count signals from buffer queues 426 to identify low priority buffer queues, to arbitrate between low priority buffer queues 426 (those buffer queues that contain a portion of a packet) and to transmit a single to de-queue output 744 indicating which of the low party buffer queues should receive a de-queue signal. Output 744 receive signals from selector circuit 740, 742 and gives priority to the signal received from high priority selector circuit 740. In other words, output 744 outputs a de-queue signal to the queue buffer identified by the signal from selector circuit 740 over any output queue identified by the low priority selector circuit 742. If no signal is received from high priority selector circuit 740 or if the signal indicates that there are no high priority buffer queues, output 744 then outputs a de-queue signal to the queue buffer identified by the signal from selector circuit 742. In one implementation, the de-queue signal is transmitted by output 744 to the chosen queue buffer 426 and to multiplexor 730 to effectuate the transfer of the data to the output.
Although the present disclosure has been described with reference to example embodiments, workers skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the claimed subject matter. For example, although different example embodiments may have been described as including one or more features providing one or more benefits, it is contemplated that the described features may be interchanged with one another or alternatively be combined with one another in the described example embodiments or in other alternative embodiments. Because the technology of the present disclosure is relatively complex, not all changes in the technology are foreseeable. The present disclosure described with reference to the example embodiments and set forth in the following claims is manifestly intended to be as broad as possible. For example, unless specifically otherwise noted, the claims reciting a single particular element also encompass a plurality of such particular elements.
Bower, Kenneth S, Gostin, Gary B
Patent | Priority | Assignee | Title |
11855903, | May 02 2018 | VOLKSWAGEN AKTIENGESELLSCHAFT | Method and computer program for sending a data packet, method and computer program for receiving a data packet, communication unit and motor vehicle with communication unit |
Patent | Priority | Assignee | Title |
5689644, | Mar 25 1996 | Fairchild Semiconductor Corporation | Network switch with arbitration sytem |
6314487, | Dec 26 1997 | Electronics and Telecommunications Research Institute | Adaptive routing controller of a crossbar core module used in a crossbar routing switch |
6510138, | Feb 25 1999 | RPX Corporation | Network switch with head of line input buffer queue clearing |
6563831, | Aug 22 1997 | FUTUREWEI TECHNOLOGIES, INC , DBA HUAWEI TECHNOLOGIES USA | Router with virtual channel allocation |
6654343, | Mar 19 2001 | Force 10 Networks, Inc | Method and system for switch fabric flow control |
6674720, | Sep 29 1999 | Hewlett Packard Enterprise Development LP | Age-based network arbitration system and method |
7187679, | Aug 22 1997 | FUTUREWEI TECHNOLOGIES, INC , DBA HUAWEI TECHNOLOGIES USA | Internet switch router |
7519054, | Jan 27 2005 | Intel Corporation | Replication of multicast data packets in a multi-stage switching system |
7583596, | Jun 28 2004 | Juniper Networks, Inc. | Priority scheduling using per-priority memory structures |
8018958, | Jun 23 2009 | Juniper Networks, Inc.; Juniper Networks, Inc | System and method for fair shared de-queue and drop arbitration in a buffer |
8085801, | Aug 08 2009 | Hewlett Packard Enterprise Development LP | Resource arbitration |
20030063618, | |||
20090077567, | |||
20090292575, | |||
20150188850, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Jan 24 2013 | BOWER, KENNETH S | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 036071 | /0498 | |
Jan 24 2013 | GOSTIN, GARY B | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 036071 | /0498 | |
Jan 25 2013 | Hewlett Packard Enterprise Development LP | (assignment on the face of the patent) | / | |||
Oct 27 2015 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Hewlett Packard Enterprise Development LP | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 037079 | /0001 |
Date | Maintenance Fee Events |
Mar 07 2023 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Date | Maintenance Schedule |
Sep 17 2022 | 4 years fee payment window open |
Mar 17 2023 | 6 months grace period start (w surcharge) |
Sep 17 2023 | patent expiry (for year 4) |
Sep 17 2025 | 2 years to revive unintentionally abandoned end. (for year 4) |
Sep 17 2026 | 8 years fee payment window open |
Mar 17 2027 | 6 months grace period start (w surcharge) |
Sep 17 2027 | patent expiry (for year 8) |
Sep 17 2029 | 2 years to revive unintentionally abandoned end. (for year 8) |
Sep 17 2030 | 12 years fee payment window open |
Mar 17 2031 | 6 months grace period start (w surcharge) |
Sep 17 2031 | patent expiry (for year 12) |
Sep 17 2033 | 2 years to revive unintentionally abandoned end. (for year 12) |