In one embodiment, the present invention includes a method for recording a time stamp counter (tsc) value of a first tsc counter of a processor before a system suspension, accessing the stored tsc value after the system suspension, and directly updating a thread offset value associated with a first thread executing on a first core of the processor with the stored tsc value, without performing a synchronization between a plurality of cores of the processor. Other embodiments are described and claimed.
|
15. A processor comprising:
a first time stamp count (tsc) counter to count according to a bus clock frequency; and
a first core including a second tsc counter to count according to a core clock frequency, and a first thread offset storage to store a thread offset value for a first thread to be executed on the first core, wherein the thread offset value is to be directly updated with a value of the first tsc counter recorded responsive to an indication of a suspension of a system including the processor and recorded prior to the suspension of the system.
6. A method comprising:
responsive to a request from a processor for a system suspension, storing by a controller of the processor, a time stamp counter (tsc) value of an uncore tsc counter of the processor in a memory before the system suspension;
accessing, by the processor, a stored tsc value from the memory after the system suspension; and
executing program instructions of a first thread on a first core of the processor including instructions to update a first thread offset value that is associated with the first thread, with the stored tsc value without performing a synchronization between a plurality of cores of the processor.
20. A system comprising:
a processor having a first time stamp count (tsc) counter to count according to a bus clock frequency, a first core including a first thread offset storage to store a first thread offset value for a first thread and a second tsc counter to count according to a core clock frequency, and a second core including a second thread offset storage to store a second thread offset value for a second thread and a third tsc counter to count according to the core clock frequency, wherein the first and second thread offset values are to be directly updated with a value of the first tsc counter recorded in response to an indication of a suspension of the system, wherein direct update of the first and second thread offset values can occur at different times; and
a dynamic random access memory (DRAM) coupled to the processor.
1. An apparatus comprising:
a first logic including a first time stamp count (tsc) counter to count according to a bus clock frequency, a first core snapshot storage to store a value of the first tsc counter, and circuitry including a snapshot controller coupled to the first core snapshot storage to initiate storage of the value of the first tsc counter in the first core snapshot storage responsive to an indication of a system suspension; and
a first core including a second tsc counter to count according to a core clock frequency, and a first thread offset storage to store a first thread offset value for a first thread to be executed in the first core, wherein the first thread offset value is to be directly updated with a synchronization value corresponding to the value of the first tsc counter recorded prior to occurrence of the system suspension.
2. The apparatus of
3. The apparatus of
4. The apparatus of
5. The apparatus of
7. The method of
8. The method of
9. The method of
10. The method of
11. The method of
13. The method of
14. The method of
16. The processor of
17. The processor of
18. The processor of
19. The processor
21. The system of
22. The system of
23. The system of
24. The system of
|
Many processors include a time stamp count (TSC) counter which is typically implemented as a counter that increments at a fixed rate. Some known TSC counters are clocked at a bus clock rate and a multiplier value is added each bus clock cycle. That is, if the bus clock is at 100 megahertz (MHz) and the processor is stamped for operation at 2000 MHz, a value of 20 is added to the TSC count every bus clock. In general, for a multicore processor there is a single hardware TSC counter for the number of cores present in the processor. The resolved value of the TSC for a given core is this TSC counter value plus a core offset.
Keeping TSC values for each core synchronized in a multi-socket, multi-core and multi-threaded system can be important for software. Certain software uses the TSC value as a stamp to record the order of events, e.g., in a database transaction. Having time going backwards between two processors can cause adverse results, and even can be fatal to some operations. In addition, there can be great complexity involved in enabling a TSC value to be synchronized between multiple cores.
When software is to update the TSC value for a given core, the processor reads the TSC counter and computes a new value. That is, the hardware TSC counter is never written. Because this operation is effectively writing to a moving target, the software is never guaranteed the value it could receive. That is, the various software operations to update the TSC value for a given core can be delayed indefinitely. If the delay occurs in between the software calculation of the new value and writing of this value, the calculation is invalid and the flow will have to be re-performed. Such operations can be common in a single processor system.
Greater complexity is involved in updating TSC values across multiple processors, particularly where these values need to be synchronized for correct software operation. To perform this writing of a TSC value for a core of a multiprocessor system, a more complex flow involving barriers and synchronizations. e.g., pairwise synchronization, is needed, which raises complexity and can be time consuming.
In various embodiments, a time stamp count (TSC) value for a given core (and/or one or more threads executing on the core) can be calculated independently of other cores. Such core TSC values may be based in part on an independent core TSC counter, along with various offsets as will be described below. To this end, reading and writing directly to a core offset can be performed, bypassing the need for synchronization between multiple cores of a multicore processor. As all cores/threads within a socket share a single hardware counter, writing directly to an offset for a core effectively allows software to update a core's offset at its leisure. Note this direct writing may be to a thread offset for each of multiple threads executing on one or more cores, as will be described in detail below. In various cases, a core TSC value can be used as a light-weight stamping mechanism to order transactions or to determine the current time of day, for example.
Referring now to
With reference to
Still referring to
Regardless of the state of the cores, uncore TSC counter 120 continues to count at its determined ratio and frequency. When different cores enter and exit from different low power states (e.g., so-called C-states of the Advanced Configuration and Power Interface (ACPI) Specification rev. 4 (Jun. 16, 2009)) or other alternate frequency states, the snapshot control mechanism may be used to implement a core frequency update.
In one embodiment, processor microcode may be used to write a register, causing a snapshot to occur and then it reads the snapshot value, polling to ensure the snapshot has been effected. As seen in
Still referring to
In addition, core logic 150 may include a core TSC counter 180. This counter may be controlled to appear to run at a stamped frequency for the processor. This counter may be updated by a controllable adder 160. In general, core TSC counter 180 may be clocked at a given rate of a core clock frequency. In addition, at predetermined intervals one of a large and/or small adjustment value, stored in adjustment storages 165 and 170 may be added to the count value to thus increment core TSC counter 180 to provide for adjustments when the core is actually executing at a different frequency than the stamped frequency.
That is, a processor is stamped with a particular frequency, e.g., 2 gigahertz (GHz) or 3 GHz, which is the rate at which the core TSC counter is to appear to run. However, because of implementation considerations such as a turbo mode and low power states, the core may be running at a different frequency, e.g., 1 GHz, 4 GHz or other such value. For purposes of various software, the illusion that the core is running at the stamped frequency should be maintained. Because the ratio at which a core may be running versus the ratio corresponding to the stamped frequency may not be multiples of one another, these small and large adjustments can be added to the core TSC value. In one embodiment, the small adjustment can be added every X core clocks, and the large adjustment may be added, e.g., as a least common multiple of when an X core clock lines up with a bus clock. Thus, the core TSC output in general is seen as a stairstep pattern. In one embodiment, the large and small adjustments may be predetermined and programmably calculated, e.g., by processor microcode when a frequency is changed. In one embodiment, this calculation may be in accordance with Equation 1:
where StampedRatio is the stamped processor frequency and CurrentRatio is the frequency at which a core is operating.
Using Equation 1, the resulting integer quotient is the small adjustment value, and the integer remainder is the large adjustment value. For ease of discussion, these values can be calculated in a slightly simpler fashion in accordance with Equation. 2.
4*Stamped Ratio=(Small*Current Ratio)+(Large−Small) [Eq. 2]
In both equations, 4 is an artifact of the core TSC counter, which adds every fourth clock. This rate merely saves power by avoiding clocking the core TSC counter at a higher rate than it can be read. The correction period is the least common multiple of 4 (since a count occurs every fourth Mclk) and the current ratio divided by the current ratio, which is the greatest common denominator (GCD) of 4 and current ratio multiplied by current ratio. Note the number four is used in this implementation, but other values can be used depending on a resolution at which the TSC counter is read, balanced with the power to be saved by clocking at a lower rate. Table 1 below shows example pseudo code to calculate the large and small adjustments in accordance with an embodiment of the present invention.
TABLE 1
>
correctionPeriod :: (Int) −> (Int)
>
correctionPeriod (currentratio) = div (leastCommonMultiple(currentratio, 4))
currentratio
The correctioncount is the GCD of 4 and currentratio mulitpled by 4. This is
the number of 4x Mclks in each correction period.
>
correctionCount :: (Int) −> (Int)
>
correctionCount (currentratio) = div (leastCommonMultiple(currentratio, 4)) 4
The correctioncount is the correct value based off the maxratio at the correction period.
>
correctionCountAtCorrection :: (Int, Int) −> (Int)
>
correctionCountAtCorrection (currentratio, maxratio) = (correctionPeriod
(currentratio)) * maxratio
The TSC values are modulo division and remainder of the correct counter at the
correction point and the number of 4 clock periods in that correction.
>
tscValues :: (Int, Int) −> (Int, Int)
>
TSCValues (currentratio, maxratio) = (x, y)
>
wherein x = div (correctCountAtCorrection(currentratio, maxratio))
(correctionCount(currentratio))
>
y = mod (correctCountAtCorrection(currentratio, maxratio))
(correctionCount(currentratio))
Note that the small and large adjustments may be adjusted responsive to the reset signal by snapshot controller 130, in some embodiments.
In one embodiment, controllable adder 160 adds the small increment value stored in storage 170 every fourth core clock and adds the large adjustment value stored in storage 165 every fourth core clock that lines up with a bus clock. Referring now to
In various embodiments, the two adjustment values stored in storages 165 and 170 can be exposed to software to allow the OS to cause time to speed up marginally without having large discontinuities with time. That is, the OS or other supervisor software can program the small or large adjustment amount, allowing software to tune the core TSC counter to catch up with real time incrementally.
Still referring to
In one embodiment, when a read TSC instruction is executed, microcode operates to read the core TSC value and sum the core TSC value, the thread offset and the core snapshot, obtaining an effective TSC value for that thread. In some embodiments a write TSC instruction, which may be a privileged instruction, may cause microcode to sum the core snapshot plus the core TSC counter and calculate what the thread offset would need to be in order to write it to whatever value that the thread wanted, namely a user-defined value. As such, the thread offset value could be a positive or negative number. For example, if the core snapshot was 1000, the core TSC was 2000, and the program instructed a write value of 5000, the thread offset value would become 2000. Note that this write TSC operation acts as a direct write to the thread offset value. In various embodiments, instead of performing a write of the desired core TSC value and then generating a thread offset value whenever software is able to do so, direct access to the thread offset value can occur so that the desired value is directly written to the thread offset value storage. In one embodiment the thread offset value may be stored in a model specific register (MSR). Thus in these embodiments thread offset storages 185 may be implemented as MSRs. Accordingly, a write to this MSR may be performed using a write to MSR instruction. In some embodiments, supervisor software, e.g., an OS, VMM or other privileged software may be used to perform this write to the MSR.
Thus in various embodiments, this thread offset value may be directly written, avoiding the need to perform complex and time consuming synchronization operations between multiple threads. Referring now to
As seen, method 200 may begin by recording a TSC value (block 210). In some embodiments, the method of
After some amount of time passes (block 230), the system may resume from suspension (block 240). This time may conclude when a user re-initiates operation, or responsive to expiration of a platform timer. At this time, the various time stamp counters, both of the uncore and cores may be initialized to zero on reset. In various embodiments, all TSC counters may be initialized (e.g., all are reset concurrently). Still further in a multi-socket system, all sockets may be reset at the same time.
Because software that was running prior to the suspension had been reading the TSC counter and expecting that value to increase over time, the OS can restore the TSC counter, either to the value before suspension or to a current wall clock time. This decision may be made by the OS, e.g., through a control value. In one embodiment, the wall clock time may be obtained from a real time clock of the system. Thus the recorded TSC value may be read (block 250). Then this recorded TSC value (either the previously stored value or the obtained real time clock value) may be written directly to the thread offset value (block 260). In one embodiment, this thread offset value may be directly written to every thread offset storage for all active threads pending in all active cores at the time of the reset. Note that this write operation may be performed without the need for a synchronization between multiple threads and/or cores.
In this way, a thread synchronization may be realized (block 270) efficiently, without the need for barrier or other synchronization operations. With this recorded thread offset value, a given thread may then execute on a core, using the thread offset value to reflect time as seen by the thread with respect to the core TSC counter (and the uncore TSC counter). Thus at this time, normal counting operations continue with both the uncore TSC counter and the active core TSC counters beginning counting from zero. In one embodiment, at this same time core snapshot storages 140 may also be reset to zero. Thus time as seen by the various threads may be maintained by receipt of the synchronization value and the various calculations discussed above with regard to the core TSC counter value, and the thread offset value (and any updated snapshot values when a frequency is updated for a given core). For other thread offset values or for cores that are not powered up at the same time as the system, the recorded TSC value may be later written when such threads/cores begin execution. Thus, this recorded TSC value acts as a synchronization value that enables updating offsets in different threads/cores to occur at different times but receive the same synchronization value. Still further, this synchronization value can be provided at different times without the need for a synchronization/barrier operation. While shown with this particular implementation in the embodiment of
Referring now to
As shown in
Coupled between front end units 310 and execution units 320 is an out-of-order (OOO) engine 315 that may be used to receive the micro-instructions and prepare them for execution. More specifically OOO engine 315 may include various buffers to re-order micro-instruction flow and allocate various resources needed for execution, as well as to provide renaming of logical registers onto storage locations within various register files such as register file 330 and extended register file 335. Register file 330 may include separate register files for integer and floating point operations. Extended register file 335 may provide storage for vector-sized units, e.g., 256 or 512 bits per register.
Various resources may be present in execution units 320, including, for example, various integer, floating point, and single instruction multiple data (SIMD) logic units, among other specialized hardware. For example, such execution units may include one or more arithmetic logic units (ALUs) 322. In addition, a timing logic 324 in accordance with an embodiment of the present invention may be present. Such timing logic may be used to execute certain of the operations involved in performing TSC updates, including controlling one or more ALUs 322 and to perform various operations to directly update a thread offset value. As will be discussed further below, additional logic for handling such updates may be present in another part of a processor, such as an uncore. Also shown in
As shown in
Referring now to
In other embodiments, a processor architecture may include emulation features such that the processor can execute instructions of a first ISA, referred to as a source ISA, where the architecture is according to a second ISA, referred to as a target ISA. In general, software, including both the OS and application programs, is compiled to the source ISA, and hardware implements the target ISA designed specifically for a given hardware implementation with special performance and/or energy efficiency features.
Referring now to
Still referring to
Embodiments may be implemented in many different system types. Referring now to
Still referring to
Furthermore, chipset 790 includes an interface 792 to couple chipset 790 with a high performance graphics engine 738, by a P-P interconnect 739. In turn, chipset 790 may be coupled to a first bus 716 via an interface 796. As shown in
While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover all such modifications and variations as fall within the true spirit and scope of this present invention.
Dixon, Martin G., Shrall, Jeremy J., Parthasarathy, Rajesh S.
Patent | Priority | Assignee | Title |
10122484, | Jun 30 2017 | Intel Corporation | Technologies for internal time syncrhonization |
10142088, | Jan 31 2014 | University of North Dakota | Network clock skew estimation and calibration |
11175712, | Mar 11 2013 | Daedalus Prime LLC | Controlling operating voltage of a processor |
11507167, | Mar 11 2013 | Daedalus Prime LLC | Controlling operating voltage of a processor |
11669146, | Jun 20 2018 | Intel Corporation | System, apparatus and method for responsive autonomous hardware performance state control of a processor |
11687135, | May 31 2013 | TAHOE RESEARCH, LTD | Controlling power delivery to a processor via a bypass |
11822409, | Mar 11 2013 | Daedauls Prime LLC | Controlling operating frequency of a processor |
11880231, | Dec 14 2020 | Microsoft Technology Licensing, LLC | Accurate timestamp or derived counter value generation on a complex CPU |
11921564, | Feb 28 2022 | CORPORATION, INTEL | Saving and restoring configuration and status information with reduced latency |
9395788, | Mar 28 2014 | Intel Corporation | Power state transition analysis |
9720441, | Sep 30 2016 | International Business Machines Corporation | Generating time-of-day values without causing execution stalls |
Patent | Priority | Assignee | Title |
6823516, | Aug 10 1999 | Sony Corporation of America | System and method for dynamically adjusting to CPU performance changes |
7043654, | Dec 31 2002 | Intel Corporation | Selecting a first clock signal based on a comparison between a selected first clock signal and a second clock signal |
7921317, | Sep 02 2008 | Unisys Corporation | Method and apparatus for synchronizing central processing units in a multiprocessor apparatus |
8122278, | Apr 28 2009 | Oracle America, Inc | Clock skew measurement for multiprocessor systems |
8139069, | Nov 03 2006 | Nvidia Corporation | Method and system for improving data coherency in a parallel rendering system |
20040049706, | |||
20070260907, | |||
20090222683, | |||
20090222684, | |||
20100058095, | |||
20100275053, | |||
CN101135929, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Dec 22 2009 | Intel Corporation | (assignment on the face of the patent) | / | |||
Dec 28 2009 | DIXON, MARTIN G | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 024012 | /0903 | |
Jan 05 2010 | SHRALL, JEREMY J | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 024012 | /0903 | |
Jan 06 2010 | PARTHASARATHY, RAJESH S | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 024012 | /0903 |
Date | Maintenance Fee Events |
Feb 25 2014 | ASPN: Payor Number Assigned. |
Sep 28 2017 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Sep 22 2021 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Date | Maintenance Schedule |
Apr 15 2017 | 4 years fee payment window open |
Oct 15 2017 | 6 months grace period start (w surcharge) |
Apr 15 2018 | patent expiry (for year 4) |
Apr 15 2020 | 2 years to revive unintentionally abandoned end. (for year 4) |
Apr 15 2021 | 8 years fee payment window open |
Oct 15 2021 | 6 months grace period start (w surcharge) |
Apr 15 2022 | patent expiry (for year 8) |
Apr 15 2024 | 2 years to revive unintentionally abandoned end. (for year 8) |
Apr 15 2025 | 12 years fee payment window open |
Oct 15 2025 | 6 months grace period start (w surcharge) |
Apr 15 2026 | patent expiry (for year 12) |
Apr 15 2028 | 2 years to revive unintentionally abandoned end. (for year 12) |