A cold plate has blades arranged to be interleaved with memory modules or memory module sockets. A liquid cooling loop is thermally coupled to the blades of the cold plate.
|
14. A method for removing heat from a plurality of memory modules cooled by a single liquid cooling loop coupled to interleaved cold plate blades of a cold plate comprising:
conducting heat from a first side of each memory module of the plurality of memory modules to a cold plate blade of the cold blades, the cold plate blade includes a sloped surface and a surface that is not sloped, the first side cooled by the surface that is not sloped of the cold plate blade;
conducting heat from a second side of each memory module of the plurality of memory modules to each frame blade of a frame, each frame blade includes a sloped surface and a surface that is not sloped, the second side cooled by the surface that is not sloped of the frame blade;
conducting heat from the frame blades to the cold plate blades; and
conducting heat from the cold plate blades to liquid flowing in the liquid cooling loop.
6. A computer system comprising:
at least one central processing unit;
a bank of memory module sockets having a plurality of memory module sockets;
a core logic;
at least one bus coupling the at least one central processing unit, the bank of memory module sockets, and the core logic;
a cold plate having a plurality of cold plate blades interleaved with the bank of memory module sockets, to cool a plurality of memory modules when the plurality of memory modules are inserted in the bank of memory module sockets;
a frame having a plurality of frame blades supported in the frame, the plurality of frame blades arranged to be interposed between the plurality of memory module sockets and the plurality of cold plate blades when the frame is coupled to the cold plate;
a liquid cooling loop having an inlet and an outlet, and in thermal contact with the plurality of cold plate blades; and
a liquid cooling loop pump and cooling unit coupled to the inlet and the outlet of the liquid cooling loop,
wherein the plurality of cold plate blades and the plurality of frame blades each have a sloped surface and a surface that is not sloped.
1. An apparatus for cooling memory modules comprising:
a cold plate comprising:
a liquid cooling loop having an inlet and an outlet; and
a plurality of cold plate blades thermally coupled to the cooling loop, the plurality of cold plate blades arranged to be interleaved with a plurality of memory modules; and
a frame comprising:
a plurality of frame blades arranged to be interposed between the plurality of memory modules and the plurality of cold plate blades when the frame is coupled to the cold plate,
wherein at least one first cold plate blade of the plurality of cold plate blades includes a sloped surface, and at least one second cold plate blade of the plurality of cold plate blades includes a surface that is not sloped, and at least one first frame blade of the plurality of frame blades includes a sloped surface, and at least one second frame blade of the plurality of frame blades includes a surface that is not sloped, and wherein a memory module of the plurality of memory modules has a first side cooled by the surface that is not sloped of the at least one second cold plate blade, and the memory module has a second side cooled by the surface that is not sloped of the at least one second frame blade.
2. The apparatus for cooling memory modules according to
3. The apparatus for cooling memory modules according to
4. The apparatus for cooling memory modules according to
5. The apparatus for cooling memory modules according to
7. The computer system according to
wherein the frame pivots along the hinge between an open position and a closed position, and in the closed position, the plurality of frame blades are interposed between the plurality of cold plate blades.
8. The computer system according to
a plurality of memory modules inserted into the plurality of memory module sockets of the bank of memory module sockets;
wherein when the frame is in the closed position, each memory module of the plurality of memory modules has a first side cooled by thermal contact with a surface of a frame blade of the plurality of frame blades that is not sloped, and a second side that is cooled by thermal contact with a surface of a cold plate blade of the plurality of cold plate blades that is not sloped, with sloped surfaces of each of the plurality of frame blades in thermal contact with sloped surfaces of a corresponding one of the plurality of cold plate blades.
9. The computer system according to
10. The computer system according to
first and second handles having a locking mechanisms that engage locking features of the frame;
wherein the plurality of frame blades are interposed between the plurality of cold plate blades.
11. The computer system according to
a plurality of memory modules inserted into the plurality of memory module sockets of the bank of memory sockets;
wherein each memory module of the plurality of memory modules has a first side cooled by thermal contact with a surface of a frame blade of the plurality of frame blades that is not sloped, and a second side that is cooled by thermal contact with a surface of a cold plate blade of the plurality of cold plate blades that is not sloped, with sloped surfaces of each of the plurality of frame blades in thermal contact with sloped surfaces of a corresponding one of the plurality of cold plate blades.
12. The computer system according to
13. The computer system according to
a plurality of memory modules inserted into the plurality of memory module sockets of the bank of memory sockets;
wherein when the frame is coupled to the cold plate, each memory module of the plurality of memory modules has a first side cooled by thermal contact with a surface of a frame blade of the plurality of frame blades that is not sloped, and a second side that is cooled by thermal contact with a surface of a cold plate blade of the plurality of cold plate blades that is not sloped, with sloped surfaces of each of the plurality of frame blades in thermal contact with sloped surfaces of a corresponding one blades of the plurality of cold plate blades.
15. The method according to
16. The method according to
17. The method according to
|
The present patent application is related to the following patent applications: COOLING MEMORY MODULES USING COLD PLATE BLADES COUPLED TO THE MEMORY MODULES VIA CLIPS by Timothy Rau and Glenn C. Simon, and assigned US Patent Application Publication 2012/0026670, A FRAME HAVING FRAME BLADES THAT PARTICIPATE IN COOLING MEMORY by Timothy Rau and Glenn C. Simon, and assigned US Patent Application Publication 2012/0020004, and COOLING MEMORY-MODULES USING WEDGE-SHAPED HEAT SPREADERS IN THERMAL CONTACT WITH COLD PLATE BLADES AND MEMORY MODULES by Timothy Rau, Glenn C. Simon, and Bryan Bolich, and assigned World Intellectual Property Organization Publication 2011/053313. All related applications, and the present application, were filed on Oct. 30, 2009.
In the art of computing, individual memory integrated circuits (ICs) are often provided on a dual in-line memory module (DIMM). Often a heat spreader is attached over the memory ICs to dissipate the heat generated by the memory ICs across the length of the DIMM. However, it is often desirable to provide additional cooling.
Typically, DIMM sockets are positioned on a motherboard in close proximity to each other, thereby simplifying routing of memory signal traces on the motherboard and minimizing space used by memory. A typical separation between adjacent DIMMs is 10 millimeters.
Two methods known in the art for providing additional cooling are air cooling and liquid cooling. Because of the close spacing of adjacent DIMMs, both methods often use space above the DIMM. Typically, air cooling uses a solid heat conducting metal or vapor chambers and associated tubing to conduct heat from the heat spreader to a heatsink above the DIMM.
Typically, liquid cooling uses a suitable liquid, such as propylene glycol or ethylene glycol, mixed with water, to conduct heat from the heat spreader to the liquid. The heat is removed as the liquid is pumped through a channel associated with each DIMM. The liquid is then pumped to a heat exchanger, where heat is removed from the liquid. Typically, tubing is coupled to each DIMM along the top of the top of the DIMM.
The Figures depict embodiments, implementations, and configurations of the invention, and not the invention itself.
In the foregoing description, numerous details are set forth to provide an understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these details. While the invention has been disclosed with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover such modifications and variations as fall within the true spirit and scope of the invention.
As discussed in the Background section above, commercially available dual in-line memory module (DIMM) cooling solutions use space above the DIMMs to facilitate cooling. For air cooling, heatsinks are positioned above the DIMMs. Furthermore, cooling fans and air channels are often needed to move air over the heatsinks.
Commercially available liquid cooling solutions have a liquid inlet and outlet for each DIMM. The inlets and outlets, along with the associated tubing, consume space above the DIMMs. Furthermore, the need to connect tubing to the inlet and outlet of each DIMM makes assembly and servicing cumbersome.
In accordance with embodiments of the present invention, a cold plate having a liquid inlet and outlet is provided for each block of DIMMs. DIMM sockets are attached to a motherboard, and may have a standard spacing of 10 millimeters between adjacent DIMM sockets. The cold plate includes a series of wedge-shaped blades that are positioned proximate the DIMM sockets such that a surface of a wedge-shaped blade is positioned adjacent to a DIMM heat spreader when a DIMM is installed. Note that in other configurations, it is possible to use DIMMs without heat spreaders, with the surface of a blade in direct contact with the individual memory integrated circuits (ICs) on the DIMM.
A frame having blades is positioned, over each block of DIMMs. In one embodiment, the frame is attached to the cold plate via a hinge, and pivots into an installed and closed position. In another embodiment, the frame is separate from the cold plate, and is installed into a final position by latching the frame to the cold plate at two ends of the frame.
The blades of the frame are attached to the frame using spring-loaded mechanisms that allow the blades to have a certain amount of movement within the frame. Each blade of the frame has a first surface adapted to contact a heat spreader of a DIMM, and a second surface adapted to contact a wedge-shaped blade of the cold plate.
When assembled, a cold plate/frame configuration positioned around a bank of DIMMs, in accordance with embodiments of the present invention, uses little additional space compared to a standard bank of DIMMs of the prior art. Installation and servicing are simplified, since after the frame is removed or pivoted to an open position, DIM Ms can be added or removed, as in the prior art.
Computer system 10 includes one or more central processing units (CPUs) 12, core logic 14, DIMMs 16, bus 18, cold plate and frame 20 (which includes blades 17, liquid inlet 22, and liquid outlet 24), tubing 26, and liquid cooling loop pump/cooling unit 28.
CPUs 12 represents CPUs known in the art, such as several CPUs in discrete packages and multi-core CPUs in a single package. Core logic 14 represents core logic known in the art, such as a south bridge, a north bridge, memory controllers, I/O controllers, and the like. As the art of computers continues to advance, some of these functions, such as the memory controllers, are provided in the CPU package. Bus 18 represents one or more buses known in the art for connecting CPUs 12, core logic 14, and DIMMs 16.
Liquid cooling loop pump/cooling unit 28 pumps and cools liquid coolant using techniques known in the art. Any suitable coolant liquid, such as propylene glycol or ethylene glycol, mixed with water, can be used. The liquid may be cooled using a simple heat exchanger and fan, or by more advanced techniques, such as Peltier coolers or heat pumps. Also note that the function provided by unit 28 may be implemented at a variety of levels, such as in the computer system, within a rack, within a row of racks, or within a data center. It is also possible to integrate the liquid cooling function with a data center air conditioning system.
Note that it may also be desirable to cool CPUs 12 using a liquid cooling loop. In such a configuration, the loop can also flow through the CPUs, or a separate loop may be provided. For simplicity, the liquid cooling loop is only shown as cooling DIMMs 16. The liquid flows through tubing 26 in the direction show by the arrows in the tubing. The cold plate of cold plate and frame 20 includes an inlet 22 and an outlet 24, both of which are coupled to tubing 26.
The liquid cooling loop is in thermal contact with blades 17. In
Cold plate 32 of cold plate and frame 20 includes fixed wedge-shaped blades 34, which are thermally coupled to liquid cooling loop 36. In a configuration supporting eight DIMMs as shown in
Frame 38 includes wedge-shaped blades 42, which have a certain degree of movement provided by a spring-load mechanism, which will be discussed in greater detail below. In a configuration supporting eight DIMMs as shown in
In
Note that before frame 38 is lowered into place, each DIMM 16 has one surface adjacent to a sloped surface of a blade 34 (such as surface 67), and another surface adjacent to a surface of a blade 34 that is not sloped (such as surface 69). As frame 38 is lowered into place, the sloped surfaces of blades 42 (such as surface 71) contact the sloped surfaces of blades 34, and the surfaces of blades 42 that are not sloped (such as surface 73) contact a DIMM. In the closed position shown in
The cooperating sloped edges of the blades 34 and 42 provide lateral force to the DIMMs 16 to enhance thermal coupling. A spring-loaded mechanism, which will be discussed in greater detail with reference to
When frame 38 is in the closed position, blades 42 are in contact with DIMMs 16 and blades 34 of cold plate 32, as shown in
Note that when frame 38 is in the closed position, gaps will typically exist between mounting members 70, and the inner walls of cavities 72, retention members 74, and cap bar 78. In essence, the mounting members are “floating” within mounting cavities 72, with downward pressure being exerted by the springs 76. Accordingly, each blade 42 can accommodate variations caused by thermal expansion and contraction, and variations caused by manufacturing and assembly tolerances.
In the embodiments shown in
Table 1 below shows temperature measurements taken at points A, B, C, D, and E of
In a typical server computer system, it is desirable to maintain case temperatures at or below 85° C. As can be seen in Table 1, the embodiments shown in
TABLE 1 | ||||
Temperature gradient | ||||
Temperature | Delta T | |||
Location | (deg C.) | (deg C.) | ||
Low liquid flow | A | 81 | ||
rate (0.3 | 13 | |||
liters/min) | B | 68 | ||
1 | ||||
C | 67 | |||
11 | ||||
D | 56 | |||
24 | ||||
E | 32 | |||
High liquid flow | A | 72 | ||
rate (1.1 | 12 | |||
liters/min) | B | 60 | ||
4 | ||||
C | 56 | |||
11 | ||||
D | 45 | |||
17 | ||||
E | 28 | |||
While the embodiments shown in
In
Block 100A, in conjunction with block 100, represents the embodiments shown in
Embodiments of the present invention provide many advantages over the prior art. Liquid cooling loop connections remain fixed as DIMMs are added or removed. In contrast, prior art configurations provide liquid inlets and outlets for each DIMM, thereby causing the addition and removal of DIMMs to be cumbersome and time consuming. With embodiments of the present invention, a single inlet and outlet is provided for a block of DIMMs, and the inlet/outlet connections need only be coupled once during the manufacturing process.
Embodiments of the present invention require little extra space above the DIMMs, as is shown in the Figures. Prior art air and liquid cooling solutions often consume space above the DIMMs. In addition, embodiments of the present invention have a system board “footprint” similar to prior art DIMM blocks. The only extra area required is the area reserved for the cooling loop along the sides of the DIMM block, and the area reserved for the inlets and outlets, and cooling loop connections. Also, space is saved by eliminating the need for cooling fans to direct airflow over the DIMMs. Of course, acoustic levels may also be reduced. Finally, embodiments of the present invention provide simple and tool-free memory configuration, since the frame is easily removed from the cold plate using one or more handles on the frame, thereby providing access to the DIMMs
In the foregoing description, numerous details are set forth to provide an understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these details. While the invention has been disclosed with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover such modifications and variations as fall within the true spirit and scope of the invention.
Patent | Priority | Assignee | Title |
10021811, | May 24 2016 | ASETEK DANMARK A/S | Single ended cooling module rows and assemblies for thermal management of in-line memory modules |
10143108, | Nov 21 2016 | Cooler Master Co., Ltd. | Heat dissipating device |
10334757, | Nov 15 2012 | International Business Machines Corporation | Cooling methods for electronic components |
10368430, | Jun 06 2017 | Cooler Master Co., Ltd. | Heat dissipation device |
10462932, | Nov 01 2017 | Hewlett Packard Enterprise Development LP | Memory module cooler with vapor chamber device connected to heat pipes |
10520259, | Aug 17 2017 | Hewlett Packard Enterprise Development LP | Memory module cooler with rotatable cooling tube |
10602602, | Apr 28 2018 | EMC IP HOLDING COMPANY LLC | Heat sink for plug-in card, plug-in card including heat sink, and associated manufacturing method |
10602640, | Nov 14 2018 | Hewlett Packard Enterprise Development LP | Memory coolers |
10705578, | Nov 15 2018 | Hewlett Packard Enterprise Development LP | Heat removal from memory modules |
10834852, | Nov 15 2012 | International Business Machines Corporation | Cooling methods for electronic components |
10869381, | Apr 28 2018 | EMC IP HOLDING COMPANY LLC | Heat sink for plug-in card, plug-in card including heat sink, and associated manufacturing method |
11058032, | Nov 01 2017 | Hewlett Packard Enterprise Development LP | Memory module cooler with vapor chamber device connected to heat pipes |
11266046, | Mar 19 2019 | LE COMMISSARIAT À L ÉNERGIE ATOMIQUE ET AUX ÉNERGIES ALTERNATIVES | Heatsink for multiple memory modules |
11301008, | Oct 24 2016 | EUROTECH S P A | Cooled electronic circuit board |
11687130, | Jan 14 2022 | Dell Products L.P. | Heater apparatus-integrated peripheral component interconnect card for a computing device |
11788802, | Sep 29 2020 | Delta Electronics, Inc. | Water cooling device and collector thereof |
11924996, | Sep 30 2020 | CoolIT Systems, Inc | Liquid-cooling devices, and systems, to cool multi-chip modules |
12075587, | Feb 17 2022 | Hamilton Sundstrand Corporation | Heat transfer interfaces for circuit card assembly (CCA) modules |
12087108, | Feb 17 2021 | Samsung Electronics Co., Ltd. | Memory apparatus for vehicle and temperature control method of the memory apparatus |
12108566, | Mar 21 2022 | Baidu USA LLC | Server memory array cooling hardware |
8767403, | Oct 30 2009 | Hewlett Packard Enterprise Development LP | Frame having frame blades that participate in cooling memory modules |
9068784, | Nov 15 2012 | International Business Machines Corporation | Cooling system for electronics |
9089076, | Jul 06 2012 | International Business Machines Corporation | Cooling system for electronics |
9144178, | Mar 01 2013 | International Business Machines Corporation | Selective clamping of electronics card to coolant-cooled structure |
9158348, | Jun 29 2012 | ASETEK DANMARK A S | Server memory cooling apparatus |
9258925, | Mar 01 2013 | International Business Machines Corporation | Selective clamping of electronics card to coolant-cooled structure |
9509102, | Jan 16 2015 | TE Connectivity Solutions GmbH | Pluggable module for a communication system |
9513064, | Nov 15 2012 | International Business Machines Corporation | Cooling system for electronics |
9591787, | Mar 01 2013 | International Business Machines Corporation | Selective clamping of electronics card to coolant-cooled structure |
9786578, | Jan 27 2014 | LENOVO GLOBAL TECHNOLOGIES INTERNATIONAL LTD | Orthogonally hinged individualized memory module cooling |
9848517, | May 22 2014 | FG SRC LLC | System and method for thermally coupling memory devices to a memory controller in a computer memory board |
9867315, | Jun 29 2012 | ASETEK DANMARK A/S | Server memory cooling apparatus |
D800674, | May 24 2016 | ASETEK DANMARK A/S; ASETEK DANMARK A S | Cooling plate row for in-line memory |
D800675, | May 24 2016 | ASETEK DANMARK A/S; ASETEK DANMARK A S | Set of cooling plate rows for in-line memory |
Patent | Priority | Assignee | Title |
6665184, | Jul 13 2001 | LYTRON, INC | Tapered cold plate |
6853554, | Feb 22 2001 | Hewlett Packard Enterprise Development LP | Thermal connection layer |
7023701, | May 05 2003 | Polaris Innovations Limited | Device for cooling memory modules |
7151668, | Oct 28 2004 | Muskin, Inc. | Memory heat sink |
7286355, | Sep 11 2002 | Cooling system for electronic devices | |
7289331, | Mar 30 2005 | International Business Machines Corporation | Interposable heat sink for adjacent memory modules |
7339793, | Mar 30 2005 | LENOVO INTERNATIONAL LIMITED | Interposable heat sink for adjacent memory modules |
7342797, | Mar 30 2005 | LENOVO INTERNATIONAL LIMITED | Interposable heat sink for adjacent memory modules |
7539020, | Feb 16 2006 | Liebert Corporation | Liquid cooling loops for server applications |
7599184, | Feb 16 2006 | Vertiv Corporation | Liquid cooling loops for server applications |
7907398, | Oct 02 2008 | Dell Products L.P. | Liquid cooling system |
7933125, | Mar 24 2008 | Fujitsu Limited | Board unit and electronic apparatus |
7965509, | Apr 06 2009 | LENOVO INTERNATIONAL LIMITED | High performance dual-in-line memory (DIMM) array liquid cooling assembly and method |
7969736, | Feb 08 2010 | International Business Machines Corporation | System for cooling memory modules |
8004841, | May 06 2008 | GLOBALFOUNDRIES Inc | Method and apparatus of water cooling several parallel circuit cards each containing several chip packages |
8081473, | Aug 04 2008 | International Business Machines Corporation | Apparatus and method of direct water cooling several parallel circuit cards each containing several chip packages |
8289710, | Feb 16 2006 | Vertiv Corporation | Liquid cooling systems for server applications |
8385069, | May 24 2010 | LENOVO INTERNATIONAL LIMITED | Liquid coolant conduit secured in an unused socket for memory module cooling |
8493738, | May 06 2011 | International Business Machines Corporation | Cooled electronic system with thermal spreaders coupling electronics cards to cold rails |
20060250772, | |||
20080084668, | |||
20080264613, | |||
20090002951, | |||
20100085712, | |||
20100091447, | |||
20120020004, | |||
20120026670, | |||
20120279686, | |||
20130120926, | |||
WO2011053313, | |||
WO2013078828, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Oct 29 2009 | RAU, TIMOTHY | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 027417 | 0117 | |
Oct 29 2009 | SIMON, GLENN C | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 027417 | 0117 | |
Oct 30 2009 | Hewlett-Packard Development Company, L.P. | (assignment on the face of the patent) | ||||
Oct 27 2015 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Hewlett Packard Enterprise Development LP | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 037079 | 0001 |
Date | Maintenance Fee Events |
Mar 21 2017 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Mar 23 2021 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Date | Maintenance Schedule |
Oct 29 2016 | 4 years fee payment window open |
Apr 29 2017 | 6 months grace period start (w surcharge) |
Oct 29 2017 | patent expiry (for year 4) |
Oct 29 2019 | 2 years to revive unintentionally abandoned end. (for year 4) |
Oct 29 2020 | 8 years fee payment window open |
Apr 29 2021 | 6 months grace period start (w surcharge) |
Oct 29 2021 | patent expiry (for year 8) |
Oct 29 2023 | 2 years to revive unintentionally abandoned end. (for year 8) |
Oct 29 2024 | 12 years fee payment window open |
Apr 29 2025 | 6 months grace period start (w surcharge) |
Oct 29 2025 | patent expiry (for year 12) |
Oct 29 2027 | 2 years to revive unintentionally abandoned end. (for year 12) |