Comments on: The Memory Area Network At The Heart Of IBM’s Power10 https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Fri, 26 Apr 2024 10:58:25 +0000 hourly 1 https://wordpress.org/?v=6.7.1 By: Slideshare downloader https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-223666 Fri, 26 Apr 2024 10:58:25 +0000 http://www.nextplatform.com/?p=137067#comment-223666 Fascinating read on the Memory Area Network of IBM’s Power10! It’s great to see the details on how this innovative architecture is capable of delivering such high performance and efficiency. Definitely looking forward to seeing the real-world applications of this technology in the future.

]]> By: JayN https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154501 Sat, 12 Sep 2020 16:53:26 +0000 http://www.nextplatform.com/?p=137067#comment-154501 This wiki description indicates an “up to” 52 bit physical address space capability for Ice Lake. Did Intel go into details of this for the Ice Lake Server implementation, recently presented at Hotchips 2020?

“The 5-level paging scheme supports a Linear Address space up to 57 bits and a physical address space up to 52 bits, increasing the virtual memory space to 128 petabytes, up from 256 terabytes, and the addressable physical memory to 4 petabytes, up from 64 terabytes.[14][13]”

https://en.wikipedia.org/wiki/Ice_Lake_(microprocessor)

]]> By: Mark Robert Funk https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154449 Fri, 11 Sep 2020 23:04:46 +0000 http://www.nextplatform.com/?p=137067#comment-154449 In reply to Timothy Prickett Morgan.

First, doing fine. Got asked back to teach DB, architecture, and a class on the first part of any development process, all remote. Students seem to like it. Great retirement “job”. As to this machine’s programming storage model, I’d love to. I need a connection or two though. I know Mr. Starke and he might still know me from IBM i, but could I ask you to make (re)introductions to folks there in the know? I can take it from there.

]]> By: Eric Olson https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154426 Fri, 11 Sep 2020 17:32:27 +0000 http://www.nextplatform.com/?p=137067#comment-154426 In reply to Timothy Prickett Morgan.

Probably Mark’s last name is better written without a final trailing y.

I would imagine that memory gets partitioned somehow between processors so that cache invalidation messages don’t overwhelm the memory interconnects at scale. How flexible this is would be an interesting topic to report on. Given how innovative the hardware is, I find it strange that IBM still seems to focus their marketing efforts on nontechnical business types rather than the engineers that the sensible decision maker relies on for guidance.

Along slightly different lines, as demonstrated by the ascent of x86 and now ARM, it seems the way up involves getting a bunch of young developers involved in writing software. As a significant amount of software these days is written on single-user desktops and notebook computers, it seems unlikely that Power10 will become popular at large scale unless it first becomes affordable at small scales.

Unfortunately, having a separate memory controller attached through a fast SerDes link may increase prices on the desktop beyond Power9, which was already too expensive to be attractive to software developers.

My vote for an affordable, entry-level system is based on a heartfelt desire for the modern power architecture to become popular enough that it survives into the future.

]]> By: Wrink https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154177 Tue, 08 Sep 2020 03:18:12 +0000 http://www.nextplatform.com/?p=137067#comment-154177 In reply to Timothy Prickett Morgan.

Yes, there is no authoritative documentation outside of the branding, so I’ll point you to these pieces instead.

https://archive.eetasia.com/www.eetasia.com/ART_8800718995_499486_NT_dcd3d606.HTM

The papers I referenced are available in IEEE Explore, presented at IEDM and such premier conferences from some 2-4 years ago. Its too bad the researchers and companies hype up the “memristors” and such, to mislead people in order to sell their papers. We probably have more ReRAM papers than actual ReRAM bits ever produced.

]]> By: Timothy Prickett Morgan https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154172 Tue, 08 Sep 2020 00:30:45 +0000 http://www.nextplatform.com/?p=137067#comment-154172 In reply to Wrink.

And of course, there is this statement:

https://thememoryguy.com/emerging-memories-today-the-technologies-mram-reram-pcm-xpoint-fram-etc/

]]> By: Timothy Prickett Morgan https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154171 Tue, 08 Sep 2020 00:29:12 +0000 http://www.nextplatform.com/?p=137067#comment-154171 In reply to Mark Funk.

As you might imagine, we would love for you to take a hard look at this and give us your thoughts, Mr Funky. Hope you are well.

]]> By: Timothy Prickett Morgan https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154170 Tue, 08 Sep 2020 00:28:24 +0000 http://www.nextplatform.com/?p=137067#comment-154170 In reply to Wrink.

I thought the consensus was that it was a variant of ReRAM and was not a spin on PCM… See: https://www.theregister.com/2015/07/28/intel_micron_3d_xpoint/

And:
https://en.wikipedia.org/wiki/3D_XPoint

Not that I necessarily trust Wikipedia, mind you. I have not seen the papers you refer to.

]]> By: Mark Funk https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-154037 Sun, 06 Sep 2020 06:58:49 +0000 http://www.nextplatform.com/?p=137067#comment-154037 Very impressive, having an ability for the cores of one node being able to access scads of terabytes of memory residing both locally and remote. But this article left me with a question. Are accesses fully cache coherent across the memory cluster or does cache itself need to be managed by software?
For example, picture a workload running only on the cores of a Node A. I would expect that, no matter the source location of the memory, the cores in Node A would be capable of moving data between their own caches in a cache coherent fashion (i.e. no involvement by software). But now picture some IO device on Node B altering some part of the memory being used by the workload running on Node A, say simply reading a page from persistent storage or packets from some communications device. Did the caches of the cores on Node A see the change such that their associated cache lines were invalidated? Would subsequent accesses of the remote memory-mapped real address space by Node A then access the changed data in Node B?
Or suppose we go in the other direction. Suppose that some of the pages of that memory remote from Node A but residing on a Node B need to be written to some IO device, say an SSD array. The IO device on Node B drives the write to the SSD. But let’s have a cache on Node A holding changed data normally residing in the memory of Node B and it is that changed data that needs to be included in the IO write. Did the cache on Node A automatically see the need for its data to be included in the IO write or is there first a need for software to force the cached/changed data back into Node B.
Again, impressive, but there are software architecture aspects of this that might need to be factored in to be able to use such massive memory in a single workload, even if it is running just on one node. Can anybody comment on the level of cache coherence actually supported here?

]]> By: Wrink https://www.nextplatform.com/2020/09/03/the-memory-area-network-at-the-heart-of-ibms-power10/#comment-153996 Sat, 05 Sep 2020 22:53:59 +0000 http://www.nextplatform.com/?p=137067#comment-153996 “3D XPoint (really a tweak of ReRAM)”.

While your architectural speculations are highly enjoyable and can be be spot-on (as the one on Power10 was) – you ought to get real data on what should not have to be speculation.

3-D Xpoint is Phase Change Memory. Its same fundamental memory for both Intel and Micron in terms of silicon (in fact Intel is currently buying it all from Micron, who owns the Fab). No one else has any persistent memory in production.

ReRAM has always been and will likely always be an “emerging” memory. The papers from Micron prove how and why its impossible to make any product out of ReRAM. HP is the most prominent failure after backing it, after relentlessly hyping ReRAM as a “memristor”.

]]>