Comments on: Hunting the Mythical Automatic Parallelizing Compiler https://www.nextplatform.com/2015/04/01/hunting-the-mythical-automatic-parallelizing-compiler/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Mon, 23 Apr 2018 11:13:15 +0000 hourly 1 https://wordpress.org/?v=6.7.1 By: John D McCalpin, Ph.D. https://www.nextplatform.com/2015/04/01/hunting-the-mythical-automatic-parallelizing-compiler/#comment-500 Thu, 02 Apr 2015 17:40:05 +0000 http://www.nextplatform.com/?p=534#comment-500 Two of the (many) serious problems with generating parallel applications are:

(1) Programming languages that are fundamentally serial, with limited ability to specify particular kinds of parallelism bolted on as afterthoughts.

(2) Hardware architectures that are fundamentally serial, with just enough restrictions added to enable communication and synchronization to be implemented correctly.

The combination of these two factors is a big problem. Many attempts to develop programming languages that enable fine-grained parallelism have failed because fine-grained communication and synchronization are remarkably slow on current hardware. The smaller number of attempts to support fine-grained parallelism in hardware have largely failed because there are no widely used programming models that require this functionality.

At larger scales of parallelism, the inability of programming languages to specify data motion (both through the memory hierarchy and across memory hierarchies) and the inability of hardware to explicitly control data motion provide the next stumbling block. As with fine-grained parallelism, there have been numerous failed attempts to include data motion control in software, but without explicit hardware support these are doomed to be inefficient. Likewise, attempts to provide explicit control over data motion in hardware have failed because of the lack of mature software support.

The same software/hardware “chicken and egg” problem applies to the ubiquitous failure to develop secure systems. You can’t really do it without new programming languages running on new hardware.

]]> By: Jure Pecar https://www.nextplatform.com/2015/04/01/hunting-the-mythical-automatic-parallelizing-compiler/#comment-486 Thu, 02 Apr 2015 10:31:36 +0000 http://www.nextplatform.com/?p=534#comment-486 What might be missing? This:
https://www.youtube.com/watch?v=jN9L7TpMxeA

I’ve just finished the book by John Gustafson and it is extremely inspiring:
http://www.crcpress.com/product/isbn/9781482239867

]]>