IT: Machine Independent Programming on Hierarchically Nested Machines

Abstract

Andrews in his "Concurrent Programming: Principles and Practice" expresses that "concurrent programs are to sequential programs what chess is to checkers". We believe people experienced in both kinds of programming will largely agree to his statements. But what makes parallel programming so difficult? Although there are differences of opinions, none blames the difficulty of devising a parallel algorithm, as opposed to a sequential algorithm, as the core problem.

It is true that construing a parallel algorithm sometimes requires considerably more ingenuity compared to its sequential counterpart, but one can expect that a programmer who is knowledgeable about his problem can come up with a parallel algorithm (or get help from his community) to solve it. Rather, most problems occur in the process of translating the parallel algorithm into a program, as deftly described by Snyder in his seminal work "Type Architectures, Shared Memory, and the Corollary of Modest Potential".