Parallelism for paranoids or what?

I was reading through my Google Reader list of new items and I came about this very interesting post by Hua You of Intel. What struck me most is this snippet:

... With proper tools, one experienced developer can tune their parallelized application approaching linear speedup or even super-linear. But that may require hours of training, weeks of practicing and maybe even month/years of experience accumulation. On the other hand, developers are expecting more intelligent support from software tools like compiler, run time libraries or some framework that can do the auto parallelization so that developers don't have to spend too much effort on digging out more speedup. So with that, what is the least necessary to be taught to the student is of question.

from Parallelism for paranoids or what?

In my days in college, I had none of the hours of training, weeks of practicing, and months to accumulate experience. However, I think reading up about the subject -- parallel computing -- and absorbing it at my own pace was the biggest investment I have ever made as far as doing parallel programming was concerned. The sad part about that was that I didn't have a single course that offered even the slightest iota of parallel programming theory.

So on one hand, I'm thinking yes the investment in developing the skills and insights to be able to do parallel programming effectively is expensive -- training, hours spent reading, practicing, etc. -- but the rewards especially in this day and age is very much worth whatever investment is put in.

On the other hand, I understand how people tend to concentrate on "bang for the buck". It's old-fashioned to think that people still put in the hours to learn the best practices on parallel computing, and that somehow the tools won't evolve so people will have to. This is where the runtimes get improved without having the programmers think about parallel-enabling their code and somehow it just happens. Thinking of this more, it seems like fitting the parallel programming idioms underneath the sequential programming idioms should somehow work to some extent -- true, but relying on it and automation to actually be *effective* is a completely different thing.

In the end developers have to be aware of the idioms in the particular environment and/or paradigm they're operating under. Parallel computing idioms sometimes have very little in common with established sequential computing idioms -- knowing this and becoming aware of these differences is essential to getting a better handle on the thought process when it comes to parallel programming. More to the point, understanding how parallel computing works is definitely a must if you intend to leverage it in your solutions.

I personally think that the underlying theories and idioms regarding parallel programming/computing have a place in the general education of computer scientists to be. If not at the tertiary education level, then maybe in post-graduate studies where the treatment of the subject may be more in-depth. Even if not in that space, I think there is a big market waiting to be tapped as far as training and consulting goes, especially since parallelism is becoming ever more ubiquitous in consumer hardware.

This is part of a running series of posts regarding parallel computing and parallel programming. If you have questions regarding the material or the subject and you don't feel like posting them as comments, send the author an email through dean.berris@cplusplus-soup.com.