Parallel programming challenges

Multicore CPUs are mainstream nowadays. The number of CPU cores is quickly increasing. Getting the most from multicore processors is becoming an increasingly difficult task for programmers. Just adding more CPU cores will not make existing software to run faster. To get most out of the multicore CPUs you need to write your software in such way that it performs many tasks in parallel in different CPU cores. In this way you get the power from all CPU cores. The class of applications for which parallel processing is useful is growing rapidly.

Most computing intensive problems that a user will encounter at home are easily parallelizable, i.e. video encoding, gaming, photoshop filters, webbrowsing and so on. There exist whole classes of software that have been doing parallel execution, be it through threads, processes or messaging, for decades. Look at any high performance server software, look at apache, or database software.

You can read often how parallel programming is very complicated for programmers. Parallel programming can be simple and straightforward or very complicated depending what you are planning to do. Parallelism typically falls into two buckets: Data parallel and functional parallel. The first challenge is identifying what is what. The second challenge is synchronizing parallelism in as bug free way as possible while retaining the performance advantage of the parallelism.

Doing fine-grained parallelism will take long time to become mainstream because it is too abstract for most. Changing industry tool-chains to something entirely new takes many many years.

Short term solution for parallel programming problems will come in threaded/parallel frameworks that used with traditional programming languages.

The parallel programming support on popular programming languages is not up to date. C and C++ provide no assistance in keeping track of locks needed in parallel programs syncronization. In Java, the problem was at least thought about, but “synchronized” didn’t work out as well as expected.

Some programmers might even go on many years thinking that parallelism is an operating system feature, not a language issue. At the OS level, in most operating systems, the message passing primitives suck. What you usually want is a subroutine call; what the OS usually gives you is an I/O operation.

Does parallel processing require new languages? blog posting will give you some more information related to parallel programming.