Book on parallel programming

foges

Senior member
Mar 28, 2005
324
0
0
Hey, I'm looking for a book/guide on parallel programming. I had a quick look through "Java, concurrency in practice" and it seemed like a good book, but it is too focused on "how to make sure that things work" rather than "how to implement conventional things in parallel and what the applications of parallel programming are", which is what I'm looking for. Any suggestions are greatly appreciated!
 

Cogman

Lifer
Sep 19, 2000
10,283
134
106
Honestly, a good operating systems book is going to be a great resource for the concepts of concurrency and the issues associated with it. A good operating systems book will cover things like deadlock and deadlock avoidance, two of the biggest problems that you have with concurrency.

I used the dinosaur operating systems book
http://www.amazon.com/Operating-Syst.../dp/0470128720
Older editions can be had for much cheaper while still covering essentially the same things.

This is pretty abstract in its discussion about how to do concurrency, but good none the less.

The reason you are going to find the best information about concurrency in an OS book is because they are some of the first people to run into concurrency issues.
 
Last edited:

iCyborg

Golden Member
Aug 8, 2008
1,334
56
91
http://www.amazon.com/Concurrent-Pro.../dp/032143482X

I have the above book and I've quite liked it. It goes through the concepts too, but it also gives best practices and plenty of practical information. Obviously, it's Windows oriented, and he gives examples in both native (i.e. C++) and managed code (C#). I've only bought 2 Comp. Sci. books in the last 10 years or so since you can find so much online, but this was definitely a good one. It's fairly cheap too, $45.
 

foges

Senior member
Mar 28, 2005
324
0
0
Thanks for the replies

@Cogman: I had an undergrad class in computer engineering and we used just that book, so I know what deadlocks, semaphores, etc.. are. I pretty much know all the theory of how to properly implement something once you know what you want to do. I want a book that will teach me how to take advantage of parallel computing..

@iCyborg: I'll have a look into that book, I mainly use unix based systems (ubuntu + OS X)
 

LokutusofBorg

Golden Member
Mar 20, 2001
1,065
0
76
I don't remember seeing many books focused on what you're asking for, so I can't make any recommendations, but I would suggest you pick a particular application like graphics, HPC (crunching large data sets), ray tracing, or any other highly parallel work load and just research the particulars of it.

I can't imagine there being too much out there about how to implement conventional things in parallel because, well, only certain workloads are easily parallelized. They put a lot of work into integrating seamless threading into .Net 4 (automatically splitting loops up and threading them, for instance), but AFAIK it didn't make it in. I believe that was work coming out of the Microsoft Robotics division, if that also gives you an angle to research.

I've done a lot of threading over the years (.Net business development), but to boil it down to easy categories, it was always related to two things: 1) splitting tasks into background threads so the UI stays responsive, 2) writing multi-threaded services that can process multiple things at once. Along the vein of #2 there, web servers are probably one of the best examples of a highly parallel application (not that I've ever written one myself, just saying).
 

degibson

Golden Member
Mar 21, 2008
1,389
0
0
Not everyone agrees on terminology, but I often say that there's a difference between parallel and concurrent programming. The former (parallel programming) seeks to reduce runtime though explicit parallelism, whereas the latter (concurrent programming) seeks to increase throughput though concurrent processing of independent events. Some of the concepts span both areas, but the goals and implementations often differ.

That said, the best way to learn either tends to be experiential. References are fine for basic abstractions -- threads, locks, etc.
 

foges

Senior member
Mar 28, 2005
324
0
0
Ok, in that case I had a different understanding of what parallel programming is about. I guess the area that I would be most interested in is HPC, not too big on the whole graphics thing. Can anyone recommend a book on HPC then? I see that O'reilly used to have a book called High Performance Computing, but that book has apparently gone out of print..