Book on parallel programming

[foges]

Hey, I'm looking for a book/guide on parallel programming. I had a quick look through "Java, concurrency in practice" and it seemed like a good book, but it is too focused on "how to make sure that things work" rather than "how to implement conventional things in parallel and what the applications of parallel programming are", which is what I'm looking for. Any suggestions are greatly appreciated!

 

[Cogman]

Honestly, a good operating systems book is going to be a great resource for the concepts of concurrency and the issues associated with it. A good operating systems book will cover things like deadlock and deadlock avoidance, two of the biggest problems that you have with concurrency.

I used the dinosaur operating systems book
http://www.amazon.com/Operating-Syst.../dp/0470128720
Older editions can be had for much cheaper while still covering essentially the same things.

This is pretty abstract in its discussion about how to do concurrency, but good none the less.

The reason you are going to find the best information about concurrency in an OS book is because they are some of the first people to run into concurrency issues.

 

[iCyborg]

http://www.amazon.com/Concurrent-Pro.../dp/032143482X

I have the above book and I've quite liked it. It goes through the concepts too, but it also gives best practices and plenty of practical information. Obviously, it's Windows oriented, and he gives examples in both native (i.e. C++) and managed code (C#). I've only bought 2 Comp. Sci. books in the last 10 years or so since you can find so much online, but this was definitely a good one. It's fairly cheap too, $45.

 

[foges]

Thanks for the replies

@Cogman: I had an undergrad class in computer engineering and we used just that book, so I know what deadlocks, semaphores, etc.. are. I pretty much know all the theory of how to properly implement something once you know what you want to do. I want a book that will teach me how to take advantage of parallel computing..

@iCyborg: I'll have a look into that book, I mainly use unix based systems (ubuntu + OS X)

 

[LokutusofBorg]

I don't remember seeing many books focused on what you're asking for, so I can't make any recommendations, but I would suggest you pick a particular application like graphics, HPC (crunching large data sets), ray tracing, or any other highly parallel work load and just research the particulars of it.

I can't imagine there being too much out there about how to implement conventional things in parallel because, well, only certain workloads are easily parallelized. They put a lot of work into integrating seamless threading into .Net 4 (automatically splitting loops up and threading them, for instance), but AFAIK it didn't make it in. I believe that was work coming out of the Microsoft Robotics division, if that also gives you an angle to research.

I've done a lot of threading over the years (.Net business development), but to boil it down to easy categories, it was always related to two things: 1) splitting tasks into background threads so the UI stays responsive, 2) writing multi-threaded services that can process multiple things at once. Along the vein of #2 there, web servers are probably one of the best examples of a highly parallel application (not that I've ever written one myself, just saying).

 

[degibson]

Not everyone agrees on terminology, but I often say that there's a difference between parallel and concurrent programming. The former (parallel programming) seeks to reduce runtime though explicit parallelism, whereas the latter (concurrent programming) seeks to increase throughput though concurrent processing of independent events. Some of the concepts span both areas, but the goals and implementations often differ.

That said, the best way to learn either tends to be experiential. References are fine for basic abstractions -- threads, locks, etc.

 

[foges]

Ok, in that case I had a different understanding of what parallel programming is about. I guess the area that I would be most interested in is HPC, not too big on the whole graphics thing. Can anyone recommend a book on HPC then? I see that O'reilly used to have a book called High Performance Computing, but that book has apparently gone out of print..