Parallel Codes and Memory Models

[degibson]

This post is a little unusual in that I don't actually have a problem, but I'd like to poll the users here to see what their experience, opinion, and gripes are regarding parallel programming in general and memory models in particular.

Basically, I have been 'accused' of having an unrealistic vision of parallel coding, as I do it quite a bit... but it remains an important problem, and I want to gain some intuition as to how others reason about it. That said, any on-topic comments would be appreciated... or if you like, comment on these (canned) questions.

1) In what language(s) have you written parallel applications or toy programs? What abstractions (MPI, threads, TBB, Map/Reduce... something else)? I'm especially interested in hearing about non-POSIX platform parallelism, and parallelism in languages 'higher' than C++, as my expertise is limited in those cases.

2) What parallelization strategies are popular? Data-parallel, control-parallel, pipeline-parallel, something else?

3) What about synchronization? Locks, STM, non-blocking wait-free...?

4) Here is one I am particularly curious: How many folks have been burned by either a language-level memory model (like Java) or a hardware-level memory model (e.g. x86/Processor Consistency or SPARC/TSO,RMO)? My intuition suggests that most folks will have nothing to say in this category... am I wrong?

PS -- I'll be away for a week, so I won't be reading replies right away...

 

[Crusty]

I'll get back to you in a few weeks 😛. I've done lots of multi threading in C# apps, but they are hardly parallel 😉

 

[DaveSimmons]

Most desktop applications have no use for single-task parallel programming, they just need separate worker threads for long-running or blocking tasks.

Even games just need separate threads for much of the work. Processing one task like input or audio in multiple parallel threads per task is pointless most of the time.

I've never had a use for the styles of coding you mention writing Windows desktop applications in C++, just worker threads. There were no issues to speak of.