Callbacks and threads...

[SunnyD]

I just had a brief discussion with an intern (which is relevant to the project I'm working on at the moment) regarding callbacks and such and it got me thinking... are callbacks okay to use in a multithreaded environment, more importantly across threads?

I started thinking about it, and I may be missing something on how callbacks work, but wouldn't it be possible for a thread using a callback into another thread (presumably it's parent for most sane purposes) to preempt its parent, err, for lack of any better way to describe it, BY its parent?

I suppose one of the things I am fuzzy on is does the callback run in the context of the thread or in the context of the thread that callback is from (ie: its parent)?

Further, assuming many threads using the same callback, I assume unless the callback in inherently atomic each thread can preempt other threads in the callback, thus again preempting the callback's original context? (Essentially snowballing)

(For me this is just fun stuff to think about, hence why I ask)

 

[Snapster]

I'm by no means an expert but I'd like to chip in as threading fascinates me. Callbacks are a natural part of threading so of course they are okay to use in a multithreaded environment. In an over simplified example, say you have an FTP application which is uploading two files at once, each upload has its own thread and will be invoking a callback to update the UI on progress and obvious again once the process is finished.

As for cross thread invoking, I'm not sure of how or what you mean without further example so I cannot comment much, but given that a thread is started by passing in a method to start and is within the context of that thread I'm struggling a little to see how one would invoke a method in another executing thread. In practise you can access another thread's context to suspend/wait/resume etc although I doing this can be like walking a minefield of problems and really should be up to a controlling/parent to dictate thread actions.

I guess it also depends on the language/environment whether there is something like a thread manager which handles pools of threads or if they are created all manually. I know in some of the threading I've done in .Net the invoking is executed in the context of the thread and it's up to either you to do blocking/sync lock of common callback methods and cross thread accessed members. In some cases you could have a callback from a thread which spawns off another callback if an invoke is required to say the main UI thread which forces queuing of executing callbacks.

 

[degibson]

Originally posted by: SunnyD
I just had a brief discussion with an intern (which is relevant to the project I'm working on at the moment) regarding callbacks and such and it got me thinking... are callbacks okay to use in a multithreaded environment, more importantly across threads?

You bet! Snapster is right on in his observation that the notion of a callback is a fundamental concepts to a lot of threaded servers and services.

I started thinking about it, and I may be missing something on how callbacks work, but wouldn't it be possible for a thread using a callback into another thread (presumably it's parent for most sane purposes) to preempt its parent, err, for lack of any better way to describe it, BY its parent?

Yes, it is usually possible for a thread to put a callback on another thread's queue, say Thread A sending a callback to Thread B. It is a common method, if an overly restrictive method, to communicate control between threads. However, just sending the callback doesn't necessarily preempt Thread B -- Thread B will execute the callback when it is done with it's current callback, and possibly after other queued callbacks.

I suppose one of the things I am fuzzy on is does the callback run in the context of the thread or in the context of the thread that callback is from (ie: its parent)?

This really depends on your notion of what a thread really is. If you mean 'thread as in processes', then the execution always happens in the context of the local process... the 'thread' executing the callback. Those 'threads' are running in completely separate virtual address spaces. If you mean 'thread as in threads in shared memory' then the answer is both, as there is no difference between thread contexts as far as memory is concerned.

I'm really not entirely sure what you mean here, since threads usually enjoy a peering relationship, and hierarchies are usually a process-level notion.

Further, assuming many threads using the same callback, I assume unless the callback in inherently atomic each thread can preempt other threads in the callback, thus again preempting the callback's original context? (Essentially snowballing)

(For me this is just fun stuff to think about, hence why I ask)

Whether or not this happens depends on how the callback is synchronized. Yes, some callbacks are made atomic implicitly by whatever entity is scheduling the callback. However, if the callback doesn't actually operate on shared state (e.g. heap variables or static globals), or does so in a synchronized way (e.g. with a set of locks) then the same static callback (e.g. the same function) can operate in parallel on multiple threads.

It seems a bit hairy until you get your mind around it, but function-scope variables end up on the individual thread's stack, and are therefore safe from inter-thread races (unless you do really ugly things not worth mentioning here).

In particular, there isn't anything magical about two or more threads executing the same piece of code -- in general, that notion doesn't imply mutual exclusivity or preemption.

I hope this helps...

Edit: Fixed a grammatical error.

 

[SunnyD]

Originally posted by: degibson
It seems a bit hairy until you get your mind around it, but function-scope variables end up on the individual thread's stack, and are therefore safe from inter-thread races (unless you do really ugly things not worth mentioning here).

In particular, there isn't anything magical about two or more threads executing the same piece of code -- in general, that notion doesn't imply mutual exclusivity or preemption.

I hope this helps...

Edit: Fixed a grammatical error.

I'll say!

To answer the previous question - I'm generally assuming a thread spawned by a process, as in a child thread, not a child process. So I guess what's going on then is that when a callback happens in a thread, that function is called on that thread's stack rather than the parent's (owner of the callback) stack, making things okay...

... but in the case that the callback modifies something of the owner's that is scoped outside of the function (global's or statics or whatnot) then considerations need to be made for them to be thread-safe?

 

[Crusty]

Originally posted by: SunnyD

Originally posted by: degibson
It seems a bit hairy until you get your mind around it, but function-scope variables end up on the individual thread's stack, and are therefore safe from inter-thread races (unless you do really ugly things not worth mentioning here).

In particular, there isn't anything magical about two or more threads executing the same piece of code -- in general, that notion doesn't imply mutual exclusivity or preemption.

I hope this helps...

Edit: Fixed a grammatical error.

I'll say!

To answer the previous question - I'm generally assuming a thread spawned by a process, as in a child thread, not a child process. So I guess what's going on then is that when a callback happens in a thread, that function is called on that thread's stack rather than the parent's (owner of the callback) stack, making things okay...

... but in the case that the callback modifies something of the owner's that is scoped outside of the function (global's or statics or whatnot) then considerations need to be made for them to be thread-safe?

Exactly, you'll either need to make sure the operation is atomic, or you need to force it to be atomic via synchronization.

For simple multi threaded apps I normally use a blocking Queue to transfer information between threads. Your worker thread will block so long as your Queue is empty, but as soon as a different thread puts information in the Queue your worker thread will unblock and process the data. Depending on the language would change how you implement it, but with C# it's fairly simple.

 

[Markbnj]

Do too much of this stuff and you'll have a tangled ball of potential deadlock conditions. About the only threading that doesn't scare me is the "go do job and report back" variety, where the data is read-only or exclusive. Callbacks... I just felt a chill and the hairs on the back of my neck got all tingly.

 

[SunnyD]

Originally posted by: Markbnj
Do too much of this stuff and you'll have a tangled ball of potential deadlock conditions. About the only threading that doesn't scare me is the "go do job and report back" variety, where the data is read-only or exclusive. Callbacks... I just felt a chill and the hairs on the back of my neck got all tingly.

Heh, that made me laugh. I agree, worker threads yes. Most everything else, bad.

 

[degibson]

Originally posted by: SunnyD

Originally posted by: Markbnj
Do too much of this stuff and you'll have a tangled ball of potential deadlock conditions. About the only threading that doesn't scare me is the "go do job and report back" variety, where the data is read-only or exclusive. Callbacks... I just felt a chill and the hairs on the back of my neck got all tingly.

Heh, that made me laugh. I agree, worker threads yes. Most everything else, bad.

Markbnj is correct to be afraid of exotic threading techniques -- it is very much a black art, even these days, to get threads to interact in nontrivial ways.