asynchronous control of threads

I am attempting to build a library that is transparent to the client code. A shared resource is used by many threads with their own synchronization code, but every once in a while, ALL threads need to be stopped for some background control thread to update this resource before proceeding. I have tried many solutions, all of which have given me code that almost works...

• Total synchronization -- a mutex controls each and every access to the resource. This is very inefficient because so much time is spent arbitrating which thread gets to access the resource, and most of the time this is overkill -- arbitration isn't needed a lot of the time, most things can safely happen in parallel.
• Signals -- this one is painfully close to working brilliantly, but every once in a while signals go somewhere they're not supposed to. There seems no way to guard against this, and most advice I see in the literature boils down to "Don't mix signals and threads". And there's worse -- I've had pthread_kill hang on me.
• mprotect -- a variation on signals, but theoretically more safe since the signal originates in the thread that needs to be controlled... at least delivery to the correct thread is guaranteed. Give threads a custom stack, a signal handler that waits on some sort of queue, and a spare signal stack, then set the stack as PROT_NONE when you want threads to halt. Does not work. Signals can't be caught when the stack can't be written or read, even when there's an extra signal stack available.
• longjmp -- Give threads custom stacks and have threads save their status occasionally with setjmp and longjmp. When needed, instead of suspending threads, kill them dead, then create new ones to take their place and prerecorded state via longjmp. Does not work. There seem to be inconsistences between the stack of new threads and the stack left by the old thread.

I've got a few ideas left, but none of them are particularly elegant, all involving some sort of regimented system to control access or kernel-level thread calls. I was hoping to do this transparently. Is this impossible under pthreads?

Mutexes are the gold standard for this kind of thing. If you have really excessive contetion, you could create a control or "scheduling" thread. It handles mutex contention issues by queuing requests, setting conditions, and calling pthread_cond_signal() to wake a thread. Unless this is what you meant by "signal".

Do you have a copy of Richard Stevens book? -- 'Advanced Programming in the UNIX Environment'

It covers all of these topics.

If you're using Linux, pthread had some issues before kernel 2.6 release, and pthread_cond_signal was one of them, if I remember correctly. It may still have other issues. I don't know. I don't code in Linux very often.

Yes, if I was willing to give up transparency and just force everything to run synchronously in regimented lock-step I could get this working, but that would eliminate all parallelism, the whole point of this environment. I'm looking for nontraditional solutions, essentially.

So far, linux has been the BEST environment for this... Solaris occasionally has odd problems with signals, and OSX's pthreads and signals implimentation is so blighted the whole thing just churns itself down into a sticky mass the instant I hit 'enter'. I'm slowly, slowly getting closer to an implimentation that works on all three...

Thanks for the book reccomendation, I'll check it out.