> > All hard-sync inter-thread comms,, (eg. TThread.synchronize), where
> > the sender has to wait for the receiver, invites deadlocks. To avoid
> this,
> > and improve performance, use queued comms. When communicating to
> > the main thread from a secondary thread, this means PostMessage.
>
> You also have to make sure that the main thread processes all of the
queued
> messages, or else memory can be leaked. Don't just exit the message queue
> and terminate the threads, without also checking for any last-minute
> messages that may have been queued after the threads were signalled to
> terminate but before they actually did so.
Yes. Typically, this means pushing a suicide-request message 'round the
loop' both to signal the thread to terminate and ensure that all previous
messages/objects have been handled or purged.
> > When the message is received in a main-thread message-handler, it
> > contains an object with data that has been 'abandoned' by the read
> > thread and so can use the data easily.
>
> A safer way would be to copy the data into a thread-safe container that
the
> main thread owns, such as a TThreadList, and then the worker thread can
> signal the main thread when data has been placed into it. When the main
> thread receives the message, it can process all of the data that is
> currently in the list. This way, the main thread can process multiple
data
> blocks quickly since it does not have to serialize them one at a time.
While this certainly will work, and is often done, I'm not convinced that
the added safety/performance, if any, is worth the extra complication and
reduced flexibility.
I'm sure that you are right in that the main thread, if held up for a while,
will subsequently more quickly process 100 objects from a thread-safe queue
than 100 objects posted in separate Windows messages. OTOH, this is not the
whole story. After the main thread has handled the 100 objects from the
thread-safe queue, it then gets 99 PostMessages and polls the empty queue 99
times for nothing. So, for the thread-safe queue, no saving on handling
PostMessage receipts, extra code to be run in popping the queue plus an
extra CS acquire/release for each queue check :(
There is a persistent rumour that Windows occasionally loses messages. I
think that this fallacy arises from the behaviour of timers, which the
Windows docs imply work by *queueing* messages. If the main thread is tied
up for a time longer than a timer interval, it appears that a *timer message
gets lost', hence the erroneous extrapolation that all windows messages are
at risk of occasional 'loss'. Since timer signalling is implemented by
two-state event-flags in a list of timers maintained by the message-queue,
unlike custom messages that are *really* queued up, the behaviour is
different Windows does not lose messages. If it did, all Windows versions
would collapse faster than W95 on a bad day.
When communicating in an asynchronous manner between the main thread and
secondary thread/s, objects can be in a queue to the secondary thread, being
processed by the secondary thread, or be 'on their way back' in a posted
Windows message. Using an 'auxilliary' thread-safe list can ensure that the
objects queued back are freed. This does not help with objects in the other
two states. Queueing off a suicide-request, however, ensures that, when the
message gets back to the main thread, all objects have ben flushed out and
the secondary thread is either dead or circling the drain.
Sadly, Delphi, (at least my D5 anyway), keeps a CS-protected count of
TThreads for the dubiously-useful reference-counted lifetime-management of a
window used by synchronize. Together with the CS that protects the
memory-manager, this means that, due to possible priority-inversion, even
setting the priority of a secondary thread to tpTimeCritical on exiting will
not absolutely ensure that a 'freeOnTerminate' TThread has freed itself when
the suicide-note is received in the main thread. Why Borland was so
concerned over lifetime-management of one window is beyond me - I would
create the thing anyway at app. startup, whether TThread was used or not,
and just leave it there. Anyway, I have never seen any problem resulting
from this annoyance, no matter whether the app is closing down or not.
>
> You have to be careful that the main thread does not slow down too much,
or
> else you can push data into the queue faster than it can be processed, so
> the queue will fill up over time, will will lose messages, and thus will
> also cause memory leaks when PostMessage() fails but the reading thread is
> not cleaning up after itself.
Yes, this can happen if, as you say, the reading thread does not check the
result returned from PostMessage or no other mechanism is applied to
restrict the number of objects that can be outstanding.
> Using a thread-safe container is safer in
> that regard, as the data will not be leaked or lost, albeit in the case of
> lost messages and overflowing queues, the data may not be processed for
> awhile, but it will eventually.
You are right but, IMHO, your thread-safe container is in the wrong place
<g>.
If an app continually receives data faster than it can process it, something
will have to give eventually, no matter how you do the inter-thread comms.
Using posted messages, ths is likely to be a failed PM when 10000 objects
have been queued up or a memory-allocation failure when trying to create an
inter-thread comms object. Both these are easily detectable, but a
memory-full exception is likely to mean that your app is finished anyway and
has just run too long with an overload. If a thread-safe queue is used, the
terminal conditions are likely to be similar.
Memory allocation failures are generally disastrous and should be avoided,
no matter how the app works. PM fails are not quite as bad, but
nevertheless usually mean that data has been received that can not be
forwarded for processing and must be discarded. This does not matter much
with UDP since disappearing datagrams are a fact of life. It does matter
with TCP however. A termporarily-overloaded app should not read data from
the input stream and then, afterwards, discard it. Much better if the data
is not read at all during a short overload - TCP will then shut down the
input stream by shrinking the window size to zero. When the overload is
over, the data stream can then continue without loss.
I perfer to cap the number of available inter-thread comms objects by
creating a pool of them at app startup, ie. creating a fixed number of them
and pushing all the references onto a 'thread-safe queue' - a
Producer-Consumer queue on which threads can wait, if neccesary, for
objects. Any thread, that wants to read data has to pop an object from the
pool first before it can read data into it. When the object is eventually
finished with, it is pushed back onto the pool queue for re-use.
This has the following advantages:
1) Memory use for inter-thread comms objects is capped at a known level.
This can be fixed at compile-time, or be a configuration or user-tweakable
value.
2) Popping an object reference from a queue is faster than an object create.
Pushing an object back onto a queue is faster than an object destroy. The
more complex the object, the greater the saving.
3) Instead of a probably-fatal memory exception, an empty pool just blocks
the caller until objects become available. This allows an app to
auto-recover from temporary overloads.
4) If a read-thread becomes temporarily blocked on an empty pool, it cannot
reach the subsequent read call. It cannot therefore read any data from the
protocol stack until an object is pushed back onto the pool and the
read-thread becomes ready/running again, ie. until there is room for the
data. This preserves the integrity of TCP data streams, (and similar
guaranteed-delivery protocols), during temporary overloads.
5) The number of outstanding posted messages is capped at the number of
available objects. If the pool contains less than 10000 objects, PM cannot
fail because it cannot contain more than [pool size] messages.
6) Windows message queues are not the only class of queue to get
protection. Other Producer-Consumer queues are also guaranteed to have no
more than [pool size] messages in them. As long as the these other queue
classes can hold [pool size] objects, they do not need any overflow
protection code at all, save perhaps some 'catastrophic error'
exception-generating on gross over-queueing in case of a misbehaving
producer thread that gets stuck in a loop. No provision is required, in
normal operation, to block a producer thread if some artificial queue limit
is reached - it cannot happen unless there is a disastrous bug. This makes
P-C queues simpler and faster.
7) Pool depth is easily monitored, eg. by a simple main-thread timer that
dumps the pool level to a status-bar every second or so. Any object that
gets lost, or any duplicate object posting, will soon be noticed during
development/testing without any 3rd-party memory-managers with their complex
reports.
8) Adding special debug code to find duplicate/lost objects is fairly
trivial to add to the pool queue.
The downside is that all inter-thread objects are all created at startup and
so a chunk of memory is always unavailable for other purposes, even when the
app is mostly idle. IMHO, most developers would happily put up with this to
allow efficient and safe inter-thread comms.
Rgds,
Martin
