I'm developing an application that should perform certain sql queries in different MSSQL servers after specified intervals. My thoughts are about make an array of threads and in each thread run timer in which task will run. Does it really needs to make each thread for each timer, or should I just distinguish one timer for each task (whatever timer will produce thread)? Any thoughts about implementation?
Thanks a lot, guys!
I doubt that you need to have one thread per task. It would probably suffice to create one timer per task. If a timer fires whilst another task is running then the second task will have to queue up but it doesn't sound like that will be a great problem.
If you are going to use a Delphi TTimer to do this you'll need to make sure that your service has a message queue and runs a message loop on that queue. You may wish to run that message queue on a separate thread but if you do make sure that the TTimer objects are created on that thread so that they are associated with the right message queue.
You ask in the comments how to run a message loop in a thread. The following code should suffice:
repeat
try
Application.HandleMessage;
except
Application.HandleException(Application);
end;
until Terminated;//this is the Terminated property of the thread
This will give you all the bells and whistles of the Delphi message loop. If you want a very standard message loop you can use this:
procedure PerformThreadLoop;
var
Msg: TMsg;
begin
repeat
Try
while PeekMessage(Msg, 0, 0, 0, PM_REMOVE) do begin
TranslateMessage(Msg);
DispatchMessage(Msg);
end;
WaitMessage;
Except
Application.HandleException(Self);
End;
until Terminated;//this is the Terminated property of the thread
end;
If all you want is to pump WM_TIMER messages both will work but I personally would be inclined to go with the second option, the raw Win32 API version.
Related
I am reading Nick Hodges online and I have discovered the Queue but it is not behaving as I've expected and I couldn't understand what him and the documentation say. Look at this code:
TThread.CreateAnonymousThread(
procedure
begin
TThread.Queue(TThread.Current, procedure
begin
Memo1.Lines.Clear;
Memo1.Lines.Add('start');
end);
Sleep(2000);
TThread.Synchronize(TThread.Current, procedure
begin
Memo1.Lines.Add('end');
end);
end
).Start;
I always use Synchronize but this time I have tried with Queue because according to Nick it is better in case of multiple requests since they won't be "serialized" and executed one by one. The code above works fine. Why this is not working instead?
TThread.CreateAnonymousThread(
procedure
begin
TThread.Queue(TThread.Current, procedure
begin
Memo1.Lines.Clear;
Memo1.Lines.Add('start');
end);
Sleep(2000);
TThread.Queue(TThread.Current, procedure
begin
Memo1.Lines.Add('end');
end);
end
).Start;
In this case the Memo outputs the start but not the end. When I call:
Synchronize on the first and Synchronize on the second it works
Queue on the first and Synchronize on the second it works
Queue both times it's not working because I see only the start in the memo
The difference between queue and synchronize is that Synchronize() puts the call in a queue and waits for that call to be completed and Queue() puts the call in the queue and directly returns control to the thread.
However... and this is not mentioned in the official documentation, when a thread finishes, all the calls placed in the queue with Queue(AThread, AMethod), where AThread is its own thread, are removed.
You can see that clearly in the source of TThread.Destroy() where RemoveQueuedEvents(Self) is called.
RemoveQueuedEvents removes queued method calls. [...] If AThread is specified, then all method calls queued by this thread are removed.
So directly after your last Queue() your thread ends, TThread.Destroy() is executed and that last call(s) is/are removed from the queue.
There are some things you can do to solve this.
Like mentioned in the comments you can call TThread.Queue(nil, AMethod). B.T.W. Calling TThread.Queue(AMethod) is the same as TThread.Queue(Self, AMethod) so you'll always need to use the nil-variant if the thread is going to end and you want the call to finish.
But... If you still need the thread active when executing the call (for some data from it), you'll need to block the thread from exiting. You can do that by using Synchronize() as last queue-method. Note that the last synchronize doesn't have to be a real procedure. You can just call synchronize to a dummy-procedure at the end of the TThread.Execute like Synchronize(DummySync) (example). The queue is FIFO so the thread will wait until all calls in the queue are processed (including the empty dummysync).
Some extra information can be found on these pages
Ensure all TThread.Queue methods complete before thread self-destructs
http://www.uweraabe.de/Blog/2011/01/30/synchronize-and-queue-with-parameters/
I have discovered that if a method queued with TThread.Queue calls a method that invokes TApplication.WndProc (e.g. ShowMessage) then subsequent queued methods are allowed to run before the original method has completed. Worse still, they don't seem to be invoked in FIFO order.
[Edit: Actually they do start in FIFO order. With ShowMessage it looks like the later one ran first because there is a call to CheckSynchronize before the dialog appears. This unqueues the next method and runs it, not returning until the latter method has completed. Only then does the dialog appear.]
I'm trying to ensure that all methods queued from the worker thread to run in the VCL thread run in strict FIFO order, and that they all complete before the worker thread is destroyed.
My other constraint is that I am trying to maintain strict separation of the GUI from the business logic. The thread in this case is part of the business logic layer so I can't use PostMessage from an OnTerminate handler to arrange for the thread to be destroyed (as recommended by a number of contributors elsewhere). So I'm setting FreeOnTerminate := True in a final queued method just before TThread.Execute exits. (Hence the need for them to execute in strict FIFO order.)
This is how my TThread.Execute method ends:
finally
// Queue a final method to execute in the main thread that will set an event
// allowing this thread to exit. This ensures that this thread can't exit
// until all of the queued procedures have run.
Queue(
procedure
begin
if Assigned(fOnComplete) then
begin
fOnComplete(Self);
// Handler sets fWorker.FreeOnTerminate := True and fWorker := nil
end;
SetEvent(fCanExit);
end);
WaitForSingleObject(fCanExit, INFINITE);
end;
but as I said this doesn't work because this queued method executes before some of the earlier queued methods.
Can anyone suggest a simple and clean way to make this work, or a simple and clean alternative?
[The only idea I've come up with so far that maintains separation of concerns and modularity is to give my TThread subclass a WndProc of its own. Then I can use PostMessage directly to this WndProc instead of the main form's. But I'm hoping for something a bit more light-weight.]
Thanks for the answers and comments so far. I now understand that my code above with a queued SetEvent and WaitForSingleObject is functionally equivalent to calling Synchronize at the end instead of Queue because Queue and Synchronize share the same queue. I tried Synchronize first and it failed for the same reason as the code above fails - the earlier queued methods invoke message handling so the final Synchronize method runs before the earlier queued methods have completed.
So I'm still stuck with the original problem, which now boils down to: Can I cleanly ensure that all of the queued methods have completed before the worker thread is freed, and can I cleanly free the worker thread without using PostMessage, which requires a window handle to post to (that my business layer doesn't have access to).
I've also updated the title better to reflect the original problem, although I'd be happy for an alternative solution that doesn't use TThread.Queue if appropriate. If someone can think up a better title then please edit it.
Another update: This answer by David Heffernan suggests using PostMessage with a special AllocateHWnd in the general case if TThread.Queue isn't available or suitable. Significantly, it's never safe to use PostMessage to the main form because the window can be spontaneously recreated changing its handle, which would cause all subsequent messages to the old handle to be lost. This makes a strong argument for me adopting this particular solution, since there's no additional overhead to creating a hidden window in my case since any application using PostMessage should do this - i.e. my separation of concerns argument is irrelevant.
TThread.Queue() is a FIFO queue. In fact, it shares the same queue that Thread.Sychronize() uses. But you are correct that message handling does cause queued methods to execute. This is because TApplication.Idle() calls CheckSynchronize() whenever the message queue goes idle after processing new messages. So if a queued/synched method invokes message processing, that can allow other queued/synched methods to being running even if the earlier method is still running.
If you want to ensure a queue method is called before the thread terminates, you should be using Synchronize() instead of Queue(), or use the OnTerminate event instead (which is triggered by Synchronize()). What you are doing in your finally block is effectively the same as what the OnTerminate event already does natively.
Setting FreeOnTerminate := True in a queued method is asking for a memory leak. FreeOnTerminate is evaluated immediately when Execute() exits, before DoTerminate() is called to trigger the OnTerminate event (which is an oversight in my opinion, as evaluating it that early prevents OnTerminate from deciding at termination time whether a thread should free itself or not after OnTerminate exits). So if the queued method runs after Execute() has exited, there is no guarantee that FreeOnTerminate will be set in time. Waiting for a queued method to finish before returning control to the thread is exactly what Synchronize() is meant for. Synchronize() is synchronous, it waits for the method to exit. Queue() is asynchronous, it does not wait at all.
I fixed this problem by adding a call to Synchronize() at the end of my Execute() method. This forces the thread to wait till all of the calls added with Queue() are completed on the main thread before the call added with Synchronize() can be called.
TMyThread = class (TThread)
private
procedure QueueMethod;
procedure DummySync;
protected
procedure Execute; override;
end;
procedure TMyThread.QueueMethod;
begin
// Do something on the main thread
UpdateSomething;
end;
procedure TMyThread.DummySync;
begin
// You don't need to do anything here. It's just used
// as a fence to stop the thread ending before all the
// Queued messages are processed.
end;
procedure TMyThread.Execute;
begin
while SomeCondition do
begin
// Some process
Queue(QueueMethod);
end;
Synchronize(DummySync);
end;
This is the solution I finally adopted.
I used a Delphi TCountdownEvent to track the number of outstanding queued methods from my thread, incrementing the count just before queuing a method, and decrementing it as the final act of the queued method.
Just before my override of TThread.Execute returns, it waits for the TCountdownEvent object to be signalled, i.e. when the count reaches zero. This is the crucial step that guarantees that all of the queued methods have completed before Execute returns.
Once all of the queued methods are complete, it calls Synchronize with an OnComplete handler - thanks to Remy for pointing out that this is equivalent to but simpler than my original code that used Queue and WaitForSingleObject. (OnComplete is like OnTerminate, but called before Execute returns so that the handler can modify FreeOnTerminate.)
The only wrinkle is that TCountdownEvent.AddCount works only if the count is already greater than zero. So I wrote a class helper to implement ForceAddCount:
procedure TCountdownEventHelper.ForceAddCount(aCount: Integer);
begin
if not TryAddCount(aCount) then
begin
Reset(aCount);
end;
end;
Normally this would be risky, but in my case we know that by the time the thread starts waiting for number of queued methods outstanding to reach zero no more methods can be queued (so from this point once the count hits zero it will stay at zero).
This doesn't completely solve the problem of queued methods that handle messages, in that individual queued methods could still appear to run out of order. But I do now have the guarantee that all queued methods run asynchronously but will have completed before the thread exits. This was the primary goal, because it allows the thread to clean itself up without the risk of losing queued methods.
A few thoughts:
FreeOnTerminate is not the end of the world if you want your thread to delete itself.
Semaphores let you maintain a count should you feel the need, there are such constructs.
There's nothing to stop you writing or using your own queueing primitives and AllocateHWnd if you want some fine grained control.
I am using TThread, i just need to find out if thread is still running and terminate the application if it takes to long.
Ex:
MyThread := TMyThread.Create;
MyThread .Resume;
MyThread.Terminate;
total := 0;
while MyThread.isRunning = true do // < i need something like this
begin
// show message to user to wait a little longer we are still doing stuff
sleep(1000);
total := total + 1000;
if total > 60000 then
exit;
end;
Is this possible with Delphi?
The straight answer to your question is the Finished property of TThread. However, I don't recommend that. It leads you to that rather nasty Sleep based loop that you present in the question.
There are a number of cleaner options, including at least the following:
Use WaitFor to block until the thread completes. This will block the calling thread which will preclude showing UI, or responding to the user.
Use MsgWaitForMultipleObjects in a loop, passing the thread handle. This allows you to wait until the thread completes, but also service the UI.
Implement an event handler for the OnTerminate event. This allows you to be notified in an event driven manner of the thread's demise.
I recommend that you use one of these options instead of your home-grown Sleep based loop. In my experience, option 3 generally leads to the cleanest code. But it is hard to give rigid advice without knowing all the requirements. Only you have that knowledge.
I have thread where downloading xml file a this xml file i want to parse in other class. Now I have prepare thread, where I can download it.
In MainForm I'm using WaitForSingleObject, but this function freeze MainFrom, so how can I prevent reezing and wait for value?
Thanks.
First, design your thread with events and invoke them using Synchronize.
Create thread in MainForm and assign event handlers to it:
Thread := TMyThread.Create(True);
Thread.OnDownload := DoDownload;
Thread.OnParse := DoParse;
// Other Events
Thread.Start;
You should not WaitFor* in MainForm. Avoid Windows messages method because your app will loose platform portability, just linked to WinXX compilation.
It's not really a good design to wait on the main thread, but in some cases it can be necessary. However, while you wait, you still need to process messages coming to the thread.
Take a look at MsgWaitForMultipleObjects at MSDN.
I'm using Delphi 2006 and have a bit of a problem with an application I'm developing.
I have a form that creates a thread which calls a function that performs a lengthy operation, lets call it LengthyProcess. Inside the LengthyProcess function we also call several Dll functions which also create threads of their own.
The problem that I am having is that if I don't use the Synchronize function of my thread to call LengthyProcess the thread stops responding (the main thread is still responding fine). I don't want to use Synchronize because that means the main thread is waiting for LengthyProcess to finish and therefore defeats the purpose of creating a separate thread.
I have tracked the problem down to a function inside the dll that creates a thread and then calls WaitFor, this is all done using TThread by the way. WaitFor checks to see if the CurrentThreadID is equal to the MainThreadID and if it is then it will call CheckSychronization, and all is fine. So if we use Synchronize then the CurrentThreadID will equal the MainThreadID however if we do not use Synchronize then of course CurrentThreadID <> MainThreadID, and when this happens WaitFor tells the current thread (the thread I created) to wait for the thread created by the DLL and so CheckSynchronization never gets called and my thread ends up waiting forever for the thread created in the dll.
I hope this makes sense, sorry I don't know any better way to explain it. Has anyone else had this issue and knows how to solve it please?
If your secondary thread "stops responding," then I assume it has a message pump. (Otherwise, you need to explain what it stops responding to.) You appear to also wish for the thread to be able to detect when the tertiary thread finishes running. (The "primary" thread here is the VCL thread, which isn't involved at all.)
You tried using WaitFor, but were disappointed when you discovered that it blocks. That's what it has always been designed to do, though. Its behavior in the main thread is where it gets weird, so it's safe to call from the VCL thread even though it was never really meant to be used that way originally.
To process messages and wait for threads to finish running, you need to use one or more of the wait functions from the Windows API. Start with MsgWaitForMultipleObjects. It can wait for various types of kernel handles, including thread handles, but also notify you when messages are available. The idea is that you'll call that function in a loop. When it says messages are available, handle them, and then loop again to continue waiting.
The following is just an outline. You'll want to check the documentation for all the API functions used, and combine that with the rest of the knowledge you have about your own threads.
procedure TSecondaryThread.Execute;
var
ret: DWord;
ThreadHandle: THandle;
Msg: TMsg;
begin
ThreadHandle := TertiaryThread.Handle;
repeat
ret := MsgWaitForMultipleObjects(1, ThreadHandle, False, Infinite, qs_AllEvents);
case ret of
Wait_Object_0: begin
// The thread terminated. Do something about it.
CloseHandle(ThreadHandle);
PostQuitMessage(0);
// Put *something* in the parameter so further calls to MWFMO
// will have a valid handle. May as well use a handle to something
// that will never become signaled so all we'll get are more
// messages. I'm pretty sure you can't pass an empty array of
// handles; there must be at least one, and it must be valid.
ThreadHandle := Self.Handle;
end;
Wait_Object_0 + 1: begin
// At least one message is available. Handle *all* of
// them before calling MsgWaitForMultipleObjects again
while PeekMessage(Msg, 0, 0, 0, pm_Remove) do
case Msg.Message of
wm_Quit: begin
// Do something about terminating the tertiary thread.
// Then stop the message loop and the waiting loop.
Exit;
end;
else begin
TranslateMessage(Msg);
DispatchMessage(Msg);
end;
end;
end;
Wait_Timeout: Assert(False, 'Infinity has passed');
Wait_Failed: RaiseLastOSError;
else Assert(False, 'Unexpected return value');
end;
until False;
end;
The part about handling all the messages is important. As soon as you call GetMessage, PeekMessage, or WaitMessage, the OS marks all messages in the queue as "old," but MsgWaitForMultipleObjects will only return when there is a "new" message on the queue — one that arrived after the last call to PeekMessage.
HI, Thanks for your reply, yes i realize that my question isn't very clear and somewhat confusing; so i'll try to clarify things a bit, here goes..
All of the threads described below are derived from TThread.
I have a form which starts a thread but does not wait for it. The thread started by the form calls a function that performs a long task.
The function calls another function in a DLL, the function in the DLL starts a thread and waits for it. The thread started by the DLL function calls another function via synchronize.
Form->Starts a thread but does not wait->The thread calls a functions->The function calls a DLL function->The Dll function starts a thread and waits->The thread started by the DLL function calls another function via synchronize i.e Synchronize(UpdateRecords).
The problem is that the call to synchronize never returns because, from what i can see, it has entered some sort of dead lock.
How synchronize works: Synchronize puts the method call into a queue and sets an event, Synchronize then waits for the event to become signaled. When the main thread is idle it will process the method calls that are waiting in the queue, after it has processed a method call it will signal the associated event so that the thread that initiated the synchronization can continue on.
The thread that was started by the form does not use synchronize to call the function that performs the long task, if it does use synchronize then the application does not dead lock, but this defeats the purpose of use a thread for the long process.
I've tracked down the problem, it seems to be that the TApplication object created by the dll is not processing messages and has a handle of 0, how this happened I don't know (I didn't write the DLL, it was written by someone else), but it is a cause of the problem because it will never process the method called queued by synchronize.
I mentioned earlier that if i call the function that performs the long process from my thread using synchronize then the application does not dead lock. This is because the main thread will be responsible for calling the function that performs the long process. So the long process function calls a DLL function which starts another thread and then calls WaitFor. WaitFor checks to see if the current thread is the main thread, and if it is, it processes method calls which have been queued by synchronize, continuously in a loop until the thread the thread that it is waiting for is released (i.e. the method it queued via synchronize gets called and the wait event is signaled).
In WaitFor, if the current thread is not the main thread then WaitFor simply blocks until the thread it is waiting for is released.
Anyway i can't do anything about the application object in the dll because that dll is quite complex and used by a larger system. I guess i can expose a method in the dll which can process the methods in the synchronization queue, i can then call this method from my application while its idle.
Anyway again thanks for your help but i've solved this problem now.
Using the TThread class or even the Application object in a Delphi DLL is extremely unsafe. The RTL and VCL core classes, globals and singleton objects are designed to exist once per process and can't handle the namespace duplication and incomplete initialization in a stadard DLL library.
You may get around it by building with runtime packages (RTL & VCL are enough; you may also build your own with just the system units you reallt need), in the EXE and all DLLs referencing the runtime units (Forms and Classes, especially) - they get single shared namespace and the full EXE initialization sequence this way.
If you can't modify the DLL at all, you may try to set it's Application.Handle, MainThreadID, SyncEvent and WakeMainThread to the corresponding values in the main EXE module - this may work, but it's just as ugly as it looks like and it doesn't cover all edge-cases (the classes and important globals will still be duplicated).