I have doubt about multithreaded application(ip scanner). When i put large ip range like 192.168.0.1 to 192.168.5.1 and thread limit as 99 .So when i run my application there should be 101 threads running at a time(99 threads(ScannerChild) + Main thread + Scannerthread) and when scanning is done 99 scannerchild and 1 scannethreads would be terminated and only 1 thread should run that time(main thread). But sometimes thread count is going to 102 and after scanning thread count is not coming to 1 it shows threadcount as 2 in task manager. Whats the problem ?
code for Scannerthread
/
/Creating constructor of scannerthread
Constructor ScannerThread.Create(CreateSuspended: Boolean );
Begin
Inherited Create(CreateSuspended);
Freeonterminate:= true; //Freeonterminate is true
End;
{ScannerThread Thread }
procedure ScannerThread.Execute;
var
I : integer;
ScannerCh : array of ScannerChild; //array of ScannerChild
IpList : TStringlist; //Iplist as tstringlist
IPs: Integer; //ipcount is count of iplist
Begin
ScannerchCount:=0; //Initialising scannerchcount as 0
IpList:=TStringList.Create;//creating stringlist
IF GetNumberOfIpsInRange(Ip_From, Ip_To, IpList) Then //Function call that returns iplist if TRUE
Begin
Try
IF Assigned(LvHosts) Then //Clearing LvHosts field
LvHosts.Clear;
IPs := IpList.Count; //Ipcount is given value of iplists count
SetLength(ScannerCh, IPs); //Setting length of scannerch as ipcount
I:=0;
Repeat
While ScannerChcount > tcount-1 do //Checking if is greater than tcount(thread input) by user
Sleep(30);
ScannerCh[I]:=ScannerChild.Create(True, IpList[i]);
ScannerCh[I].FreeOnTerminate:=True;
ScannerCh[I].OnTerminate:= ScanchildTerminated; // Event scanchildterminated occurs on termination of Scannerch thread
ScannerCh[I].LvHostname := LvHosts; //Lhostname is private listview of scannechild
ScannerCh[I].Resume;
ScannerChCount:=Scannerchcount+1; //Incrementing scannerchcounts
I:=I+1;
Sleep(20); //Sleep after each thread is created so that threads will enter critical section properly
until I = IPs;
Scannerch:=nil;
If Assigned(IpList) Then //Free iplist
FreeAndNil(IpList);
Except
On E: Exception do
Begin
ShowMessage('Invalid operation :' + E.Message); //Showexception message
If Assigned(IpList) Then //Free iplist
FreeAndNil(IpList);
end;
End;
End
Else
Begin
Ipscan.lbResult.caption:='Invalid Ip Range';
Exit;
End;
Repeat //Main Thread Waiting For Ip scan Threads to finish
Sleep(100);
until ScannerChCount = 0;
End;
Scannerchild code
Constructor ScannerChild.Create(CreateSuspended: Boolean; IP: String);
Begin
Inherited Create(CreateSuspended);
//FCriticalsection := TCriticalSection.create; //Creating critical section
IPToScan:=IP;
End;
//Execution procedure for scannerchild
procedure ScannerChild.Execute;
Var
MainOutput : TListItem;//Listitem variable for adding listitems
Hostname : String; //Hostname is declared as string
Begin
Try
MainOutput:=LvHostname.Items.Add; //Adding items to mainoutput
MainOutput.Caption:=IPToScan;
Hostname := IPAddrToName(IPToScan);
If Hostname <> EmptyStr Then
Begin
MainOutput.SubItems.Add(IPAddrToName(IPToScan)); //Displaying output
End
Else
Mainoutput.subitems.add('No host');
Finally
End;
End;
//this event get called when scannerch thread terminates
procedure Scannerthread.ScanchildTerminated( Sender : TObject );
Begin
ScannerChCount:=ScannerchCount-1; // Decrementing scannerchcount
End;
There are plenty of problems here. I'm going to give you some general advice as well as try to answer your question.
Your accessing of the GUI outside of the main thread is wrong, as we've said before. No need to cover that again, please go back over your previous questions.
The design of your threading is poor. If you would ask a high level question about that we could help you fix it. I'd be happy if you asked a question that allowed me to demonstrate a simple thread pool.
As well as the problems with the threading design, you've got no separation of concerns. No modularity. The threading and the tasks and the GUI code are all mixed in with each other. You need to keep the concerns separate to make the code maintainable and well factored. If you'd only ask us how to design your program rather than to fix the bugs in your weak design we could help you.
All of the calls to Sleep and the polling are symptoms of this bad design. There should be no sleeping.
Your code has way too many comments that make it hard to read. There's no need to comment a statement like i := i+1. The effect of that is self-evident.
You need to learn how to debug threaded code. The interactive debugger is not so useful. It interferes with the timing of thread execution. Use trace logging to debug such problems. Until you learn how to do this you cannot expect to make progress. I repeat, it is critical that you learn how to debug.
As to the problem you asked about, you have a data race on the ScannerChCount variable. So the threads are probably terminating correctly but you are counting them incorrectly.
Use InterlockedIncrement and InterlockedDecrement to modify it in a thread safe manner. That is both in the child termination code and the controller thread.
You might think this is not needed because ScanChildTerminated which decrements the counter is an OnTerminate event and so executed by the main thread. But the controller thread code that increments the counter is not executed in the main thread.
If you don't yet know what a data race is, then you have started multi threaded programming too soon. Rather than my explain it I would refer you to the shared data sections of any good text book on parallel programming. Or Wikipedia: http://en.m.wikipedia.org/wiki/Race_condition.
I want to write from multiple threads/processes to a TListBox called 'listMessages' and I have this two procedures in order to do this :
1- With adding object :
procedure Log(Msg: String; Color: TColor);
begin
listMessages.Items.AddObject(Msg, Pointer(Color));
listMessages.ItemIndex := listMessages.Items.Count -1;
end;
2- With TIdCriticalSection called protectListMessages :
procedure TMainForm.safelyLogMessage(mess : String);
begin
protectlistMessages.Enter;
try
listMessages.Items.Add(mess);
listMessages.ItemIndex := listMessages.Items.Count -1;
finally
protectListMessages.Leave;
end;
end;
Can you tell me which is best(fast + thread safe) or show me a third way to write messages to my TListBox from my threads/processes ?
Neither of your options is any good. You need to use option 3!
The point is that all access to UI controls must execute on the main thread. Use TThread.Synchronize or TThread.Queue to marshal UI code onto the main UI thread. Once you do this, the code will not need any further serialization because the very act of getting it to run on the UI thread serializes it.
The code might look like this:
procedure TMainForm.Log(const Msg: string; const Color: TColor);
var
Proc: TThreadProcedure;
begin
Proc :=
procedure
begin
ListBox1.AddItem(Msg, Pointer(Color));
ListBox1.ItemIndex := ListBox1.Count-1;
end;
if GetCurrentThreadId = MainThreadID then
Proc()
else
TThread.Queue(nil, Proc);
end;
In your update you state that you need to write to the list box from a different process. This cannot be achieved with any of the code in the question. You need inter-process communication (IPC) for that. Sending windows messages would be a reasonable approach to take, but there are other IPC options available. But I think that you mis-speak when you use the term process. I suspect that you don't mean process, but what you do mean, I have no idea.
I want to write from multiple threads/processes to a TListBox called 'listMessages' and I have this two procedures in order to do this :
1- With adding object :
procedure Log(Msg: String; Color: TColor);
begin
listMessages.Items.AddObject(Msg, Pointer(Color));
listMessages.ItemIndex := listMessages.Items.Count -1;
end;
2- With TIdCriticalSection called protectListMessages :
procedure TMainForm.safelyLogMessage(mess : String);
begin
protectlistMessages.Enter;
try
listMessages.Items.Add(mess);
listMessages.ItemIndex := listMessages.Items.Count -1;
finally
protectListMessages.Leave;
end;
end;
Can you tell me which is best(fast + thread safe) or show me a third way to write messages to my TListBox from my threads/processes ?
Neither of your options is any good. You need to use option 3!
The point is that all access to UI controls must execute on the main thread. Use TThread.Synchronize or TThread.Queue to marshal UI code onto the main UI thread. Once you do this, the code will not need any further serialization because the very act of getting it to run on the UI thread serializes it.
The code might look like this:
procedure TMainForm.Log(const Msg: string; const Color: TColor);
var
Proc: TThreadProcedure;
begin
Proc :=
procedure
begin
ListBox1.AddItem(Msg, Pointer(Color));
ListBox1.ItemIndex := ListBox1.Count-1;
end;
if GetCurrentThreadId = MainThreadID then
Proc()
else
TThread.Queue(nil, Proc);
end;
In your update you state that you need to write to the list box from a different process. This cannot be achieved with any of the code in the question. You need inter-process communication (IPC) for that. Sending windows messages would be a reasonable approach to take, but there are other IPC options available. But I think that you mis-speak when you use the term process. I suspect that you don't mean process, but what you do mean, I have no idea.
There is a multi-threaded app, which runs 24/7. A correct disposal of resources as well as a proper exceptions handling (including EAccessViolation) are key factors.
I am a little bit stuck on understanding how to correctly nest exception-handling blocks within a thread functions.
Among TMyThread.Execute there are two helper-function:
function LoadHtml(const AUrl: sting): string - a simple wrapper for TIdHTTP.Get()
function ParsePage(const Id: string): TOffers - a parser/DB-updater function
Execute starts querying database for the initial recordset of IDs. Then it starts a while not rs.Eof do loop where calls ParsePage which is a main processor.
ParsePage loads HTML (care of LoadHtml), then performs some string parsing operations and finally updates database.
Here is a code structure: (details omitted for brevity)
{Wrapper-function to load HTML page}
function TMyThread.LoadHtml(const AUrl: string): string;
var
Response: TStringStream;
HTTP: TIdHTTP;
begin
Result := '';
Response := TStringStream.Create('');
try
try
HTTP := TIdHTTP.Create(nil);
HTTP.ReadTimeout := 10000;
HTTP.Response.KeepAlive := false;
try
HTTP.Get(AUrl, Response);
if HTTP.ResponseCode = 200 then Result := Response.DataString;
finally
HTTP.Free;
end;
finally
Response.Free;
end;
except
//This code will run only on exception and *after* freeing all resources?
on E: EIdHTTPProtocolException do
if E.ErrorCode = 404 then
raise EMyOwnHTTPNotFoundError.Create('Page not found');
else
HandleErrorAndLogItToDB(E.Class);
end;
end;
{Loads HTML, processes it and updates DB}
function TMyThread.ParsePage(const Id: string): TOffers;
var
RawHTML: string;
Offer: TOffer; //a simple record to store key offer details;
begin
Result := TOffers.Create;
try {top-level try..except block}
try {Critical request. If it fails I want to move}
RawHTML := LoadHtml('http://onlinetrade.com/offer.html?id=' + Id);
except
on E: EMyOwnHTTPNotFoundError do {Defined in function LoadHtml()}
//Update DB: product does not exist.
else
HandleErrorAndLogItToDB(E.Class);
end;
end;
try
//Preform some basing string operations on RawHTML
except
on E: Exception do HandleErrorAndLogItToDB(E.Class);
end;
try {Iterate through some blocks of data and put them in the Offers: TList}
for i := 0 to N do
begin
//Set up TOffer record
Result.Add(Offer);
end
finally
FreeAndNil(Offer);
end;
except
on E: Exception do
begin
HandleErrorAndLogItToDB(E.Class);
FreeAndNil(Result);
raise; {does this return control to Execute?}
end;
end;
end;
Now Execute:
procedure TMyThread.Execute;
var
j: Integer;
s: string;
Offers: TOffers; {Is a simple TList to store a collection of TOffer (record)}
begin
inherited;
CoInitialize(nil); {ADO is in da house}
try {top-level try..except block}
try {nested try..finally to call CoUninitialize}
try {A critical operation which sources all further operations}
rs := AdoQuery('GetSomeRecords ' + IntToStr(SomeId));
except
on E: Exception do
begin
HandleErrorAndLogItToDB(E.Class);
Exit; {DB-query error means no reason to continue}
end;
end;
while not rs.EOF do
begin
try //a loop top-level try..except handler
Offers := ParsePage(rs.Fields['Id'].Value);
try //nested resource freeer
begin
try //nested try..except to handle DB queries
for j := 0 to N do with Offers.Items[j] do
AdoUpdateDB; //Update DB
Synchronize(UpdateProgressBar);
except
on E: Exception do
begin
HandleErrorAndLogItToDB(E.Class);
Continue; //as suggested
raise; //as suggested
end;
end;
rs.MoveNext;
end;
finally
FreeAndNil(Offers);
end;
except
on E: Exception do HandleErrorAndLogItToDB(E.Class);
end;
end; //end while..do loop
Synchronize(ResetProgressBar);
finally
CoUnitialize;
end;
except
on E: Exception do
begin
//Make everything possible to keep the thread running. No matter of:
//- HTTP/404 - Not Found exceptions (which I handle)
//- UpdateDatabase fails
//- String operation exceptions
//If anything critical occurs, Execute() shall just go to the next offer
//even if the current one is not properly processed.
end;
end;
Looking at this code I think, I try to handle too many exceptions which I probably do not need to handle, just passing them to the most outer try..except handler in Execute. There are just a couple of exceptions I really need to handle: initial database query and EMyOwnHTTPNotFoundError (to set a flag that the offer doesn't exist). I read somewhere a suggestion not to explicitly chase for exception handling unless you really need it...
However it is very important that the thread keeps on running no matter of which exceptions are thrown inside/outside any code blocks. The idea is to completely ignore exceptions and never break either while..do loop or stop the thread. At the same time, correctly disposing resources is also a must.
I would be grateful for any suggestions/comments on how to improve this code.
There is no good way to "handle" an access violation, or really any exception that does not indicate a specific condition that your code has already planned for. (For example, a File Not Found exception can be handled in a good way if you can simply tell the user to ask for another file.)
If an exception is raised as the result of a bug, it means something is happening in your code that you did not plan for. Your code rests on a bunch of assumptions about things going right, things working as planned, and when an unexpected exception is raised, it means that those assumptions no longer necessarily hold. The best thing to do at that point is produce an error report to send back to you, and then shut down as quickly as possible.
Why? Because one of your assumptions that may no longer hold is that "critical data is in a valid, non-corrupted state." If the program keeps going, blindly following the assumption that all its data is good and then acting on it, it can turn a small problem into a much bigger one very, very quickly.
I completely understand the desire to make the program keep on going no matter what, but unfortunately it fundamentally conflicts with reality. The only sane thing to do when you get an unhandled exception--especially something like an access violation that can only be the result of buggy code of some variety--is to produce an error report and shut down.
If downtime is a Very Bad Thing, you could have something in place to make sure to start back up again as quickly as possible. This will keep you running, but it will reset your invariants (fundamental assumptions) and clear out the corrupt data. But for the love of all that is binary, shut the program down, and do it right away.
Then take the error report and fix your bug.
In general, is it possible in a TThread.Execute procedure
to call a TDataModule method, in which there is no visual activity involved?
Thanks to all, Massimo.
The easiest way to go is to use TThread.Synchronize to invoke a method in your data module.
However, if you do not wish to do that, even when no visual activity is involved, you should determine whether or not you need to add a critical section to protect you.
Any access to any standard or third-party VCL component, whether it is visual (TButton) or non-visual (datasets) should be considered UNSAFE. Any access to a local data object (like a private field or global variable) must also be protected by critical sections.
Here's a direct call from a from background thread to your data module:
if Assigned(MyDataModule) then MyDataModule.DoSomething(a,b,c);
Here's the code in your data module, which I am showing you a sample bit of code that makes sure that we are the only thread touching FList right now:
/// DoSomething: Note this method must be thread-safe!
procedure TMyDataModule.DoSomething(a:TMyObject1;b:TMyObject2;c:TMyObject3);
begin
FCriticalSection.Enter;
try
if not FList.Contains(a) then
FList.Add(a);
...
finally
FCriticalSection.Leave;
end;
end;
/// elsewhere in the same data module, wherever anybody modifies or checks the state
/// (content) of FList, wrap the method with a critical section like this:
function TMyDataModule.HasItem(a:TMyObject1):Boolean;
begin
FCriticalSection.Enter;
try
result := FList.Contains(a);
finally
FCriticalSection.Leave;
end;
end;
Some starter rules for Delphi multi-threaded programming, in a nutshell are:
Don't do anything that could create a Race Condition.
Don't forget to use synchronization primitives like Critical Sections, Mutexes, etc, to protect against concurrency issues including Race Conditions, whenever you are accessing any data fields in your class (data module) or ANY globals. If you use these improperly you add deadlocks to your list of problems. So this is not a good place to mess up.
If you must access a VCL component or object in any way, do so indirectly via PostMessage, TThread.Synchronize, or some other thread-safe equivalent way of signaling the main thread that you need something done.
Think about what happens when you're shutting down. Maybe you could check if your data module even exists, since it might have gone away, before you invoke its methods.
Short answer: yes
Long answer: The problem with Windows is that all the GUI activity should be done in a single thread. (Well, the above statement can be expanded, amended, enhanced etc. but for our discussion is enough). So, if you are sure that in your TDataModule method there isn't any 'GUI thing' involved (beware, this can be even a ShowMessage call) then go ahead.
UPDATE: Of course, there are techniques to update your GUI from a secondary thread, but this implies some sort of preparation (message passing, Synchronize etc.). Isn't something very hard, just that you cannot 'blindly' call from another thread a method who changes the GUI.
To use our industries favorite answer when asked anything: It depends.
If you have a method on your datamodule that is completely self contained (ie could be a static method), you shouldn't have any problem.
Example
TMyDataModule = class(TDataModule)
public
function AddOne(const Value: Integer): Integer;
end;
function TMyDataModule.AddOne(const Value: Integer): Integer;
begin
Result := Value + 1;
end;
If on the other hand, the method uses any global state, you might get into trouble when calling it from multiple threads.
Example
TMyDataModule = class(TDataModule)
private
FNumber: Integer
public
function AddOne(const Value: Integer): Integer;
end;
function TMyDataModule.AddOne(const Value: Integer): Integer;
begin
FNumber := Value
//***** A context switch here will mess up the result of (at least) one thread.
Result := FNumber + 1;
end;
Global state should be interpreted very wide. A TQuery, a TTable, refreshing the GUI, using any global variable, ... is all global state and isn't thread safe.
Yes, my question is very vague.
My program is a graphical statistics app, it has to display Gantt chart, by means of TChart, describing the states, alarms or machined orders of one or more Tool Machine.
On the supervisor PC a server (equipped with a TIdTcpServer and some DB components)
is listening to my app on the LAN.
The main-form client allows the final user to choice a range of dates (period) and
the units (machines) to query the server. After that, the user press a button (there are
3 functionalities): a new form (and Datamodule) is created to display the results.
The work of collecting data is completed by a thread because:
1) it can be a long job so it could freeze the GUI;
2) the user can launch more than one form to see various results.
I have a basic Datamodule (with a TIdTcpClient with several function to collect the data),
a basic form (never instantiated, with a lot of characteristics common to all data form, and the definition of the worker thread).
unit dtmPDoxClientU;
TdtmPDoxClient = class(TDataModule)
IdTCPClient: TIdTCPClient;
...
function GetData(...): boolean;
...
end;
unit frmChartBaseFormU;
TfrmChartBaseForm = class(TForm)
...
TheThread: TThreadClient;
procedure WMThreadComm(var Message: TMessage); message WM_THREADCOMM;
procedure ListenThreadEvents(var Message: TMessage); virtual;
procedure ExecuteInThread(AThread: TThreadClient); virtual;
end;
TThreadClient = class(TThread)
private
public
Task: integer;
Module: TfrmChartBaseForm;
procedure Execute; override;
property Terminated;
end;
procedure TfrmChartBaseForm.FormCreate(Sender: TObject);
...
TheThread := TThreadClient.Create(true);
with TheThread do begin
Module := self;
FreeOnTerminate := true;
end;//with
end;//FormCreate
procedure TfrmChartBaseForm.WMThreadComm(var Message: TMessage);
begin
ListenThreadEvents(Message);
end;//WMThreadComm
procedure TfrmChartBaseForm.ListenThreadEvents(var Message: TMessage);
begin
// do override in derived classes
end;//ListenThreadEvents
procedure TfrmChartBaseForm.ExecuteInThread(AThread: TThreadClient);
begin
// do override in derived classes
end;//ExecuteInThread
procedure TThreadClient.Execute;
begin
with Module do begin
ExecuteInThread(self);
end;//with
end;//Execute
Furthermore, using VFI, I also have two units:
unit dtmPDoxClientDataOIU;
TdtmPDoxClientDataOI = class(TdtmPDoxClient)
cdsClient_IS: TClientDataSet;
...
dsr_I: TDataSource;
...
private
public
end;
unit frmPDoxClientDataOIU;
TfrmPDoxClientDataOI = class(TfrmChartBaseForm)
ChartOI: TChart;
...
procedure FormCreate(Sender: TObject);
public
{ Public declarations }
dtmPDoxClientDataOI: TdtmPDoxClientDataOI;
procedure ListenThreadEvents(var Message: TMessage); override;
procedure ExecuteInThread(AThread: TThreadClient); override;
end;
procedure TfrmPDoxClientDataOI.FormCreate(Sender: TObject);
begin
inherited;
dtmPDoxClientDataOI := TdtmPDoxClientDataOI.Create(self);
TheThread.Task := 1;
TheThread.Resume;
end;//FormCreate
procedure TfrmPDoxClientDataOI.ListenThreadEvents(var Message: TMessage);
begin
if (Message.WParam = 1) then begin
case Message.LParam of
//GUI tasks, using ClientDataset already compiled and not re-used
end;//case
end;//if
end;//ListenThreadEvents
procedure TfrmPDoxClientDataOI.ExecuteInThread(AThread: TThreadClient);
begin
while not AThread.Terminated and (AThread.Task <> 0) do begin
case AThread.Task of
1: begin
if dtmPDoxClientDataOI.GetData(...) then
if not AThread.Terminated then begin
PostMessage(Handle,WM_THREADCOMM,1,1);
AThread.Task := 2;
end //if
else
AThread.Task := 0;
end;//1
... etc...
end;//case
end;//while
end;//ExecuteInThread
So, when the final user presses the button, a new form and its own datamodule and
thread are created; the thread uses its own datamodule by means of ExecuteInThread
function. When data are ready, a PostMessage is sent to the form, which updates
the chart.
Like Lieven writes, it depends.
If you have database components on the datamodule, you have to know if the are thread safe, or to make them threadsafe.
Some database components require a seperate session object per thread.
There is a problem where you work with datamodule in Thread:
If you terminate your thread in OnDestroy event of form and are waiting for it (WaitFor) - you'll have a deadlock.
Main UI thread set lock
procedure TCustomForm.BeforeDestruction;
begin
GlobalNameSpace.BeginWrite;
and your thread will wait infinitely in it's datamodule destructor with the same
destructor TDataModule.Destroy;
begin
if not (csDestroying in ComponentState) then GlobalNameSpace.BeginWrite;
So, if you want to wait for your threads when close MainForm, do it in OnClose event or in Project's main file
Or you can destroy it in Synchronize