DELPHI: Multithreaded client/server datasnap error - multithreading

This is my first post here - so be gentle :-)
I want to build a client/server application that uses datasnap for data transport.
This is a fairly simple task - and there are lots of examples to learn from.
BUT - Having a Datasnap server (build from Delphi XE wizard) I find myself running into a problem, and I hope someone can guide me into the right direction.
Server and Client run on same PC (that is the design for now).
Server is running Session lifecycle.
Server and Client shares a class (posted below)..
The Server provides a simple method - GetServerObject which uses the GetNewObject method.
The Server itself is a VCL application - main form is fmServer.
OnCreate instatiates the Servers FormObject property (FormObject := TMyDataObject.Create);
function TServerMethods2.GetNewObject: TMyDataObject;
begin
Result := TMyDataObject.Create;
end;
function TServerMethods2.GetServerObject: TMyDataObject;
begin
Result := GetNewObject;
if not Result.Assign(fmServer.FormObject) then
raise Exception.Create('Server error : Assign failed!');
end;
All this is pretty trivial - and my problem only appears if I twist my Client application into a multithreaded monster :-) (read - more than 1 thread).
So here is the Thread code for the client.
TDataThread = class(TThread)
private
DSConn: TSQLConnection;
protected
procedure Execute; override;
public
constructor Create(aConn: TSQLConnection); overload;
end;
constructor TDataThread.Create(aConn: TSQLConnection);
begin
inherited Create(False);
DSConn := aConn.CloneConnection;
FreeOnTerminate := true;
end;
procedure TDataThread.Execute;
var
DSMethod: TServerMethods2Client;
aDataObject : TMyDataObject;
begin
NameThreadForDebugging('Data');
{ Place thread code here }
DSMethod := nil;
try
while not terminated do
begin
sleep(10);
if DSConn.Connected then
begin
try
if DSMethod = nil then
DSMethod := TServerMethods2Client.Create(DSConn.DBXConnection,false);
if DSMethod <> nil then
try
aDataObject := DSMethod.GetserverObject;
finally
freeandnil(aDataObject);
end;
except
freeandnil(DSMethod);
DSConn.Connected := False;
end
end
else
begin
// connect
try
sleep(100);
DSConn.Open;
except
;
end;
end;
end;
finally
freeandnil(DSMethod);
DSConn.Close;
freeandnil(DSConn);
end;
When I create more than 1 of these threads - eventually I will get an error (being "cannot instatiate ... " or some "remote dbx error ..." .. and so on.
I simply cannot get this to work - so that I can spawn hundreds of threads/connections to a datasnap server.
I know this question is tricky - but my hope is that someone is smarter than me :-)
If I try the same client thread code - but accessing a more simple server method (lets say echostring from sample) then I can run it with hundreds of threads.
Perhaps Im answering myself here - but Im too blind to realize it :-)
unit uDataObject;
interface
uses
SysUtils;
Type
TMyDataObject = class(TObject)
private
fString: String;
fInteger: Integer;
public
constructor Create; virtual;
destructor Destroy; override;
function Assign(aSource: TMyDataObject): boolean;
property aString: String read fString write fString;
property aInteger: Integer read fInteger write fInteger;
end;
implementation
{ TMyDataObject }
function TMyDataObject.Assign(aSource: TMyDataObject): boolean;
begin
if aSource <> nil then
begin
try
fString := aSource.aString;
fInteger := aSource.aInteger;
Result := True;
except
Result := false;
end;
end
else
Result := false;
end;
constructor TMyDataObject.Create;
begin
inherited;
Randomize;
fString := 'The time of creation is : ' + FormatDateTime('ddmmyyyy hh:nn:ss:zzz', Now);
fInteger := Random(100);
end;
destructor TMyDataObject.Destroy;
begin
inherited;
end;
end.
All help is appreciated

This has mostly been answered in the comments and the bug report, but... The problem you are seeing is caused by a multithreading issue in XE's marshaller code. If two threads (or two clients) call a server server method which takes in or return user defined types (any type which will use the marshaller/unmarshaller) at the same time, then an exception could happen.
I am unaware of a perfect workaround for XE, but if it is possible to not use user-defined types, then you shouldn't see multithreading issues.
Mat

When the simple server method is working, i think your problem has to be found i somethin the "real" code is doing or using.
It could be in the connection (try changing your simpler code to use the connection)
Your problem can also be then CloneConnection. The Cloned connection is freed, when the connection it is cloned from is freed. See
http://docwiki.embarcadero.com/VCL/en/SqlExpr.TSQLConnection.CloneConnection

Related

How do I get a boolean result from a function using OmniThreadLibrary?

I have a Delphi (Windows) application created using Delphi 10 that has some blocking calls that I would like to convert to using threads. Unfortunately for me, these are not procedures, but functions. (And information on how to return function results from threads appears to be much more limited.) I am trying to familiarize myself with the OmniThreadLibrary, since it seems to be the most flexible and best supported threading library for Delphi, but I'm having trouble with understanding how to do this.
I have been able to get the various OmniThreadLibrary routines to work well with procedures, but when I try to set up a function, I get an error about capturing the result. When I use OmniThreadLibrary's Future example as a starting point, I can get the function to work, but I can't figure out how to connect to the event monitor, how to send messages from the task, etc. So, it seems as if I'm overlooking something no matter which way I try to solve this problem.
Currently, my program does something like this:
If myPing(IPAddress) then
begin
//Do other things hereā€¦
end;
Because myPing is blocking, and I actually need it to wait until myPing returns true before processing further, the application gets sluggish during this process. I'd like to put the myPing call in a thread, which would solve the sluggishness problem, but I can't figure out how to do that in the form of a function using OmniThreadLibrary. (Unless I use a future, in which case I can't figure out how to connect to the Event Monitor.)
Edit 1: Since my original post, I have made a little progress. I was able to connect the Event Monitor to the Future by adding Parallel.TaskConfig.MonitorWith(Form1.OmniEventMonitor1) to my code, right after the function. However, I still can't figure out how to send messages to that event monitor from within the Future function.
Edit 2: I now have some sample code. My first attempt was similar to this:
function myPing(HostName: string): IOmniFuture<boolean>;
begin
Result := Parallel.Future<boolean>(function: boolean
begin
Result := False;
//Do actual ping here... Set Result := True if successful.
end
);
end;
The basic function worked, but did not allow me to send any messages to the TOmniEventMonitor. I was able to get that part working by changing the code to this:
function myPing(HostName: string): IOmniFuture<boolean>;
begin
Result := Parallel.Future<boolean>(function: boolean
begin
Result := False;
//Do actual ping here... Set Result := True if successful.
end,
Parallel.TaskConfig.MonitorWith(Form1.OmniEventMonitor1)
);
end;
Now, I can successfully monitor the OnTaskTerminated event, but I still can't send messages to the Event Monitor from the task. By changing my code once again, I can access the task itself and send messages using task.Comm.Send(), but the messages don't reach the EventMonitor:
function myPing(HostName: string): IOmniFuture<boolean>;
begin
Result := Parallel.Future<boolean>(function(const task: IOmniTask): boolean
begin
Result := False;
//Do actual ping here... Set Result := True if successful.
task.Comm.Send(0,'Test 1');
end,
Parallel.TaskConfig.MonitorWith(Form1.OmniEventMonitor1)
);
end;
Here's a simple example on how to retrieve the function result from the async call. It does not use an "OmniEventMonitor" but instead calls a function once the async call returns ("Ping" is defined in PingU.pas, but not of importance here):
unit MainFormU;
interface
uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, StdCtrls, Menus;
type
TPingResultEvent = procedure (const bResult: Boolean) of object;
TOnTerminateTestForm = class(TForm)
LogMemo: TMemo;
MainMenu: TMainMenu;
PingMenu: TMenuItem;
procedure PingMenuClick(Sender: TObject);
private
procedure BackgroundPing (const sServer: String;
const OnResult: TPingResultEvent);
procedure PingResult (const bResult: Boolean);
public
{ Public declarations }
end;
var
OnTerminateTestForm: TOnTerminateTestForm;
implementation
{$R *.dfm}
uses PingU, OtlParallel, OtlTaskControl;
procedure TOnTerminateTestForm.PingMenuClick (Sender: TObject);
var
sServer : String;
begin
if (InputQuery ('Ping computer', 'Computer name:', sServer)) then
if (sServer <> '') then
begin
PingMenu.Enabled := false;
LogMemo.Lines.Add (Format ('Pinging %s',[sServer]));
BackgroundPing (sServer, PingResult);
end; { if }
end; { TOnTerminateTestForm.PingMenuClick }
procedure TOnTerminateTestForm.BackgroundPing (const sServer: String;
const OnResult: TPingResultEvent);
var
bResult : Boolean;
begin
Parallel.Async (
procedure
begin
bResult := Ping (sServer);
end,
Parallel.TaskConfig.OnTerminated(
procedure (const task: IOmniTaskControl)
begin
// executed in main thread after the async has finished
if Assigned (OnResult) then
OnResult (bResult);
end
)
);
end; { TOnTerminateTestForm.BackgroundPing }
procedure TOnTerminateTestForm.PingResult (const bResult: Boolean);
begin
PingMenu.Enabled := true;
LogMemo.Lines.Add ('Ping result = ' + BoolToStr (bResult, true));
end; { TOnTerminateTestForm.PingResult }
end.
Code source: Get a function result asynchronously in Delphi using Omni Thread Library

Overbyte NamedPipes; Send msg on a different thread

I'm fairly new to IPC(interprocess communication). Doing my research, I decided on the Named pipes.
My application consists of 2 parts: a monitoring app, and a UI dashboard. The dashboard receives updates from the monitor constantly and shows stats, the user should be able to change certain parameters of the monitor through the dashboard (refresh rate, restart process,scheduled task...) so it has to be a bidirectional communication. The Monitor would later become a service application, but that's a plan for later.
Getting to the point, I used the Pipes unit by Overbyte(Francois Piette) v1.01
and followed the example made on their website. When connecting the pipes on the main thread, it functions normally. But due to my monitor needing a separate thread to (monitor-send message-sleep-loop...), when I try to connect on the other thread, I get an error "The notify window and the component window do not exist in the same thread!"
What I need to know is, can named pipes communicate using a separate thread? (Judging by the error msg, I feel it may be fetching the window handle on the main UI and would not work on a different thread)
Is there a better way to implement my application? or named pipes better than Overbyte?
Sample of my code:
TThreadMonitor = Class(TThread)
private
PipeClient1: TPipeClient;
listOfProcesses: Array of String;
protected
procedure Execute; override;
public
constructor Create(CreateSuspended: Boolean);
procedure ConnectPipe;
procedure SendMessage(const Msg: String);
End;
var
t: TThreadMonitor;
procedure TClientFormMain.BtnDifferentThreadClick(Sender: TObject);
begin
t := TThreadMonitor.Create(TRUE);
t.FreeOnTerminate := TRUE;
t.Start;
end;
procedure TThreadMonitor.ConnectPipe;
begin
if not PipeClient1.Connect(2000, TRUE) then
LogThis('Pipe connection failed', LogFilePath, TRUE)
else
LogThis('Pipe connected', LogFilePath, TRUE);
end;
procedure TThreadMonitor.Execute;
begin
inherited;
ConnectPipe; //<---throws exception here although getting "Pipe Connected"
while not Terminated do
begin
for i := 0 to Length(listOfProcesses) - 1 do
begin
...
MonitorProcess(listOfProcesses[i]);
...
end;
sleep(2000);
end;
end;

Delphi : How to create and use Thread locally?

My database is in a VPS and I should get some query from my tables
Because of getting query from server taking long time ( depending on Internet speed ! ) , I want to use threads to get queries
Now I create a thread and get query and then send result to my forms with sending and handling messages
I want to know is it possible to create and use a thread locally ?!?
My mean is :
procedure Requery;
var
...
begin
Create Thread;
...
Pass my Query Component to Thread
...
Getting Query in Thread;
...
Terminate and Free Thread
...
Do next jobs with Query;
...
end;
The main part is last part ( Do next jobs ... ) , I dont want to use query result in a message handler and I want to use them in the same procedure and after thread job
Is it possible ?!
I think this is not possible with Delphi TThread class and I should use other threading techniques ...
I`m using Delphi XE6
What you describe is not the best use of a thread. The calling code is blocked until the thread is finished. That negates the use of running code in parallel at all. You could just perform the query directly instead:
procedure Requery;
var
...
begin
...
// run query
// do next jobs with query
...
end;
That being said, since you are using XE6, you can create a "local" thread by using the TThread.CreateAnonymousThread() method, specifying an anonymous procedure that "captures" the variables you want it to work with, eg:
procedure Requery;
var
Event: TEvent;
H: THandle;
begin
Event := TEvent.Create;
try
TThread.CreateAnonymousThread(
procedure
begin
try
// run query in thread
finally
Event.SetEvent;
end;
end
).Start;
H := Event.Handle;
while MsgWaitForMultipleObjects(1, H, False, INFINITE, QS_ALLINPUT) = (WAIT_OBJECT_0+1) do
Application.ProcessMessages;
finally
Event.Free;
end;
// Do next jobs with query
...
end;
Alternatively:
procedure Requery;
var
Thread: TThread;
H: THandle;
begin
Thread := TThread.CreateAnonymousThread(
procedure
begin
// run query in thread
end
);
try
Thread.FreeOnTerminate := False;
H := Thread.Handle;
Thread.Start;
while MsgWaitForMultipleObjects(1, H, False, INFINITE, QS_ALLINPUT) = (WAIT_OBJECT_0+1) do
Application.ProcessMessages;
finally
Thread.Free;
end;
// Do next jobs with query
...
end;
However, threading is more useful when you let it run in the background while you do other things and then you act when the thread has finished its work. For example:
procedure TMyForm.Requery;
var
Thread: TThread;
begin
Thread := TThread.CreateAnonymousThread(
procedure
begin
// run query in thread
end
);
Thread.OnTerminate := QueryFinished;
Thread.Start;
end;
procedure TMyForm.QueryFinished(Sender: TObject);
begin
if TThread(Sender).FatalException <> nil then
begin
// something went wrong
Exit;
end;
// Do next jobs with query
end;
I think that using a thread this way isn't a good idea, but the answer is yes. You can do it.
procedure LocalThread;
var
LThread: TCustomThread; //Your thread class
LThreadResult: xxxxxxx//Your result type
begin
LThread := TCustomThread.Create(True);
try
//Assign your properties
LThread.Start;
//Option A: blocking
LThread.WaitFor;
//Option B: non blocking
while not LThread.Finished do
begin
Sleep(xx);
//Some progress here ??
end;
//Here query your thread for your result property
LThreadResult := LThread.MyResultProperty;
finally
LThread.Free;
end
//Do next jobs with LThreadResult
end;
Yes you can do that.
The way I would do it is to add an event-handler to your form.
You'll have to link the event-handler in code, but that's not that difficult.
Create a thread like so:
TMyEventHandler = procedure(Sender: TObject) of object;
type
TMyThread = class(TThread)
strict private
FDoneEvent: TMyEvent;
FDone: boolean;
FQuery: TFDQuery;
constructor Create(DoneEvent: TMyEventHandler; Query: TFDQuery);
procedure Execute; override;
function GetQuery: TFDQuery;
public
property Query read GetQuery;
end;
TForm1 = class(TForm)
FDQuery1: TFDQuery; //Do not connect the FDQuery1 to anything!
DataSource1: TDataSource;
DBGrid1: TDBGrid;
private
FOnThreadDone: TMyEventHandler;
FMyThread: TMyThread;
procedure DoThreadDone;
procedure ThreadDone;
public
property OnThreadDone: TMyEventHandler read FOnThreadDone write FOnThreadDone;
....
implementation
constructor TMyThread.Create(DoneEvent: TMyEvent; Query: TFDQuery);
begin
inherited Create(true);
FDoneEvent:= DoneEvent;
FQuery:= Query;
Start;
end;
procedure TMyThread.Execute;
begin
//Do whatever with the query
//when done do:
FDone:= true;
Synchonize(Form1.DoThreadDone);
end;
function TMyThread.GetQuery: TFDQuery;
begin
if not Done then Result:= nil else Result:= FQuery;
end;
procedure TForm1.DoThreadDone;
begin
if Assigned(FOnThreadDone) then FOnThreadDone(Self);
end;
procedure TForm1.ThreadDone(Sender: TObject);
begin
ShowMessage('Query is done');
//Now you can display the result of the query, by wiring it
//to a dataset.
MyDataSource1.Dataset:= FMyThread.Query;
FMyThread.Free;
end;
procedure TForm1.StartTheQuery;
begin
OnThreadDone:= Self.ThreadDone;
FMyThread:= TMyThread.Create(OnThreadDone, FDQuery1);
end;
Now the query will run in the background and signal your event handler when it is done. Meanwhile you can do all the mousing around and user interaction you want without having to worry. Note that you cannot use FDQuery1 at all whilst the thread is using it, and you cannot have FDQuery1 wired to a DataSource whilst it's the thread is running with it.
Leave it unwired and wire it in the ThreadDone event handler as shown.

DBX Error: Driver could not be properly initialized when use OmniThreadLibrary (but ok otherwise)

I need to make a long running task in the background. I'm using OmniThreadLibrary hopping this could help me.
I use dbexpress+mssql driver. I can connect ok when is in the main thread, but get:
Project Project1.exe raised exception class TDBXError with message
'DBX Error: Driver could not be properly initialized. Client library
may be missing, not installed properly, of the wrong version, or the
driver may be missing from the system path.'.
The connections are created in each thread, not shared datamodule:
type
TdbManager = class(TObject)
private
{ private declarations }
FCon: TSQLConnection;
public
{ public declarations }
procedure Open(Driver:String; aparams:TStringList);overload;
procedure Close;
constructor Create;
destructor Destroy;override;
end;
constructor TdbManager.Create;
begin
inherited Create;
FCon := TSQLConnection.Create(nil);
end;
procedure TdbManager.Open(Driver: String; aparams: TStringList);
var
i: Integer;
key:string;
begin
FCon.DriverName := Driver;
for i := 0 to params.Count - 1 do
begin
key := params.Names[i];
FCon.Params.Values[key] := params.Values[key];
end;
LogMsg('Open DB '+ Driver + ': ' + FHost + '\' + FDatabase);
FCon.Open;
LogMsg('Done.');
end;
And the background task is executed:
procedure TBackupPlan.OnScheduleTrigger(Sender: TScheduledEvent);
begin
Parallel.Async(procedure
begin
ExecuteDataTask( Sender.Name );
end);
end;
procedure TBackupPlan.ExecuteDataTask(const Name: String);
var
db:TdbManager;
begin
db := nil;
db := TSqlServerManager.Create;
db.Open(self.Driver, options);
result := db;
end;
If I execute this directly, open ok. If I use Parallel.Async then get the error. What is happend here?
I found information about this here:
http://docwiki.embarcadero.com/RADStudio/en/DbExpress_Database_Specific_Information
MSSQL Driver Requires Calls to CoInitialize and CoUninitialize for Console Applications and Worker Threads
The MSSQL driver does not call CoInitialize or CoUninitialize. Earlier
versions of the MSSQL driver, which is a COM driver, called
CoInitialize and CoUninitialize directly, which is not a good
practice. VCL applications take care of these calls for you, so VCL
applications do not require calling CoInitialize and CoUninitialize.
However, applications using the MSSQL driver in console applications
or in worker threads need to call CoInitialize/CoUninitialize. If this
call is not made, you will see the following error message: "DBX Error: Driver could not be properly initialized. Client library may be
missing, not installed properly, of the wrong version, or the driver
may be missing from the system path." For help on CoInitialize, see
the CoInitialize Function on MSDN.

Create event and share variables

I am using Delphi 2007 and threads.
My problem (sorry, i'll try to explain better):
1) I created a file "utilities.pas" where i have the function i use more.
2) I created a new program, in this program i have one thread
3) in the execute method of the thread i call one function in my file "utilities.pas".
this function connects to an ftp using clever components (tclftp). This components logs the server responce in a dedicated event. What i would like to do is to save the log in a stringlist and then send the stringlist back to the calling thread.
This is part of the file "utilities.pas":
// I created TEventHandlers because it's the only way to assign the event runtime
// without having a class
type
TEventHandlers = class
procedure clFtp1SendCommand(Sender: TObject; const AText: string);
end;
var EvHandler: TEventHandlers;
// this is the porcedure called from the thread. i want to send the stringlist
// back to it containing the ftp log
procedure Test(VAR slMain: tStringlist);
var cFTP: TclFtp;
begin
cFTP := TclFtp.Create(nil);
cFTP.Server := 'XXX';
cFTP.UserName := 'XXX';
cFTP.Password := 'XXX';
cFTP.OnSendCommand := EvHandler.clFtp1SendCommand;
// i connect to the ftp
cFTP.Open;
FreeAndNil(cFTP);
end;
procedure TEventHandlers.clFtp1SendCommand(Sender: TObject; const AText: string);
begin
// here the component (cftp) sends me back the answer from the server.
// i am logging it
// HERE IT'S THE PROBLEM:
// I can't reach slMain from here.....
slmain.add(Atext);
end;
this is the calling thread:
procedure TCalcThread.Execute;
var slMain: tstringlist;
begin
inherited;
slmain := tstringlist.create(nil);
Test(slmain);
if slMain.count > 0 then
slMain.savetofile('c:\a.txt');
// i won't free the list box now, but in the thread terminated.
end;
this is the main program:
procedure TfMain.ThreadTerminated(Sender: TObject);
Var ExThread: TCalcThread;
begin
ExThread := (Sender as TCalcThread);
if ExThread.slMain.Count > 0 then
ExThread.slMain.SaveToFile('LOG\Errori.log');
freeandnil(slMain);
end;
Please can anybody help me in solving this? I really don't know what to do.
I hope now it more clear.
p.s. thanks for all the answer..
Another approach would be to have your thread object have its own instance of the stringlist and its own cFTP. If you need to have one "master thread" that everything writes to (perhaps for a summary of what each thread accomplished), use this class:
TThreadStringList by Tilo Eckert
http://www.swissdelphicenter.ch/torry/showcode.php?id=2167
I think one (BAD) approach would be to create a pool of components in the main thread or at design time, and assign one to each thread. i.e. 5 instances of cFTP, 5 stringlists, 5 threads.
Update: Martin James points out why this is a terrible idea, and I agree. So don't do this. Post stays as a deterrent.
Intercept the event within the thread class, and fire an own typed event from within that handler. Synchronize this call! And try to prevent the global variable. All this as follows:
type
TFtpSendCommandEvent = procedure(Mail: TStrings; const AText: String) of object;
TMyThread = class(TThread)
private
FclFtp: TclFtp;
FslMail: TStrings;
FOnFtpSendCommand: TFtpSendCommandEvent;
FText: String;
procedure clFtpSendCommand(Sender: TObject; const AText: String);
procedure DoFtpSendCommand;
protected
procedure Execute; override;
public
// You could add this property as parameter to the constructor to prevent the
// need to assign it separately
property OnFtpSendCommand: TFtpSendCommandEvent read FOnFtpSendCommand
write FOnFtpSendCommand;
end;
// If you dont want to make this a property or private field of the thread class:
var
EvHandler: TFtpSendCommandEvent;
{ TMyThread }
procedure TMyThread.clFtpSendCommand(Sender: TObject; const AText: string);
begin
// Store the AText parameter temporarily in a private field: Synchronize only
// takes a parameterless method
FText := AText;
Synchronize(DoFtpSendCommand);
end;
procedure TMyThread.DoFtpSendCommand;
begin
if Assigned(FOnFtpSendCommand) then
FOnFtpSendCommand(FslMail, FText);
// Or, if you really like to use that global variable:
if Assigned(EvHandler) then
EvHandler(FslMail, FText);
end;
procedure TMyThread.Execute;
begin
...
FclFtp := TclFtp.Create(nil);
FslMail := TStringList.Create(nil);
try
FclFtp.Server := 'XXX';
FclFtp.UserName := 'XXX';
FclFtp.Password := 'XXX';
FclFtp.OnSendCommand := clFtpSendCommand;
FclFtp.Open;
finally
FreeAndNil(FclFtp);
FreeAndNil(FslMail);
end;
...
end;

Resources