Application is built around a stand-alone application using in-process COM servers as plugins. A communication plugin is interfaced with a C# layer through interop and after calling an C# async connection method (itself using C# thread) "GetFunctionPointerForDelegate" is used to notify status changed to Delphi COM server.
Stand-alone application (Delphi .exe)
In-process COM server (Delphi .dll/regsvr32)
C# interop communication layer (C# .dll assemblies/regasm)
It appears distinct threadIDs are callbacking Delphi COM server through callback pointers after logging threadID received by callback methods. As these callbacks are received asynchronously (afaik), i was wondering if it needs to :
a) process them directly when fired (in calling thread context?) :
Application.mainthreadID
- COM Server threadId
- callback (threadId=1) : callback processing...
- callback (threadId=2) : callback processing...
b) dispatch these callbacks to COM server mainthread (to process them in COM server thread context) :
Application.mainthreadID
- COM Server threadId
- custom dynamic "WndProc"
- callback (threadId=1) : Dispatch/PostMessage...
- callback (threadId=2) : Dispatch/PostMessage...
In order to dispatch these C# callbacks to COM server mainthread, a window subclassing is dynamically created (within COM server) with a custom "WndProc" receiving custom WM_XXX dispatched messages :
TFooCOMServer.CallbackEntryPoint1(...)
PostMessage(hWnd, WM_CUSTOM_CALLBACK1, wParam, lParam);
TFooCOMServer.CustonWndProc( Msg : TMesssage);
...
Case Msg.message of
WM_CUSTOM_CALLBACK1 : OnCustomCallBack(param1,param2);
...
end;
My questions about this COM/multi-thread application pattern are :
Does this dispatcher makes sens ? Does it worth the while to synchronize through "WndProc" process message callbacks processing ?
If yes, how callback method parameters should be marshalled from callback entry point to "WndProc" dispatcher target ? In case of interface parameter type (dispatched through PostMessage "lParam or wParam" integer), how to dispatch safely this interface pointer and what about interface refcounting/life cycle, should i increment or marshall (to "WndProc" thread) these interface pointers ?
Thanks in advance for your help on this subject.
Related
Let me give you a bigger picture of the problem... I am designing a ROS2-based system with multiple ROS2 nodes each containing a wrapper part (ROS2 layer) and driver/module part where my low-level logic is implemented. The wrapper part is using some ROS2-specific communication mechanisms (topics, services, actions...) to exchange the data/commands between the nodes.
Now, one of the nodes in the system should establish an MQTT connection with the Google Cloud Platform, keep the connectivity alive and allow data exchange between the Cloud and ROS2 system. For that purpose, I am using iot-device-sdk-embedded-c SDK from Google.
It has iotc_connect() blocking function for establishing and keeping connection with the Cloud so the challenge I am facing with is to simultaneously keep the ROS2 node spinning while keeping MQTT connectivity alive.
My idea was to launch a thread from ROS2 wrapper that will be used for establishing/keeping MQTT connectivity and use a callback function as an argument for the thread function that will enable me to forward the data received from the Cloud ithin the thread directly to ROS2 layer. Launching a separate thread for handling connectivity and data exchange would enable my ROS2 node to properly spin and rest synchronized with the rest of the ROS2 system.
ROS2_Wrapper.cpp
thread mqtt_thread(MqttConnHandler::ConnectToMqttServer, &MqttThreadCallback);
mqtt_thread.detach();
...
void MqttThreadCallback(void* data, size_t size){
}
MqttThreadCallback() should be called every time I receive the command/config data from the Cloud.
However, I am not sure how can I fire the callback function within the thread because I have two layers of nested callbacks within the thread:
my_thread.cpp
ConnectToMqttServer(void (*MqttThreadCallback)(void*, size_t)){
...
iotc_connect(...,&OnConnectionStateChanged);
...
}
OnConnectionStateChanged(...){
...
case IOTC_CONNECTION_STATE_OPENED:
iotc_subscribe(...,&iotc_mqttlogic_subscribe_callback,...);
...
}
iotc_mqttlogic_subscribe_callback(...){
//The place where data from the Cloud are received
}
iotc_connect() contains OnConnectionStateChanged() callback from where iotc_subscribe() function is called at the moment connection is established. iotc_subscribe() contains iotc_mqttlogic_subscribe_callback() where data from the Cloud are received.
I am not sure how can I mount the data from iotc_mqttlogic_subscribe_callback() up to the thread caller. Do you have any suggestions? Perhaps using the threads is not the best approach?
Usually C libraries provide an optional additional argument called user_data for this purpose:
extern iotc_state_t iotc_subscribe(iotc_context_handle_t iotc_h,
const char* topic, const iotc_mqtt_qos_t qos,
iotc_user_subscription_callback_t* callback,
void* user_data);
That way you can cast your callback function pointer to void when calling subscribe and catch it as argument in the iotc_mqttlogic_subscribe_callback function call. Where you should recast the data back to the function pointer type and use it.
In addition, you may find yourself in need to pass more data to the callback (mutex to protect the data, loggers from higher level code...). In that case, the best practice is to wrap all this info in a new class of your choice and pass a pointer to the instance in the callback.
We created a Qt HTTP server derived from QTcpServer.
Each incoming client connection is handled in a new thread like this:
void WebClientThread::run()
{
// Configure the web client socket
m_socket = new QTcpSocket();
connect(m_socket, SIGNAL(disconnected()), this, SLOT(disconnected()));
connect(m_socket, SIGNAL (error(QAbstractSocket::SocketError)), this, SLOT(socketError(QAbstractSocket::SocketError)));
// Create the actual web client = worker
WebClient client(m_socket, m_configuration, m_pEventConnection, m_pThumbnailStreams, m_server, m_macAddress, 0 );
// Thread event loop
exec();
m_pLog->LOG(L_INFO, "Webclient thread finished");
}
//
// Client disconnect
//
void WebClientThread::disconnected()
{
m_socket->deleteLater();
exit(0);
}
This code works, but we experienced application crashes when it was executed while the NTP connection of our device kicked in and the system time was corrected from the epoch 01/01/1970 to the current time.
The crash could also be reproduced when running the application and meanwhile changing the system time from a script.
The application runs fine - even when the system time changes on the fly like this:
void WebClientThread::run()
{
// Configure the web client socket
m_socket = new QTcpSocket();
connect(m_socket, SIGNAL(disconnected()), this, SLOT(disconnected()));
connect(m_socket, SIGNAL (error(QAbstractSocket::SocketError)), this, SLOT(socketError(QAbstractSocket::SocketError)));
// Create the actual web client = worker
WebClient client(m_socket, m_configuration, m_pEventConnection, m_pThumbnailStreams, m_server, m_macAddress, 0 );
// Make this thread a loop,
exec();
delete m_socket;
m_pLog->LOG(L_INFO, "Webclient thread finished");
}
//=======================================================================
//
// Client disconnect
//
void WebClientThread::disconnected()
{
exit(0);
}
Why would deleteLater() crash the application when the system time is shifted ?
Additional information:
OS = embedded linux 3.0.0. Qt = 4.8
The socket is a connection between our Qt web server application and the front end server = lighttpd. Could it be that lighttpd closes the socket when the system time shifts 47 years and the request is still being processed by our web server?
I could reproduce it by sending requests to the server while in parallel running a script that sets date to 1980, 1990 and 2000. It changes once a second.
This smells of wrong use of Qt threads. I suggest you do not subclass QThread, if you call exec() from its run() method, because it's just too easy to do things wrong way if you do that.
See for example https://wiki.qt.io/QThreads_general_usage to see how to set up a worker QObject for a QThread, but the gist of it is, create subclass of QObject and put your code there. Then move an instance of that to a QThread instance, and connect signals and slots to make things happen.
Another things, you normally shouldn't use threads for Qt Networking, like QTcpSocket. Qt is event based and asynchronous, and as long as you just use signals and slots and never block in your slot methods, there is no need for threads, they only complicate things for no benefit. Only if you have time-consuming calculations, or if your program truly needs to utilize multiple CPU cores to achieve good enough performance, only then look into multithreading.
I wish to be able to post messages from the main application of a DataSnap automation server to the RemoteDataModule instances created to service clients.
I understand from other forum entires, such as:
Delphi Multi-Threading Message Loop
That messages can be handled in threads without the need to create a Windows handle for the purpose; messages posted with PostThreadMessage.
However the DataSnap TComponentFactory creates a TApartmentThread instance for each RemoteDataModule, and it's Execute method already processes messages:
while GetMessage(msg, 0, 0, 0) do
begin
DispatchMessage(msg);
Unk._AddRef;
if Unk._Release = 1 then break;
end;
I imagine to get this to process any custom message of mine, would mean re-writing the TApartmentThread Execute method.
I have confirmed that creating a window handle to process messages in a RemoteDataModule instance works using the technique described here:
http://delphi.about.com/od/windowsshellapi/a/receive-windows-messages-in-custom-delphi-class-nonwindowed-control.htm
This uses AllocateHWnd, which can be made thread-safe thanks to the work of others:
How can I make AllocateHwnd threadsafe?
Although this presents a solution, I would like to ask; is there a recommended alternative approach?
It would appear that the only other means of processing custom messages in a TApartmentThread instance would be to rewrite the Execute method.
My web server is based on Delphi / DataSnap.
When a server method raises an exception, how can I handle it?
I tried Application.OnException but it does not work because each connection is a new thread and it only works for the IDE thread.
I want to manage them for adding the data in a log file.
UPDATE:
As #Remy Lebeau says, I do not have control aver the creation of the thread. It is a DataSnap based application that works as Web Api server. A TDSServerClass components instances my TServerMethods class. Then, when a web connection arrives, DataSnap creates a new thread and call to my TServerMethods methods. If within the method, a exception is raised, then it is lost because each thread has its stack. With Application.OnException, I only can catch unhandheld exceptions from the IDE thread. What I want to do is to log any handheld exception in a log file.
From the System.Classes.TThread.FatalException documentation:
If the Execute method raises an exception that is not caught and handled within that method, the thread terminates and sets FatalException to the exception object for that exception. Applications can check FatalException from an OnTerminate event handler to determine whether the thread terminated due to an exception.
If an exception occurs in the Execute method of a TThread object its FatalException property is assigned and the thread terminates.
You can check if an exception occurred in a thread implementing its OnTerminate event:
procedure TForm1.YourThreadTerminate(Sender: TObject);
var
ex: Exception;
begin
ex := Exception(TYourThread(Sender).FatalException);
if Assigned(ex) then
//an exception has occurred: your code here
end;
This is how you can assign the event to you thread:
var
yourThread: TYourThread;
begin
yourThread := TYourThread.Create(True);
yourThread.OnTerminate := YourThreadTerminate;
yourThread.Start;
end;
You may need to hook into the exception handling mechanism itself - JCL has code that does that, for example, or you could use third party libraries like madExcept, EurekaLog or others. They also collect stack traces and other info useful to understand what lead to an exception.
I used JCL some time ago in a Datasnap DCOM server to log unhandled exception to a custom event log. If you're going to write to a file, beware of concurrent access, because more than one thread could try to write to it. Also Event Tracing for Windows (ETW) may be a a better, although more complex way, to log to a file for diagnostic purposes.
Anyway, in this kind of applications, it's better to handle "expected" exceptions, and never let them propagate up to a generic handler. Of course, being able to trap and log "unexpected" ones is useful.
I am writing a web service which has to be able to reply to multiple http requests.
From what I understand, I will need to deal with HttpListener.
What is the best method to receive a http request(or better, multiple http requests), translate it and send the results back to the caller? How safe is to use HttpListeners on threads?
Thanks
You typically set up a main thread that accepts connections and passes the request to be handled by either a new thread or a free thread in a thread pool. I'd say you're on the right track though.
You're looking for something similar to:
while (boolProcessRequests)
{
HttpListenerContext context = null;
// this line blocks until a new request arrives
context = listener.GetContext();
Thread T = new Thread((new YourRequestProcessorClass(context)).ExecuteRequest);
T.Start();
}
Edit Detailed Description If you don't have access to a web-server and need to roll your own web-service, you would use the following structure:
One main thread that accepts connections/requests and as soon as they arrive, it passes the connection to a free threat to process. Sort of like the Hostess at a restaurant that passes you to a Waiter/Waitress who will process your request.
In this case, the Hostess (main thread) has a loop:
- Wait at the door for new arrivals
- Find a free table and seat the patrons there and call the waiter to process the request.
- Go back to the door and wait.
In the code above, the requests are packaged inside the HttpListernContext object. Once they arrive, the main thread creates a new thread and a new RequestProcessor class that is initialized with the request data (context). The RequsetProcessor then uses the Response object inside the context object to respond to the request. Obviously you need to create the YourRequestProcessorClass and function like ExecuteRequest to be run by the thread.
I'm not sure what platform you're on, but you can see a .Net example for threading here and for httplistener here.